To content
Fakultät für Informatik

AutoML25 best paper award.

Plaque Best Paper Award International Conference on Automated Machine Learning 2025 for Lennart Schneider, Bernd Bischl and Matthias Feurer for the paper "Overtuning in Hyperparameter Optimization" © Timo Heiß​/​LMU Munich
Matthias's work on Overtuning in Hyperparameter Optimization received the AutoML25 best paper award.

Our paper “Overtuning in Hyperparameter Optimization” (with Lennart Schneider & Bernd Bischl) won the Best Paper Award at the 4th AutoML Conference.

In this paper, we formally define and study overtuning, a phenomenon where hyperparameter optimization improves validation scores but can hurt generalization to new data. In addition, we relate it to (meta-)overfitting and conduct a thorough survey of existing methods from the literature. Our large-scale analysis shows overtuning is more common than we expected (although most of the time it does not hurt too much), occasionally severe, and influenced by factors like resampling strategy, dataset size, and model class. A better understanding of overtuning will help the AutoML community design more reliable optimization and selection strategies, especially in small-data regimes where robustness is critical.