Abstract
Hyperparameter optimization (HPO) for tree-based machine learning models remains computationally expensive due to exhaustive search and repeated cross-validation. This paper proposes a parallel e-fold Halving Grid Search (e-fold HGS) framework that integrates successive halving, adaptive e-fold cross-validation, and parallel execution to reduce evaluation cost. Experiments conducted with Random Forest and XGboost on six benchmark datasets show that e-fold HGS achieves statistically equivalent predictive performance to exhaustive Grid Search and original HGS, with all performance differences falling within overlapping 95% confidence intervals. At the same time, e-fold HGS delivers a substantial reduction in average execution time, achieving up to 37.5% and 46.64% in Random Forest and XGboost, respectively, with statistically significant improvements observed across the six datasets. These results demonstrate that e-fold HGS provides an efficient alternative for hyperparameter optimization without compromising predictive accuracy.
Paper Type
Poster
DOI
10.62036/ISD.2026.87
Parallel e-Fold Halving Grid Search Method for Hyperparameter Optimization of Tree-based Models
Hyperparameter optimization (HPO) for tree-based machine learning models remains computationally expensive due to exhaustive search and repeated cross-validation. This paper proposes a parallel e-fold Halving Grid Search (e-fold HGS) framework that integrates successive halving, adaptive e-fold cross-validation, and parallel execution to reduce evaluation cost. Experiments conducted with Random Forest and XGboost on six benchmark datasets show that e-fold HGS achieves statistically equivalent predictive performance to exhaustive Grid Search and original HGS, with all performance differences falling within overlapping 95% confidence intervals. At the same time, e-fold HGS delivers a substantial reduction in average execution time, achieving up to 37.5% and 46.64% in Random Forest and XGboost, respectively, with statistically significant improvements observed across the six datasets. These results demonstrate that e-fold HGS provides an efficient alternative for hyperparameter optimization without compromising predictive accuracy.
Recommended Citation
Mojeed, H.A. & Shaheed, K.(2026). Parallel e-Fold Halving Grid Search Method for Hyperparameter Optimization of Tree-based Models. In M. Valenta, B. Mannová, R. Pergl, A. Przybylek, M. Lang, H. Linger, C. Schneider, N. Iivari, & E. Insfran (Eds.), Making ISD Sustainable: Reloaded with AI and Automation (ISD2026 Proceedings). Prague, Czech Republic: Czech Technical University in Prague. ISBN: 978-80-01-07585-2. https://doi.org/10.62036/ISD.2026.87