[en] Combined Algorithm Selection and Hyperparameter Optimization (CASH) to automate the configuration of resampling strategies and classifiers for imbalanced classification tasks. Existing AutoCIL methods focus solely on single-objective optimization. However, real-world applications often involve multiple, conflicting objectives—such as predictive performance and computational cost—that must be jointly optimized. Ignoring such trade-offs limits the adaptability and practicality of current methods. In this work, we propose a
novel approach called AutoCIL-FMOBO (AutoCIL via Few-shot Multi-Objective Bayesian Optimization). Specifically, we design meta-learned deep kernel Gaussian process surrogates trained on a meta-dataset constructed from pre-evaluated results obtained by running configurations in the search space on classimbalanced datasets. Then, these surrogate models with prior optimization knowledge are combined with the Expected Hypervolume Improvement (EHVI) acquisition function in a Bayesian optimization framework to efficiently discover Pareto-optimal configurations for the target task, which enables AutoCIL-FMOBO to jointly optimize key components, such as resampling methods, classifiers, and their hyperparameters, under a multi-objective setting. Experimental results on 15 real-world class-imbalanced datasets demonstrate that our approach outperforms baselines in both effectiveness and sample efficiency, while maintaining generalization across tasks and achieving competitive performance under a multi-objective setting.
Disciplines :
Sciences informatiques
Auteur, co-auteur :
Wang, Zhaoyang
Wang, Shuo
Ernst, Damien ; Université de Liège - ULiège > Département d'électricité, électronique et informatique (Institut Montefiore) > Smart grids
Xiao, Chenguang
Langue du document :
Anglais
Titre :
Automated Class Imbalance Learning via Few-Shot Multi-Objective Bayesian Optimization With Deep Kernel Gaussian Processes
Date de publication/diffusion :
juillet 2025
Titre du périodique :
IEEE Access
ISSN :
2169-3536
Maison d'édition :
Institute of Electrical and Electronics Engineers, Etats-Unis - New Jersey
Z. Ren, T. Lin, K. Feng, Y. Zhu, Z. Liu, and K. Yan, ‘‘A systematic review on imbalanced learning methods in intelligent fault diagnosis,’’ IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1–35, 2023.
M. M. Rahman and D. N. Davis, ‘‘Addressing the class imbalance problem in medical datasets,’’ International Journal of Machine Learning and Computing, vol. 3, no. 2, p. 224, 2013.
S. Makki, Z. Assaghir, Y. Taher, R. Haque, M.-S. Hacid, and H. Zeineddine, ‘‘An experimental study with imbalanced classification approaches for credit card fraud detection,’’ Ieee Access, vol. 7, pp. 93 010–93 022, 2019.
A. Abdelkhalek and M. Mashaly, ‘‘Addressing the class imbalance problem in network intrusion detection systems using data resampling and deep learning,’’ The journal of Supercomputing, vol. 79, no. 10, pp. 10 611–10 644, 2023.
S. Wang, L. L. Minku, and X. Yao, ‘‘A systematic study of online class imbalance learning with concept drift,’’ IEEE transactions on neural networks and learning systems, vol. 29, no. 10, pp. 4802–4821, 2018.
V. Ganganwar, ‘‘An overview of classification algorithms for imbalanced datasets,’’ International Journal of Emerging Technology and Advanced Engineering, vol. 2, no. 4, pp. 42–47, 2012.
D. A. Nguyen, J. Kong, H. Wang, S. Menzel, B. Sendhoff, A. V. Kononova, and T. Bäck, ‘‘Improved automated cash optimization with tree parzen estimators for class imbalance problems,’’ in 2021 IEEE 8th international conference on data science and advanced analytics (DSAA). IEEE, 2021, pp. 1–9.
Z. Wang and S. Wang, ‘‘Zero-shot automated class imbalanced learning,’’ in International Conference on Pattern Recognition. Springer, 2024, pp. 140–155.
N. Moniz and V. Cerqueira, ‘‘Automated imbalanced classification via meta-learning,’’ Expert Systems with Applications, vol. 178, p. 115011, 2021.
P. Singh and J. Vanschoren, ‘‘Automated imbalanced learning,’’ arXiv preprint arXiv:2211.00376, 2022.
F. Hutter, L. Kotthoff, and J. Vanschoren, Automated machine learning: methods, systems, challenges. Springer Nature, 2019.
K. Wang, Q. Xue, and J. J. Lu, ‘‘Risky driver recognition with class imbalance data and automated machine learning framework,’’ International journal of environmental research and public health, vol. 18, no. 14, p. 7534, 2021.
Z. Wang, S. Wang, and D. Ernst, ‘‘Automated class imbalance learning via few-shot bayesian optimization with meta-learned deep kernel surrogates,’’ in 2025 International Joint Conference on Neural Networks (IJCNN). IEEE, 2025.
B. Bischl, M. Binder, M. Lang, T. Pielok, J. Richter, S. Coors, J. Thomas, T. Ullmann, M. Becker, A.-L. Boulesteix et al., ‘‘Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges,’’ Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 13, no. 2, p. e1484, 2023.
S. A. Renganathan and K. E. Carlson, ‘‘qpots: Efficient batch multiobjective bayesian optimization via pareto optimal thompson sampling,’’ arXiv preprint arXiv:2310.15788, 2023.
A. Maraval, M. Zimmer, A. Grosnit, and H. Bou Ammar, ‘‘End-to-end meta-bayesian optimisation with transformer neural processes,’’ Advances in Neural Information Processing Systems, vol. 36, pp. 11 246–11 260, 2023.
S. Pineda Arango and J. Grabocka, ‘‘Deep pipeline embeddings for automl,’’ in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 1907–1919.
F. Karl, T. Pielok, J. Moosbauer, F. Pfisterer, S. Coors, M. Binder, L. Schneider, J. Thomas, J. Richter, M. Lang et al., ‘‘Multi-objective hyperparameter optimization in machine learning—an overview,’’ ACM Transactions on Evolutionary Learning and Optimization, vol. 3, no. 4, pp. 1–50, 2023.
A. Morales-Hernández, I. Van Nieuwenhuyse, and S. Rojas Gonzalez, ‘‘A survey on multi-objective hyperparameter optimization algorithms for machine learning,’’ Artificial Intelligence Review, vol. 56, no. 8, pp. 8043–8093, 2023.
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, ‘‘Mobilenets: Efficient convolutional neural networks for mobile vision applications,’’ arXiv preprint arXiv:1704.04861, 2017.
T. Bai, Y. Li, Y. Shen, X. Zhang, W. Zhang, and B. Cui, ‘‘Transfer learning for bayesian optimization: A survey,’’ arXiv preprint arXiv:2302.05927, 2023.
H. He and E. A. Garcia, ‘‘Learning from imbalanced data,’’ IEEE Transactions on knowledge and data engineering, vol. 21, no. 9, pp. 1263–1284, 2009.
G. M. Weiss, ‘‘Mining with rarity: a unifying framework,’’ ACM Sigkdd Explorations Newsletter, vol. 6, no. 1, pp. 7–19, 2004.
S. Wang and X. Yao, ‘‘Using class imbalance learning for software defect prediction,’’ IEEE Transactions on Reliability, vol. 62, no. 2, pp. 434–443, 2013.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, ‘‘Smote: synthetic minority over-sampling technique,’’ Journal of artificial intelligence research, vol. 16, pp. 321–357, 2002.
I. Mani and I. Zhang, ‘‘knn approach to unbalanced data distributions: a case study involving information extraction,’’ in Proceedings of workshop on learning from imbalanced datasets, vol. 126, no. 1. ICML, 2003, pp. 1–7.
M. Lamari, N. Azizi, N. E. Hammami, A. Boukhamla, S. Cheriguene, N. Dendani, and N. E. Benzebouchi, ‘‘Smote–enn-based data sampling and improved dynamic ensemble selection for imbalanced medical data classification,’’ in Advances on Smart and Soft Computing: Proceedings of ICACIn 2020. Springer, 2021, pp. 37–49.
Z.-H. Zhou and X.-Y. Liu, ‘‘Training cost-sensitive neural networks with methods addressing the class imbalance problem,’’ IEEE Transactions on knowledge and data engineering, vol. 18, no. 1, pp. 63–77, 2005.
V. López, A. Fernández, J. G. Moreno-Torres, and F. Herrera, ‘‘Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. open problems on intrinsic data characteristics,’’ Expert Systems with Applications, vol. 39, no. 7, pp. 6585–6608, 2012.
M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter, ‘‘Efficient and robust automated machine learning,’’ Advances in neural information processing systems, vol. 28, 2015.
P. M. Vieira and F. Rodrigues, ‘‘An automated approach for binary classification on imbalanced data,’’ Knowledge and Information Systems, pp. 1–21, 2024.
P. I. Frazier, ‘‘A tutorial on bayesian optimization,’’ arXiv preprint arXiv:1807.02811, 2018.
C. E. Rasmussen, ‘‘Gaussian processes in machine learning,’’ in Summer school on machine learning. Springer, 2003, pp. 63–71.
M. Seeger, ‘‘Gaussian processes for machine learning,’’ International journal of neural systems, vol. 14, no. 02, pp. 69–106, 2004.
N. Gunantara, ‘‘A review of multi-objective optimization: Methods and its applications,’’ Cogent Engineering, vol. 5, no. 1, p. 1502242, 2018.
K. Deb, K. Sindhya, and J. Hakanen, ‘‘Multi-objective optimization,’’ in Decision sciences. CRC Press, 2016, pp. 161–200.
B. Paria, K. Kandasamy, and B. Póczos, ‘‘A flexible framework for multiobjective bayesian optimization using random scalarizations,’’ in Uncertainty in Artificial Intelligence. PMLR, 2020, pp. 766–776.
M. Laumanns and J. Ocenasek, ‘‘Bayesian optimization algorithms for multi-objective optimization,’’ in International Conference on Parallel Problem Solving from Nature. Springer, 2002, pp. 298–307.
A. Rivolli, L. P. Garcia, C. Soares, J. Vanschoren, and A. C. de Carvalho, ‘‘Meta-features for meta-learning,’’ Knowledge-Based Systems, vol. 240, p. 108101, 2022.
J. Vanschoren, ‘‘Meta-learning: A survey,’’ arXiv preprint arXiv:1810.03548, 2018.
E. Alcobaça, F. Siqueira, A. Rivolli, L. P. Garcia, J. T. Oliva, and A. C. De Carvalho, ‘‘Mfe: Towards reproducible meta-feature extraction,’’ Journal of Machine Learning Research, vol. 21, no. 111, pp. 1–5, 2020.
E. Öztürk, F. Ferreira, H. Jomaa, L. Schmidt-Thieme, J. Grabocka, and F. Hutter, ‘‘Zero-shot automl with pretrained models,’’ in International Conference on Machine Learning. PMLR, 2022, pp. 17 138–17 155.
S. P. Arango, F. Ferreira, A. Kadra, F. Hutter, and J. Grabocka, ‘‘Quick-tune: Quickly learning which pretrained model to finetune and how,’’ arXiv preprint arXiv:2306.03828, 2023.
A. G. Wilson, Z. Hu, R. Salakhutdinov, and E. P. Xing, ‘‘Deep kernel learning,’’ in Artificial intelligence and statistics. PMLR, 2016, pp. 370–378.
X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T.-S. Chua, ‘‘Neural collaborative filtering,’’ in Proceedings of the 26th international conference on world wide web, 2017, pp. 173–182.
G. Shala, T. Elsken, F. Hutter, and J. Grabocka, ‘‘Transfer nas with meta-learned bayesian surrogates,’’ in The Eleventh International Conference on Learning Representations, 2023.
M. G. Genton, ‘‘Classes of kernels for machine learning: a statistics perspective,’’ Journal of machine learning research, vol. 2, no. Dec, pp. 299–312, 2001.
M. Wistuba and J. Grabocka, ‘‘Few-shot bayesian optimization with deep kernel surrogates,’’ arXiv preprint arXiv:2101.07667, 2021.
E. Zitzler and L. Thiele, ‘‘Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach,’’ IEEE transactions on Evolutionary Computation, vol. 3, no. 4, pp. 257–271, 1999.
E. Zitzler, D. Brockhoff, and L. Thiele, ‘‘The hypervolume indicator revisited: On the design of pareto-compliant indicators via weighted integration,’’ in Evolutionary Multi-Criterion Optimization: 4th International Conference, EMO 2007, Matsushima, Japan, March 5-8, 2007. Proceedings 4. Springer, 2007, pp. 862–876.
S. Daulton, M. Balandat, and E. Bakshy, ‘‘Differentiable expected hypervolume improvement for parallel multi-objective bayesian optimization,’’ Advances in Neural Information Processing Systems, vol. 33, pp. 9851–9864, 2020.
M. T. Emmerich, K. C. Giannakoglou, and B. Naujoks, ‘‘Single-and multiobjective evolutionary optimization assisted by gaussian random field metamodels,’’ IEEE Transactions on Evolutionary Computation, vol. 10, no. 4, pp. 421–439, 2006.
J. Alcalá-Fdez, L. Sanchez, S. Garcia, M. J. del Jesus, S. Ventura, J. M. Garrell, J. Otero, C. Romero, J. Bacardit, V. M. Rivas et al., ‘‘Keel: a software tool to assess evolutionary algorithms for data mining problems,’’ Soft Computing, vol. 13, pp. 307–318, 2009.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg et al., ‘‘Scikit-learn: Machine learning in python,’’ the Journal of machine Learning research, vol. 12, pp. 2825–2830, 2011.
G. LemaÃŽtre, F. Nogueira, and C. K. Aridas, ‘‘Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning,’’ Journal of machine learning research, vol. 18, no. 17, pp. 1–5, 2017.
T. Chen and C. Guestrin, ‘‘Xgboost: A scalable tree boosting system,’’ in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785–794.
F. Biscani and D. Izzo, ‘‘A parallel global multiobjective framework for optimization: pagmo,’’ Journal of Open Source Software, vol. 5, no. 53, p. 2338, 2020.
J. Gardner, G. Pleiss, K. Q. Weinberger, D. Bindel, and A. G. Wilson, ‘‘Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration,’’ Advances in neural information processing systems, vol. 31, 2018.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., ‘‘Pytorch: An imperative style, high-performance deep learning library,’’ Advances in neural information processing systems, vol. 32, 2019.
