An efficient ensemble learning method based on multi-objective feature selection

Abstract

Ensemble learning (EL) boosts model prediction performance across various domains through two main steps: generating individual classifiers (ICs) and combining them. Creating accurate and diverse ICs is crucial for a strong ensemble, while selecting the best ICs, known as ensemble selection (ES), is critical yet challenging due to the accuracy-diversity trade-off and the lack of agreed-upon diversity metrics. This paper introduces an EL strategy that uses multi-objective feature selection (MOFS) and a feature relevance-guided selection to tackle these challenges. Our approach uses a hybrid MOFS algorithm to produce accurate and diverse ICs, and then it employs a novel knowledge-based feature-relevance-guided metric for precise diversity assessment during ES. The ES issue is cast as an optimization problem, aiming to maximize both diversity and accuracy, and an efficient ES algorithm is developed to select optimal ICs. Extensive tests on public datasets and a real-world prediction task demonstrate the effectiveness of our method, especially in achieving high accuracy.