Ensemble learning by means of a multi-objective optimization design approach for dealing with imbalanced data sets

Abstract

Ensemble learning methods have already shown to be powerful techniques for creating classifiers. However, when dealing with real-world engineering problems, class imbalance is usually found. In such scenario, canonical machine learning algorithms may not present desirable solutions, and techniques for overcoming this problem must be used. In addition to using learning algorithms that alleviate the imbalance between classes, multi-objective optimization design (MOOD) approaches can be used to improve the prediction performance of ensembles of classifiers. This paper proposes a study of different MOOD approaches for ensemble learning. First, a taxonomy on multi-objective ensemble learning (MOEL) is proposed. In it, four types of existing approaches are defined: multi-objective ensemble member generation, multi-objective ensemble member selection, multi-objective ensemble member combination, and multi-objective ensemble member selection and combination. Additionally, new approaches can be derived by combining the previous ones, such as multi-objective ensemble member generation and selection, multi-objective ensemble member generation and combination and multi-objective ensemble member generation, selection and combination. With the given taxonomy, two experiments are conducted for comparing (1) the performance of the MOEL techniques for generating and aggregating base models on several imbalanced benchmark problems and (2) the performance of MOEL techniques against other machine learning techniques in a real-world imbalanced drinking-water quality anomaly detection problem. Finally, results indicate that MOOD is able to improve the predictive performance of existing ensemble learning techniques.