Meta-evolutionary ensembles

Abstract

Ensemble methods have shown the potential to improve on the performance of individual classifiers as long as the members of the ensemble are sufficiently diverse. In this paper we propose a meta-evolutionary approach in which both individual classifiers and ensembles adapt. Ensembles compete for member classifiers, and are rewarded based on their predictive performance. Individual classifiers also evolve, competing to correctly classify test points, and are given extra rewards for getting difficult points right. In this way we aim to build small-sized optimal ensembles rather than form large-sized ensembles of individually-optimized classifiers. We use neural networks as individual classifiers, and feature selection to promote the diversity among classifiers in the. same ensemble. Experimental results on 17 data sets suggest that our algorithms can generate ensembles that are more effective than single classifiers and traditional ensemble methods. We also use the evolutionary framework to explore the role of ensemble characteristics such as size and diversity in creating accurate ensembles.