A novel multi-objective genetic algorithm based error correcting output codes

Abstract

Up to now, different genetic algorithm (GA) based error correcting output codes (ECOC) algorithms have been proposed by setting accuracy as the optimization objective. However, it was demonstrated that diversity among learners is of great significance to a robust ensemble. In this paper, we propose a multi-objective GA with setting accuracy and diversity as two objectives. To further promote diversity in an ensemble, a new individual structure is designed to accommodate heterogeneous dichotomizers. Three multi-objective ranking strategies are deployed to balance two objectives respectively. A novel genetic operator is designed to produce ECOC-compatible offspring in the evolutionary process, and a local improvement algorithm is designed to promote individuals’ fitness values. To verify the performance of our GA, a single objective ranking strategy and the design of homogeneous learner based GA are also adopted. Ten widely used ECOC algorithms and three famous ensemble algorithms are deployed for performance comparisons based on a set of the UCI data and microarray data sets. Results show that compared with other algorithms, our GA obtains higher performance in most cases due to the trade-off between performance and diversity. Besides, the accommodation of heterogeneous dichotomizers in an ensemble provides higher generalization ability compared with homogeneous ensembles.