Optimal selection of ensemble classifiers using particle swarm optimization and diversity measures

Abstract

Multiple Classifier Systems (MCSs) or ensemble methods have recently attracted genuine consideration due to their capacity to enhance prediction performance appreciably. Both experimental and theoretical investigations have demonstrated that MCSs can be useful in improving overall classification in the area of pattern recognition. Usually, such systems make an aggregate decision by combining the responses of several classifiers that form a committee. One of the most challenging issues in the classifier ensemble is selecting a suitable subset of base classifiers. Researchers have shown that MCSs produce an extensive amount of classifiers and, consequently, those classifiers have redundancy between each other. Since smaller ensembles are favored on account of storage and efficiency reasons, ensemble pruning is a critical step in the development of classifier ensembles. Two most common choices for selection criteria are combined performance and diversity measures. Nevertheless, there isn’t an agreement on this inquiry that which of them is better. In this paper, we utilized binary particle swarm optimization algorithm for pruning redundant base classifiers and acquiring an ideal ensemble from a given pool of classifiers. The proposed accuracy-diversity based pruning algorithm takes into account the accuracy of combined classifiers as well as the pairwise diversity amongst these classifiers. Comparing the performance of the proposed method using ten databases taken from the UCI Machine Learning Repository demonstrated that using diversity measures and combined performance simultaneously is appropriate for selecting a subset of classifiers.