Improved discrete artificial fish swarm algorithm combined with margin distance minimization for ensemble pruning

Abstract

Ensemble pruning aims to achieve a good result in classification using a smaller size of classifiers by finding the optimal sub-ensemble. Diversity and accuracy of classifiers are widely recognized to be two key factors for a successful ensemble. There is a tradeoff between the diversity and the accuracy of classifiers, which makes the ensemble perform the best. Existing ensemble pruning approaches always find the tradeoff using diversity measures or heuristic algorithms separately. Those pruning approaches based on diversity measures cannot exactly find the tradeoff; Those algorithms based on heuristic algorithms are not also to exhaustively search for it extracted from an initial pool of classifiers with a medium-scale or large-scale size. To address the issue, Improved Discrete Artificial Fish swarm algorithm combined with Margin distance minimization for Ensemble Pruning (IDAFMEP) is proposed using a combination of diversity measure and heuristic algorithm. First, the classifiers in a constructed initial pool are pre-pruned using Margin Distance Minimization (MDM), which can downsize the classifiers who perform badly, and markedly alleviate the computational complexity of ensemble pruning. Second, the final ensemble is efficiently achieved from the retaining classifiers after pre-pruning based on MDM using the proposed Improved discrete Artificial Fish Swarm Algorithm (IDAFSA). Experimental results on 29 datasets from the UCI Machine Learning Repository demonstrate that IDAFMEP can achieve better results than the original ensemble and other state-of-the-art pruning approaches, and that its validity and effectiveness. It provides a new research idea for ensemble pruning.