Ensemble pruning based on oblivious Chained Tabu Searches

Abstract

Ensemble pruning is an effective phase for ensemble methods to increase the predictive performance and to decrease computational overheads. This paper represents a novel ensemble pruning method named EPCTS(Ensemble Pruning via Chained Tabu Searches). EPCTS applies a chain of tabu searches for choosing models of ensemble progressively, until the best subset of them is found. These tabu searches are customized with the proposed strategy dubbed as Periodic Oblivion. This strategy revokes interdict of all tabu answers in the defined periods. EPCTS is compared with analogous ensemble pruning methods for pruning a balanced heterogeneous ensemble, focusing on 20 problems. Experimental results demonstrate that EPCTS leads to 2.65% averaged improvement in the accuracy of pruned ensemble, compared to others. Further, EPCTS leads to reduce computational overheads with dropping redundant and useless models from the ensemble. Moreover, one of the crucial issues in the ensemble learning field is making the decision to choose the type of base classifiers constructing desired ensemble. Considering the importance of the issue and due to the effectiveness of EPCTS in about 75% of datasets, EPCTS is suggested as a general tool for recognizing the type of base classifiers.