Fuzzy-rough classifier ensemble selection

Abstract

Classifier ensembles constitute one of the main research directions in machine learning and data mining. Ensembles allow higher accuracy to be achieved which is otherwise often not achievable with a single classifier. A number of approaches have been adopted for constructing classifier ensembles and aggregate ensemble decisions. In most cases, these constructed ensembles contain redundant members that, if removed, may further increase ensemble diversity and produce better results. Smaller ensembles also relax the memory and storage requirements of an ensemble system, reducing its run time overhead while improving overall efficiency. In this paper, a new approach to classifier ensemble selection based on fuzzy rough feature selection and harmony search is proposed. By transforming the ensemble predictions into training samples, classifiers are treated as features. Harmony search is then used to select a minimal subset of such artificial features that maximises the fuzzy-rough dependency measure. The resulting technique is compared against the original ensemble and ensembles formed using random selection, under both single algorithm and mixed classifier ensemble environments.