A scalable dynamic ensemble selection using fuzzy hyperboxes

Abstract

Dynamic ensemble selection (DES) systems work by estimating the level of competence of each classifier from a pool of classifiers and selecting the most competent ones for the classification of a given test instance during inference time. The majority of dynamic ensemble selection (DES) methods evaluate the competence of classifiers using the K-Nearest Neighbors to the unknown query sample. However, KNN is very sensitive to local data distribution and needs to store all data in memory. Moreover, it performs several computations for each individual query sample. Thus, relying on the KNN technique hampers the use of DES approaches for large-scale problems and situations where data distributions are non-uniform. This article introduces a novel DES framework called FH-DES, which employs fuzzy hyperboxes to generate a competence map or incompetence map for each classifier. The competence map is generated from correctly classified samples to indicate the competence level of the classifier at each data point in the feature space, whereas the incompetence map, which shows regions where the classifier has low accuracy, is generated from misclassified samples. In this way, we can assess the competence or incompetence level of the classifier just by using the map without having to process previous samples. This feature results in a more accurate dynamic selection system with lower computational complexity compared to other dynamic selection methods. Moreover, we introduce several hyperbox expansion and contraction strategies that add incremental learning capability to the framework while keeping the computational cost low. Experimental results demonstrate that FH-DES achieves high classification accuracy with lower complexity than state-of-the-art dynamic selection methods. The source code for FH-DES is available at https://github.com/redavtalab/FH-DES.