Improving a dynamic ensemble selection method based on oracle information

Abstract

This work evaluates some strategies to approximate the performance of a dynamic ensemble selection method to the oracle performance of its pool of weak classifiers. For this purpose, we evaluated different distance metrics in the K-nearest-oracles (KNORA) method, the use of statistics related to the class accuracy of each classifier in the pool and some additional information calculated by using a clustering process in the validation dataset. Moreover, different strategies are also evaluated to combine the results of the KNORA dynamic ensemble selection method with the results of its built-in K-nearest neighbour (KNN) used to define the neighbourhood of a test pattern during the ensemble creation. A strong experimental protocol based on more than 60,000 samples of handwriting digits extracted from NIST-SD19 was used to evaluate each strategy. The experiments have shown that the fusion of the KNORA results with the results of its built-in KNN is a very promising strategy.