A Novel Greedy Randomized Dynamic Ensemble Selection Algorithm

Abstract

This work proposes a novel greedy randomized dynamic ensemble selection (DES) algorithm combined with path-relinking, variable neighborhood search (VNS) and greedy randomized adaptive search procedure (GRASP) algorithms, abbreviated DyPReVNsGraspEnS. Instead of simply selecting the classifiers with the best competence under certain criterions, DyPReVNsGraspEnS makes greedy randomized selection of appropriate classifiers to form the ensemble. It realizes random multi-start search, is capable to easily escape from the local optimums, and possesses a high probability to find the optimal subensemble. It effectively strengthens the link between iterations and solves the problem of no memory by integrating the path-relinking technique. Moreover, the algorithm properly extends the neighborhood with the help of VNS, having a higher competence of global optimization. Most important of all, DyPReVNsGraspEnS is developed based upon the framework of DES paradigm, therefore, it inherits numerous advantages of DES. Empirical investigations are conducted based on twelve benchmark datasets, including six real-world credit/finance datasets and six datasets drawn from other different fields with more than two classes. To clearly articulate the performance of each strategy in the proposed algorithm, experiments for the incorporation of different strategies are implemented separately on all the datasets. The experimental results demonstrate that, in comparison with other classical ensemble learning techniques, DyPReVNsGraspEnS achieves significantly superior generalization performance. And it can not only be used for regions like credit risk assessment, but is applicable to much broader application fields as well.