Dynamic Feature Selection with Wrapper Model and Ensemble Approach based on Measures of Local Relevances and Group Diversity using Genetic Algorithm

Abstract

In the paper the novel feature selection method, using wrapper model and ensemble approach, is presented. In the proposed method features are selected dynamically, i.e. separately for each classified object. First, a set of identical one-feature classifiers using different single feature is created and next the ensemble of features (classifiers) is selected as a solution of optimization problem using genetic algorithm. As an optimality criterion, the sum of measures of features relevance and diversity of ensemble of features is adopted. Both measures are calculated using original concept of randomized reference classifier, which on average acts like classifier with evaluated feature. The performance of the proposed method was compared against six state-of-art feature selection methods using nine benchmark databases. The experimental results clearly show the effectiveness of the dynamic mode and ensemble approach in feature selection procedure.