Feature selection in high dimensional data: A specific preordonnances-based memetic algorithm

Abstract

In supervised learning scenarios, feature selection has been largely investigated in the literature because only a few features carry valuable information. This study introduces an algorithm for heterogeneous variable selection in the discrimination problem. The proposed algorithm, Specific Memetic Algorithm Preordonnances-based (SMAP), uses association techniques based on preordonnances theory and is a hybrid filter-wrapper algorithm to make full use of the benefits of each: The filter phase measures the relevance of features by their agreement with the target variable and discards those that disagree ; this leads to a reduction of the search space, while the wrapper phase measures the usefulness of subsets of features to identify the best one using a memetic algorithm based on preordonnances theory. The association is quantified by a coefficient measuring the concordance between two variables, even if one is numeric and the other is categorical (mixed). We propose a generalization of this coefficient measuring the concordance between several (more than two) heterogeneous variables. In this study, a new feature discrimination power measure combining the two coefficients is introduced to intensify and diversify the search taking a minimum amount of time. SMAP is empirically analyzed by comparing its performance to that of recently referred state-of-art approaches on seven datasets and on simulated ones using three different classifiers. The experimental results show the superiority of SMAP over comparative methods.