Region Purity-Based Local Feature Selection: A Multiobjective Perspective

Abstract

In contrast from the traditional feature selection, local feature selection (LFS) partitions the whole sample space and obtains the feature subset for each local region. However, most existing LFS algorithms lack a problem-specific objective function and instead simply apply the distance-like objective function, which limits their classification performance. In addition, obtaining a good LFS model is essentially a multi-objective optimization problem. Therefore, in this paper we propose a region purity-based LFS (RP-LFS) where, besides the proportion of the selected features and region-based distance metric, we design a novel objective function, region purity, from the perspective of combining local features with classifiers. To solve the RP-LFS, an improved non-dominated sorting genetic algorithm III is proposed. Specifically, a network-inspired crossover operator and a quick bit mutation are applied, which can improve the ability to search for better solutions. A regional feature sharing strategy between different local models is developed, which can preserve more effective features. Experimental studies on 11 UCI datasets and nine high-dimensional datasets validate the effectiveness of our proposed RP. In comparison with various state-of-the-art feature selection and LFS algorithms, RP-LFS can achieve very competitive classification accuracy while obtaining a reduced feature subset size.