An adaptive pyramid PSO for high-dimensional feature selection

Abstract

Feature selection (FS) is an important data pre-processing method in machine learning tasks. However, FS becomes an NP-hard combinatorial optimization problem due to the huge search space of high-dimensional datasets and the complex interaction among features. As a simple implementation algorithm, particle swarm optimization (PSO) has been employed for dealing with FS problems. At the same time, it still suffers from issues such as the premature convergence and falling into local optimum. In this paper, an adaptive pyramid PSO (APPSO) is proposed for FS on dataset classification problems. In APPSO, a weighted initialization strategy based on correlation and the cubic chaotic map is employed to generate particles in the initial stage, aiming to improve the diversity of a population. Then, an adaptive constrained updating strategy, based on the pyramid structure, is proposed to enhance the exploration ability of the population. Moreover, a new dynamical flip strategy (DFS), which combines feature correlation and occurrence frequencies of features, is proposed to improve the exploitation ability of the population. The proposed APPSO is compared with 8 representative wrapper-based FS methods. Experimental results show that APPSO significantly outperforms these competitors on all 18 datasets.