Set based particle swarm optimization for the feature selection problem

Abstract

Selecting the correct features when training a classification algorithm, has a significant impact on the performance of the classifier. More features provide more information, but can lead to overfitting and utilizing features that are redundant, irrelevant or too noisy. The feature selection problem (FSP) is concerned with identifying those features, from the entire set of features, that lead to the best possible classification. This article evaluates the performance of the set based particle swarm optimization (SBPSO) algorithm on the FSP. SBPSO was specifically developed to solve discrete-valued optimization problems that can be formulated as set-based problems. A wrapper based SBPSO algorithm based on a k-nearest neighbor classifier is proposed in this paper. The SBPSO wrapper algorithm was compared to three other discrete PSO wrapper algorithms on a large number of datasets of different sizes and outperformed, with statistical significance, the other algorithms on the FSP. The SBPSO algorithm can thus be considered an effective tool for solving the FSP.