Information gain ratio-based subfeature grouping empowers particle swarm optimization for feature selection

Abstract

Feature selection is a critical preprocessing step in machine learning with significant real-world applications. Despite the widespread use of particle swarm optimization (PSO) for feature selection, owing to its robust global search capabilities, developing an effective PSO method for this task is still a substantial challenge. This study introduces a novel PSO variant, ISPSO, which integrates the information gain ratio for assessing feature importance. ISPSO’s feature selection process involves partitioning features into distinct groups to establish the initial population. Recognizing that feature selection tasks are inherently binary, ISPSO replaces the traditional PSO velocity concept with a probabilistic approach. In addition, introducing a penalty term enhances the algorithm’s ability to achieve superior results. Experimental evaluations on 16 datasets consistently show that ISPSO surpasses compared algorithms, highlighting its efficiency in eliminating redundant and irrelevant features.