Information gain-based multi-objective evolutionary algorithm for feature selection

Abstract

Feature selection (FS) has garnered significant attention because of its pivotal role in enhancing the efficiency and effectiveness of various machine learning and data mining algorithms. Concurrently, multiobjective feature selection (MOFS) algorithms strive to balance the complexity of multiple optimization objectives during the FS process. These include minimizing the number of selected features while maximizing classification performance. Nonetheless, managing the complexity of feature combinations presents a formidable challenge, particularly in high-dimensional datasets. Evolutionary algorithms (EAs) are increasingly adopted in MOFS owing to their exceptional global search capabilities and robustness. Despite their strengths, EAs face difficulties in navigating expansive solution spaces and achieving a balance between exploration and exploitation. To address these challenges, this study introduces a novel information gain-based EA for MOFS, designated as IGEA. This approach utilizes a clustering method for selecting a diverse parent population, thereby enhancing individual variability and maintaining a high-quality population. Considerably, IGEA employs information gain as a metric to evaluate the contribution of features to classification tasks. This metric informs crucial operations such as crossover and mutation. Moreover, the study extensively examines the actual solutions derived from IGEA, focusing on feature correlation and redundancy. This analysis illuminates IGEA's adept handling of these aspects to refine MOFS. Experimental results on 23 widely used classification datasets confirm IGEA's superiority over five other state-of-the-art algorithms, demonstrating its enhanced effectiveness and efficiency in complex MOFS scenarios.