Multi-objective Feature Selection with a Sparsity-based Objective Function and Gradient Local Search for Multi-label Classification

Abstract

In classification, feature selection (FS) removes irrelevant and redundant features to improve the classification performance. Most of the existing FS approaches are designed specifically for single-label classification, wherein each instance is associated to a single class label. However, many real-world problems have more than one class label for each instance, also known as multi-label classification (MLC). Label-to-label interactions in MLC information presents a new challenge for FS, whereas single-label FS methods typically only consider feature-to-feature (redundancy) and feature-to-label (relevancy) information. Sparsity-based FS is an efficient and effective method for MLC which has demonstrated its ability in considering label-to-label interactions. However, existing sparsity-based approaches can be trapped at local optima due to their gradient-based search. In addition, they cannot consider different trade-off between the two objectives of FS: to minimise the number of features and to maximise the classification performance. Evolutionary multi-objective optimisation (EMO) can address the above issues, but EMO has not been applied to achieve sparsity-based FS for MLC. This paper proposes a novel sparsity-based representation, objective function, and local search method to achieve sparsity-based EMO FS for MLC. The experimental results indicate improved efficiency and effectiveness in the MLC performance in comparison to existing state-of-the-art multi-label feature selection methods.