Multi-objective Multi-label Feature Selection with an Aggregated Performance Metric and Dominance-based Initialisation

Abstract

Feature selection (FS) is an essential pre-processing step that selects small and informative feature subsets. However, most FS methods focus on single-label classification, where each instance has one class label. Multi-label classification (MLC) is increasingly more common as instances possess a set of class labels. It is more challenging to perform FS for MLC since it needs to consider the interactions between labels. Furthermore, since the target output is a set of labels, there are various metrics for evaluating MLC performance, and each metric assesses the quality of a feature subset on various aspects. However, certain MLC metrics are shown to possess conflicting behaviour. To evolve high-quality feature subsets, it is essential to consider the trade-offs between metrics, making FS for MLC even more challenging. Evolutionary multi-objective optimisation (EMO) is a great technique to optimise multiple (potentially) conflicting metrics, representing different objectives. However, given a large number of objectives, EMO algorithms usually do not evolve a diverse set of good feature subsets, especially when there is a limited computational resource. This paper proposes a novel approach to reducing the number of objectives, which is expected to maintain or improve the evolved feature subsets over the original objectives. We also propose a new decomposition mechanism based on multiple reference points and a novel initialization mechanism to enhance the quality of the evolved feature subsets. The experimental results show that the proposed algorithm can evolve diverse feature subsets with better trade-off between multiple performance metrics than recent FS algorithms for MLC.