MFSJMI: Multi-label feature selection considering join mutual information and interaction weight

Abstract

Multi-label feature selection captures a reliable and informative feature subset from high-dimensional multi-label data, which plays an important role in pattern recognition. In conventional information-theoretical based multi-label feature selection methods, the high-order feature relevance between feature and label set is evaluated using low-order mutual information. However, existing methods do not establish the theoretical basis for the low-order approximation. To fill this gap, we first identify two underlying assumptions based on high-order label distribution: Label Independence Assumption (LIA) and Paired-label Independence Assumption (PIA). Second, we systematically analyze the strengths and weaknesses of two assumptions and introduce joint mutual information to satisfy more realistic label distribution. Furthermore, by decomposing joint mutual information, an interaction weight is proposed to consider multiple label correlations. Finally, a new method considering join mutual information and interaction weight is proposed. Comprehensive experiments demonstrate the effectiveness of the proposed method on various evaluation metrics.