A unified low-order information-theoretic feature selection framework for multi-label learning

Abstract

The approximation of low-order information-theoretic terms for feature selection approaches has achieved success in addressing high-dimensional multi-label data. However, three critical issues exist in such type of approaches: (1) existing approaches are devised based on single heuristic variable correlation assumption, which biases towards some specific scene; (2) high-order variable correlations are ignored by cumulative summation low-order information-theoretic terms; (3) abundant approaches confuse researchers to devise and utilize appropriate approaches. To address these issues, two types of probability distribution assumption in terms of candidate features and labels are derived based on low-order variable correlations. Afterwords, clearing up all information-theoretic terms, we propose a unified feature selection framework including three low-order information-theoretic terms for multi-label learning named Selected Terms of Feature Selection (STFS). STFS contains high-order variable correlations in the form of low-order information-theoretic terms. Furthermore, many previous multi-label feature selection approaches can be reduced to special forms of STFS. Finally, extensive experiments conducted on twelve benchmark data sets in comparison to seven state-of-the-art approaches demonstrate the classification superiority of STFS.