Hierarchical feature selection with subtree based graph regularization

Abstract

Feature selection is an important and challenging task in machine learning and data mining. In many practical problems, the classes have a hierarchical structure. However, some existing feature selection algorithms ignored the dependence among different classes in the hierarchical structure. Other feature selection algorithms only focused on one way dependence among different classes, ignoring two-way dependence. In this paper, we propose a novel feature selection method called hierarchical feature selection with subtree based graph regularization (HFSGR), which is aimed at exploring two-way dependence among different classes. First, we construct a subtree graph using the parent–child relationships of the subtrees in a predefined tree structure, where the subtree is obtained from its internal nodes. Second, we use the l2,1-norm regularization term to encourage nearby subtrees that share similar sparsity patterns. Third, we extend our algorithm to a directed acyclic graph structure so that it can be applied to common situations. Our method is applied to eight datasets with different tree structures. Experimental comparisons of our proposed algorithm with five hierarchical feature selection algorithms, justify its effectiveness and efficiency.