Hierarchical feature extraction based on discriminant analysis

Abstract

Feature extraction is a crucial technique for data preprocessing in classification tasks such as protein classification and image classification. Datasets with tree class hierarchies have become extremely common in many practical classification tasks. However, existing flat feature extraction algorithms tend to assume that classes are independent and ignore the hierarchical information of class structure within a dataset. In this paper, we propose a hierarchical feature extraction algorithm based on discriminant analysis (HFEDA). HFEDA first decomposes the highly complex feature extraction problem into smaller problems by creating sub-datasets for non-leaf nodes according to the tree class hierarchy of dataset. Secondly, different from flat algorithms, HFEDA takes the hierarchical class structure into account in dimensionality reduction process, and calculates the projection matrices for the non-leaf nodes in the tree class hierarchy. In this way, HFEDA can just focus on discriminating the several categories under the same parent node. Finally, HFEDA does not need to determine the optimal feature subset size, which is challenging for most feature selection algorithms. Extensive experiments on different type datasets and typical classifiers demonstrate the effectiveness and efficiency of the proposed algorithm.