Fuzzy rough dimensionality reduction: A feature set partition-based approach

Abstract

Dimensionality reduction is considered in many learning methods using discriminative features to obtain optimal performance. In general, feature extraction and feature selection are two independent methods that cherry-pick the informative features in the mapping data space or original feature space. Fuzzy rough set theory is widely used in feature selection, which well keeps the interpretability of features. How to comprehensively consider the intrinsic information contained in features and combine the feature extraction with feature selection is a challenging and meaningful task. Most existing fuzzy rough-based feature selection methods select features from the whole feature set. However, some nonsignificant features may lead to a decline in useful information. Meanwhile, these existing methods directly remove redundant and irrelevant features. This study scrutinizes features to distinguish the significant degree of features and deal with them in different ways. The feature selection and feature extraction methods are combined to obtain compound features for the learning method. A feature set partition-based approach to fuzzy rough dimensionality reduction (FSPFRdr) by simultaneously considering feature selection and fuzzy similarity relation-based supervised locally linear embedding (FRSLLE) is proposed. First, the ϑ-fuzzy rough set model is defined to avoid the affection of noise, and approximation space is constructed based on fuzzy approximations and granular structure. The normalized independent classification information (NICI) is defined in the approximation space. According to NICI, the original feature set is divided into a nonsignificant feature set, a weak significant feature set, and a significant feature set. Then, the nonsignificant feature set is deleted before dimensionality reduction. Next, the proposed FRSLLE is applied to a weak significant feature set, and the embedded feature set is obtained. Furthermore, a feature evaluation function is proposed to select features from the significant and embedded feature sets. Finally, the performance of FSPFRdr is compared with thirteen dimensionality reduction algorithms on twenty-three public datasets. Experimental results on four different classifiers show that FSPFRdr has higher classification performance.