Fuzzy-rough hybrid dimensionality reduction

Abstract

Dimensionality reduction including feature selection and feature extraction is a significant preprocessing method for data analysis, which is an important application of fuzzy rough set theory. Linear discriminant analysis (LDA) is a commonly used supervised feature extraction method, which can generate transformations while retaining as much class discrimination information as possible. However, it is worth noting that linear discriminant analysis is not interpretable. To improve the interpretability of the model and avoid losing part of the original feature information when only using feature selection for dimensionality reduction, this paper proposes a hybrid dimensionality reduction method based on fuzzy rough set and linear discriminant analysis. It can obtain a reduction set that contains complementary transformation features and original features. First, the semi-feature set is generated by criterion based on the normalized fuzzy joint mutual information, and the hybrid feature set is further induced by applying LDA to the feature set. Then, the redundancy between features is evaluated by the normalized fuzzy mutual information. The significance of a feature is defined by the proposed normalized effective classification information amount and the symmetric effective classification information amount. Next, the priority value between features according to the significance of them is defined to delete the redundant feature from the hybrid feature set. Furthermore, a hybrid dimensionality reduction algorithm based on fuzzy rough set is designed further. Finally, the performance of the proposed method is compared with some methods. The experimental results show that the proposed algorithm can obtain effective feature subset, which improve the performance of a classifier.