Exploiting fuzzy rough mutual information for feature selection

Abstract

Feature selection is one of the important applications of rough set theory. Rough entropy proposed in rough set theory has been applied to feature selection. However, rough entropy is based on binary equivalence relation to divide object sets. Therefore, it applies only to nominal attribute data. To this end, this paper extends rough entropy to fuzzy rough set theory, and then proposes the fuzzy rough entropy in fuzzy approximate space. Fuzzy rough entropy decreases monotonically with the increase of the number of features. On this basis, the concepts of fuzzy joint rough entropy, fuzzy conditional rough entropy and fuzzy rough mutual information are defined. In order to measure the importance of features, inner and outer significance functions are constructed by making use of fuzzy rough mutual information. Furthermore, based on the proposed fuzzy rough entropy model, the corresponding feature selection algorithm is designed. It can directly deal with not only nominal data, but also numerical data and even mixed data. The use of inner and outer heuristic significance functions makes the proposed method select features from two complementary perspectives, so that the reduced feature set has better classification performance. In this method, the redundant features are effectively deleted by the backward redundancy elimination strategy. The proposed algorithm is compared with other algorithms on public data. The experimental results show that the proposed method is adaptive and effective.