A relative decision entropy-based feature selection approach

Abstract

Rough set theory has been proven to be an effective tool for feature selection. To avoid the exponential computation in exhaustive methods, many heuristic feature selection algorithms have been proposed in rough sets. However, these algorithms still suffer from high computational cost. In this paper, we propose a novel heuristic feature selection algorithm (called FSMRDE) in rough sets. To measure the significance of features in FSMRDE, we propose a new model of relative decision entropy, which is an extension of Shannon׳s information entropy in rough sets. Moreover, to test the effectiveness of FSMRDE, we apply it to intrusion detection and other application domains. Experimental results show that by using the relative decision entropy-based feature significance as heuristic information, FSMRDE is efficient for feature selection. In particular, FSMRDE is able to achieve good scalability for large data sets.