Feature selection considering feature relevance, redundancy and interactivity for neighbourhood decision systems

Abstract

Feature selection is an effective method to simplify data analysis and obtain key features, which improves the accuracy and generalization ability of classifiers. Neighbourhood rough set is a typical granular computing model that enables data analysis at different granularity by setting different neighbourhood radii. In recent years, feature selection based on neighbourhood rough sets has received widespread attention from researchers. However, these models still have the following shortcomings. On the one hand, they rarely simultaneously consider the relevance, the redundancy and interactivity among features, making it difficult to accurately depict classification information in the data. On the other hand, there is still a lack of effective uncertainty measures to evaluate the classification ability of feature subsets. For these reasons, this article proposes a feature selection model that considers the relevance, redundancy, and interactivity of features. Firstly, the relevance, the redundancy and interactivity between features, are defined based on the neighbourhood, respectively. Secondly, a new feature evaluation function (MMM function) is proposed based on the principles of maximum relevance, minimum redundancy, and maximum interactivity. Finally, from the perspective of maintaining classification ability, a heuristic feature selection algorithm (FS_CRRI) was designed. To evaluate the performance of the proposed algorithm, it was compared with some representative feature selection algorithms on twelve common datasets. The experimental results on three different classifiers indicate that the FS_CRRI algorithm can effectively reduce data and achieve higher classification performance.