A novel random multi-subspace based ReliefF for feature selection

Abstract

Feature selection is an important preprocessing technology for dimensionality reduction, which reduces the dimension of the dataset by acquiring a subset of features with the largest amount of information, and improves the classification accuracy to the greatest extent at the same time. Although different types of feature selection algorithms have achieved remarkable success, most of them lack the ability to mine information in different subspaces, and ignore the useful information contained in the abundant samples. In this research, a novel random multi-subspace based ReliefF (RBEFF) is proposed for feature selection. In this method, firstly, multiple feature partitions containing a large number of random subspaces with the same size are generated. Secondly, the ReliefF algorithm is used in each random subspace to obtain the local weight of the feature. The local weight vectors of random subspaces in each feature partition are combined to obtain the full weight vector. Finally, the full weight vectors of multiple feature partitions are integrated into the final weight vector, which contains the final weight of each feature in the original feature space feature. The feature selection is carried out dynamically according to the final weight vector. We evaluated the performance of the RBEFF on 28 UCI datasets with different sizes and compare RBEFF with 6 feature selection algorithms using KNN and DT classifiers’ three evaluation indicators. The comparisons and experimental results demonstrate the effectiveness, competitiveness, and superiority of RBEFF in solving feature selection problems.