Stability Investigation of Ensemble Feature Selection for High Dimensional Data Analytics

Abstract

AbstractIn the selection of feature subsets, stability is an important factor. In literature, however, stability receives less emphasis. Stability analysis of an algorithm is used to determine the reproducibility of algorithm findings. Ensemble approaches are becoming increasingly prominent in predictive analytics due to their accuracy and stability. The accuracy and stability dilemma for high-dimensional data is a significant research topic. The purpose of this research is to investigate the stability of ensemble feature selection and utilize that information to improve system accuracy in high-dimensional datasets. We conducted a stability analysis of the ensemble feature selection approaches ChS-R and SU-R using the jaccard similarity index. Ensemble approaches have been found to be more stable than previous feature selection methods such as SU and ChS for high-dimensional datasets. The average stability of the SU-R and ChS-R ensemble approaches is 56.03 and 50.71%, respectively. Accuracy improvement achieved is 4 to 5%.KeywordsHigh-dimensional dataEnsemble learningStability analysisFeature selectionJaccard index