A robust ensemble feature selection technique for high‐dimensional datasets based on minimum weight threshold method

Abstract

AbstractEnsemble feature selection (EFS) is a valuable technique for developing accurate and robust machine‐learning (ML) models. Data variation plays a crucial role in the success of EFS models; however, it also causes some outliers in the ranked lists. In this study, we proposed the minimum weight threshold method‐based EFS (MWT‐EFS) to address the outlier problem and use the true power of EFS. The proposed method employs the support vector classifier to assign weights for features, and the MWT method handles outliers in the ranked feature lists while creating the ensemble list. First, a threshold value is determined. After that, the feature weights below the threshold are replaced with this value. This approach eliminates the negative effect of outliers. After the new feature weights are assigned, the average of the feature weights is calculated (mean aggregation) for all features, and the ensemble (final) feature list is created accordingly. The experiment results showed that the proposed method significantly improves gene selection stability while maintaining classification performance and reducing computational complexity. In conclusion, the proposed method led to an accurate and robust classification that can help domain experts to make more confident decisions with less effort, resources, and time.