A Machine Learning-Based Wrapper Method for Feature Selection

Abstract

This paper presents a two-stage feature selection scheme using machine learning techniques. In the first stage a wrapper method is adopted to select various combinations of subsets of features from the original dataset. The performance of the model is evaluated by three classifiers: K-Nearest Neighbor (KNN), Support Vector Machines (SVM), and Random Forest (RF). In the second and final stage, a sequential backward feature selection Method is applied. The proposed method is demonstrated on eighteen datasets and the average classification accuracy of eighteen datasets achieved is 89.81%, 87.55%, and 89.82% using the KNN, SVM, and RF classifiers, respectively with a maximum reduced size of the subset being ten only. Comparing the proposed method to eight other feature selection methods, the former achieves better classification accuracy in terms of selecting the most useful but a smaller number of features.