Influence of Relief Feature Selection on Random Forest and Support Vector Machine Classification Algorithm

Abstract

The classification technique is one of the popular techniques used in helping humans decide the target class of a data based on machine learning principles. Unfortunately the construction of a classification model has no limits and will always evolve over time. There is no surefire way to make a perfect classification model, but there are ways that at least make the classification model better. This study applies the feature selection method to produce a more optimal classification model accuracy value. Of the many feature selection algorithms, this research chooses Relief which is combined with a classification algorithm, namely Random Forest and Support Vector Machine. This research also applies the Grid Search Optimization method in selecting the most influential features. In addition, it is also used to select the best hyperparameters to build the classification model. For splitting the data set, the K Fold Cross Validation technique is used in order to get the most optimal proportion of data splitting. Compared to the accuracy values before and after feature selection, both classification algorithms after feature selection significantly outperform the classification model before feature selection. It was also found that the model’s capabilities in the real world, through validation with new data, performed quite well.