Binary chemical reaction optimization based feature selection techniques for machine learning classification problems

Abstract

Feature selection is an important pre-processing technique for dimensionality reduction of high-dimensional data in machine learning (ML) field. In this paper, we propose a binary chemical reaction optimization (BCRO) and a hybrid binary chemical reaction optimization-binary particle swarm optimization (HBCRO-BPSO) based feature selection techniques to optimize the number of selected features and improve the classification accuracy. Three objective functions have been used for the proposed feature selection techniques to compare their performances with a BPSO and advanced binary ant colony optimization (ABACO) along with an implemented GA based feature selection approach called as binary genetic algorithm (BGA). Five ML algorithms including K-nearest neighbor (KNN), logistic regression, Naïve Bayes, decision tree, and random forest are considered for classification tasks. Experimental results tested on eleven benchmark datasets from UCI ML repository show that the proposed HBCRO-BPSO algorithm improves the average percentage of reduction in features (APRF) and average percentage of improvement in accuracy (APIA) by 5.01% and 3.83%, respectively over the existing BPSO based feature selection method; 4.58% and 3.12% over BGA; and 4.15% and 2.27% over ABACO when used with a KNN classifier.