Deluge based Genetic Algorithm for feature selection

Abstract

Feature selection methods are used to identify and remove irrelevant and redundant attributes from the original feature vector that do not have much contribution to enhance the performance of a predictive model. Meta-heuristic feature selection algorithms, used as a solution to this problem, need to have a good trade-off between exploitation and exploration of the search space. Genetic Algorithm (GA), a popular meta-heuristic algorithm, lacks exploitation capability, which in turn affects the local search ability of the algorithm. Basically, GA uses mutation operation to take care of exploitation which has certain limitations. As a result, GA gets stuck in local optima. To encounter this problem, in the present work, we have intelligently blended the Great Deluge Algorithm (GDA), a local search algorithm, with GA. Here GDA is used in place of mutation operation of the GA. Application of GDA yields a high degree of exploitation through the use of perturbation of candidate solutions. The proposed method is named as Deluge based Genetic Algorithm (DGA). We have applied the DGA on 15 publicly available standard datasets taken from the UCI dataset repository. To show the classifier independent nature of the proposed feature selection method, we have used 3 different classifiers namely K-Nearest Neighbour (KNN), Multi-layer Perceptron (MLP) and Support Vector Machine (SVM). Comparison of DGA has been performed with other contemporary algorithms like the basic version of GA, Particle Swarm Optimisation (PSO), Simulated Annealing (SA) and Histogram based Multi-Objective GA (HMOGA). From the comparison results, it has been observed that DGA performs much better than others in most of the cases. Thus, our main contributions in this paper are introduction of a new variant of GA for FS which uses GDA to strengthen its exploitational ability and application of the proposed method on 15 well-known UCI datasets using KNN, MLP and SVM classifiers.