EDDE–LNS: a new hybrid ensemblist approach for feature selection

Abstract

Feature selection is the process of selecting a subset of relevant, non-redundant features from the original ones. It is an NP-hard combinatorial optimization problem. In this paper, we propose a new feature selection method, abbreviated as EDDE–LNS, using a combination of large neighbourhood search (LNS) and a new Ensemblist Discrete Differential Evolution (EDDE). Each solution of the search space represents a feature subset of predefined size K. EDDE–LNS explores this search space by evolving a population of individuals in two phases. During the first phase, the LNS strategy is used to improve each feature subset by alternately destroying and repairing it. The proposed accuracy rate difference measure is used to determine irrelevant and redundant features that are removed during the application of the destruction process. In the second phase, the individuals resulting from the application of LNS are used as inputs to the proposed EDDE approach. EDDE is a discrete algorithm inspired by the differential evolution (DE) method. Whereas the original DE method attempts to find the best feature subset in a multidimensional space by applying simple and fast arithmetic operators to each dimension (feature) separately, the EDDE approach proposed in this paper attempts to find the best feature subset in a single dimension space by applying new ensemblist operators to a set of K features. In this way, EDDE will consider the possible interactions between features. Experiments are conducted on intrusion detection and other machine learning datasets. The results indicate that the proposed approach is able to achieve good accuracies in comparison with other well-known feature selection methods.