Recursive elimination current algorithms and a distributed computing scheme to accelerate wrapper feature selection

Abstract

Feature selection (FS) for classification tasks in machine learning and data mining has attracted significant attention. Recently, increasing metaheuristic optimization algorithms have been applied to solve wrapper FS problems. Nevertheless, the algorithms that improve existing optimizers inevitably bring higher complexity and require higher computational cost in wrapper mode. In this work, we present recursive elimination current (REC) algorithms, a novel set of algorithms for wrapper FS, which consists of the simplest feature subset representation, an ingenious structure enlightened by the recursion technique in computer science and necessary components. To some extent, the proposed algorithms, recurrent REC (REC2) and distributed REC (DiREC), dispose of the issues, including but not limited to keeping diversity of population and scalability for high-dimensionality, which are often discussed in metaheuristics-based FS research. Thereinto, DiREC is a distributed computing scheme proposed to accelerate the FS process by distributing the tasks to different computing units in a cluster of computers. A series of experiments were carried out on several representative benchmark datasets, such as Madelon from UCI machine learning repository, using REC2 and DiREC with various numbers of logical processors. It is demonstrated that the proposed algorithms perform efficiently in wrapper mode, and the distributed computing scheme is effective and yields computational time savings.