A Multiform Optimization Framework for Multi-objective Feature Selection in Classification

Abstract

Feature selection in machine learning as a key data processing technique has two conflicting goals: minimizing the classification error rate and minimizing the number of features selected. However, most of the existing multi-objective feature selection methods face the problems of easily falling into local optima and slow convergence by virtue of their problem characteristics, such as partially conflicting objectives and highly discontinuous Pareto fronts. To solve these problems, this article proposes a multiform optimization framework to solve a multi-objective feature selection task together with several auxiliary single-objective feature selection tasks in a multitask environment. The proposed framework uses the problem-solving experience of single-objective tasks to assist the multi-objective feature selection task in exploring more promising regions and accelerating the convergence speed. Specifically, a knowledge transfer strategy based on the search experience of different tasks is developed to accomplish multiform optimization. In addition, a diversity enhancement mechanism is presented to improve search ability in promising decision space areas by considering historical information about the population. In most cases, the experiment results on 27 datasets demonstrate that the proposed technique can uncover more diversified feature subsets on the Pareto front in less time than existing state-of-the-art methods.