Sparsity Preserved Pareto Optimization for Subset Selection

Abstract

Subset selection that selects a limited number of variables optimizing many given criteria, is a fundamental problem with various applications such as sparse regression and unsupervised feature selection. Among the existing algorithms for subset selection, evolutionary algorithms (EAs) have achieved great performance in obtaining the subsets with high quality. However, as the selected subsets are sparse, i.e., most variables of these solutions are zero, most existing EAs for subset selection problems pose great challenges to find the optimal solutions effectively and efficiently, especially when the size of original set is large. To this end, we propose the SPESS approach which is a sparsity preserved evolutionary algorithm for subset selection. To be specific, we design two sparsity preserved operators (i.e., sparsity preserved crossover operator and sparsity preserved mutation operator), which can be used in SPESS to ensure the sparsity of the generated solutions in the process of Pareto optimization. The main advantages of the proposed sparsity preserved operators are that they can obtain high quality solutions and improve the search efficiency of the algorithm. According to the experimental results on 12 sparse regression problems and 12 unsupervised feature selection problems, the proposed SPESS is superior over the state-of-the-arts in solving subset selection tasks and the effectiveness of the proposed two sparsity preserved operators is also verified.