High-dimensional Feature Selection Problem Research Articles

Recovery of weak signal in high dimensional linear regression by data perturbation

How to recover weak signals (i.e., small nonzero regression coefficients) is a difficult task in high dimensional feature selection problems. Both convex and nonconvex regularization methods fail to fully recover the true model whenever there exist strong columnwise correlations in design matrices or small nonzero coefficients below some threshold. To address the two challenges, we propose a procedure, Perturbed LASSO (PLA), that weakens correlations in the design matrix and strengthens signals by adding random perturbations to the design matrix. Moreover, a quantitative relationship between the selection accuracy and computing cost of PLA is derived. We theoretically prove and demonstrate using simulations that PLA substantially improves the chance of recovering weak signals and outperforms comparable methods at a limited cost of computation.

Feature selection for high-dimensional classification using a competitive swarm optimizer

When solving many machine learning problems such as classification, there exists a large number of input features. However, not all features are relevant for solving the problem, and sometimes, including irrelevant features may deteriorate the learning performance.Please check the edit made in the article title Therefore, it is essential to select the most relevant features, which is known as feature selection. Many feature selection algorithms have been developed, including evolutionary algorithms or particle swarm optimization (PSO) algorithms, to find a subset of the most important features for accomplishing a particular machine learning task. However, the traditional PSO does not perform well for large-scale optimization problems, which degrades the effectiveness of PSO for feature selection when the number of features dramatically increases. In this paper, we propose to use a very recent PSO variant, known as competitive swarm optimizer (CSO) that was dedicated to large-scale optimization, for solving high-dimensional feature selection problems. In addition, the CSO, which was originally developed for continuous optimization, is adapted to perform feature selection that can be considered as a combinatorial optimization problem. An archive technique is also introduced to reduce computational cost. Experiments on six benchmark datasets demonstrate that compared to the canonical PSO-based and a state-of-the-art PSO variant for feature selection, the proposed CSO-based feature selection algorithm not only selects a much smaller number of features, but result in better classification performance as well.

Parallel alternatives for evolutionary multi-objective optimization in unsupervised feature selection

Many machine learning and pattern recognition applications require reducing dimensionality to improve learning accuracy while irrelevant inputs are removed. This way, feature selection has become an important issue on these researching areas. Nevertheless, as in past years the number of patterns and, more specifically, the number of features to be selected have grown very fast, parallel processing constitutes an important tool to reach efficient approaches that make possible to tackle complex problems within reasonable computing times. In this paper we propose parallel multi-objective optimization approaches to cope with high-dimensional feature selection problems. Several parallel multi-objective evolutionary alternatives are proposed, and experimentally evaluated by using some synthetic and BCI (Brain–Computer Interface) benchmarks. The experimental results show that the cooperation of parallel evolving subpopulations provides improvements in the solution quality and computing time speedups depending on the parallel alternative and data profile.

COFADMM: A Computational Features Selection with Alternating Direction Method of Multipliers

Due to the explosion in size and complexity of Big Data, it is increasingly important to be able to solve problems with very large number of features. Classical feature selection procedures involves combinatorial optimization, with computational time increasing exponentially with the number of features. During the last decade, penalized regression has emerged as an attractive alternative for regularization and high dimensional feature selection problems. Alternating Direction Method of Multipliers (ADMM) optimization is suited for distributed convex optimization and distributed computing for big data. The purpose of this paper is to propose a broader algorithm COFADMM which combines the strength of convex penalized techniques in feature selection for big data and the power of the ADMM for optimization. We show that combining the ADMM algorithm with COFADMM can provide a path of solutions efficiently and quickly. COFADMM is easy to use, is available in C, Matlab upon request from the corresponding author.