High-dimensional Feature Selection Problem Research Articles

U-AEFA: Online and offline learning-based unified artificial electric field algorithm for real parameter optimization

Optimization problems in real-world scenarios require algorithms that effectively balance exploration and exploitation to avoid local optima and achieve global solutions. To address this, we propose a unified artificial electric field algorithm (U-AEFA) that integrates both offline learning (inspired by high-performing metaheuristic algorithms) and online learning (through historical data generated during evolution). U-AEFA introduces a unique three-layer population structure to enhance search efficiency, consisting of the best agent in the first layer, the top-performing agents in the second layer, and the remaining agents in the third layer. Key features of U-AEFA include (i) an effective Coulomb’s constant for improved exploration, (ii) a non-uniform mutation operator to mitigate premature convergence, and (iii) two acceleration coefficients for enhanced performance. These three factors constitute offline learning and have been implemented to improve different design elements of the algorithm. As part of online learning, it employs a difference vector reuse (DVR) strategy to evolve the first-layer agents. The algorithm is evaluated using the CEC 2017 test suite across multiple dimensions (10, 30, 50, 100), where it consistently outperforms seven state-of-the-art algorithms, demonstrating superior accuracy and convergence speed. Moreover, U-AEFA’s robustness is validated on 12 high-dimensional feature selection problems, further highlighting its effectiveness in solving complex optimization tasks. The source code of U-AEFA is available at

S-shaped grey wolf optimizer-based FOX algorithm for feature selection

The FOX algorithm is a recently developed metaheuristic approach inspired by the behavior of foxes in their natural habitat. While the FOX algorithm exhibits commendable performance, its basic version, in complex problem scenarios, may become trapped in local optima, failing to identify the optimal solution due to its weak exploitation capabilities. This research addresses a high-dimensional feature selection problem. In feature selection, the most informative features are retained while discarding irrelevant ones. An enhanced version of the FOX algorithm is proposed, aiming to mitigate its drawbacks in feature selection. The improved approach referred to as S-shaped Grey Wolf Optimizer-based FOX (FOX-GWO), which focuses on augmenting the local search capabilities of the FOX algorithm via the integration of GWO. Additionally, the introduction of an S-shaped transfer function enables the population to explore both binary options throughout the search process. Through a series of experiments on 18 datasets with varying dimensions, FOX-GWO outperforms in 83.33 % of datasets for average accuracy, 61.11 % for reduced feature dimensionality, and 72.22 % for average fitness value across the 18 datasets. Meaning it efficiently explores high-dimensional spaces. These findings highlight its practical value and potential to advance feature selection in complex data analysis, enhancing model prediction accuracy.

An improved binary particle swarm optimization algorithm for clinical cancer biomarker identification in microarray data

Background and ObjectiveThe limited number of samples and high-dimensional features in microarray data make selecting a small number of features for disease diagnosis a challenging problem. Traditional feature selection methods based on evolutionary algorithms are difficult to search for the optimal set of features in a limited time when dealing with the high-dimensional feature selection problem. New solutions are proposed to solve the above problems. MethodsIn this paper, we propose a hybrid feature selection method (C-IFBPFE) for biomarker identification in microarray data, which combines clustering and improved binary particle swarm optimization while incorporating an embedded feature elimination strategy. Firstly, an adaptive redundant feature judgment method based on correlation clustering is proposed for feature screening to reduce the search space in the subsequent stage. Secondly, we propose an improved flipping probability-based binary particle swarm optimization (IFBPSO), better applicable to the binary particle swarm optimization problem. Finally, we also design a new feature elimination (FE) strategy embedded in the binary particle swarm optimization algorithm. This strategy gradually removes poorer features during iterations to reduce the number of features and improve accuracy. ResultsWe compared C-IFBPFE with other published hybrid feature selection methods on eight public datasets and analyzed the impact of each improvement. The proposed method outperforms other current state-of-the-art feature selection methods in terms of accuracy, number of features, sensitivity, and specificity. The ablation study of this method validates the efficacy of each component, especially the proposed feature elimination strategy significantly improves the performance of the algorithm. ConclusionsThe hybrid feature selection method proposed in this paper helps address the issue of high-dimensional microarray data with few samples. It can select a small subset of features and achieve high classification accuracy on microarray datasets. Additionally, independent validation of the selected features shows that those chosen by C-IFBPFE have strong correlations with disease phenotypes and can identify important biomarkers from data related to biomedical problems.

A fuzzy transfer function based on the behavior of meta-heuristic algorithm and its application for high-dimensional feature selection problems

Many discrete optimization problems are in the class of NP-hard problems; therefore, exact algorithms cannot achieve an optimum solution in reasonable time. Approximate algorithms like binary meta-heuristic algorithms have provided good results in these problems. Since many meta-heuristic algorithms have been introduced for continuous search spaces, employing a transfer function is one of the simplest methods to map the search space to the binary one. Many transfer functions have been introduced, divided in atomic and compound transfer functions. In the atomic transfer functions, a single function is applied in the algorithm, such as S-shaped, U-shaped, V-shaped, Z-shaped and so on. Compound transfer functions employ more than one atomic transfer function in their structures, such as Mirrored Time-Varying S-shaped, χ-shaped, and Upgrade transfer functions. An improper transfer function decreases the algorithm's performance in the binary search space. Hence, several transfer functions are usually evaluated for solving a binary problem to select the best one. In this study, a fuzzy transfer function is proposed and applied in Binary Particle Swarm Optimization (FBPSO). The behavior of PSO is analyzed, and the fuzzy transfer function and its rules create a new binary position. The performance of atomic and compound transfer functions employed in BPSO and some recently introduced binary meta-heuristic algorithms are compared with FBPSO in high-dimensional feature selection problems. The experimental results show that FBPSO achieves the best solution in terms of minimum error and feature selection rate compared with other comparative algorithms.

A new binary object-oriented programming optimization algorithm for solving high-dimensional feature selection problem

Feature selection (FS) is a crucial task in machine learning applications, which aims to select the most appropriate feature subset while maintaining high classification accuracy with the minimum number of selected features. Despite the widespread usage of metaheuristics as wrapper-based FS techniques, they show reduced effectiveness and increased computational cost when applied to high-dimensional datasets. This paper presents a novel Binary Object-Oriented Programming Optimization Algorithm (BOOPOA) for FS of high dimensional datasets, where the Object-Oriented Programming Optimization Algorithm (OOPOA) is a novel optimization technique inspired by the inheritance concept of Object-Oriented programming (OOP) languages. The effectiveness of this method in solving high dimensional FS problems is validated by using 26 datasets, most of which are of high dimension (large number of features). Seven existing FS algorithms are compared with the proposed OOPOA using various metrics, including best fitness, average fitness (AVG), selection size, and computational time. The results prove the superiority of the proposed algorithm over the other FS algorithms, having an average performance of %92.5, 0.078, 0.084, %38.9, and 8.6 min for classification accuracy, best fitness, average fitness, size reduction ratio, and computational time. The outcomes demonstrate the proposed FS approach's superiority over currently used methods.

Copula entropy-based golden jackal optimization algorithm for high-dimensional feature selection problems

Feature selection (FS) is a crucial process that aims to remove unnecessary features from datasets. It plays a role in data mining and machine learning (ML) by reducing the risk associated with high-dimensional datasets. FS is considered a challenging problem that is difficult to solve efficiently due to its combinatorial nature. As the size of the problem increases, the computation time also grows. Recently, researchers have focused on metaheuristic FS algorithms specifically designed for high-dimensional datasets. Therefore, this article proposes a powerful metaheuristic algorithm called Binary Enhanced Golden Jackal Optimization (BEGJO), which is an improved version of the recently published Golden Jackal Optimization (GJO) algorithm. The original GJO algorithm faces challenges when dealing with high-dimensional FS problems, as it tends to get trapped in local optima. To address this issue, various enhancement strategies are employed to improve the efficiency of GJO. The proposed BEGJO algorithm utilizes Copula Entropy (CE) to reduce the dimensionality of high-dimensional FS problems while maintaining high classification accuracy using the K-Nearest Neighbour (K-NN) classifier. Additionally, four enhancement strategies are incorporated to enhance the exploration and exploitation capabilities of the fundamental GJO algorithm. The BEGJO algorithm is transformed into its binary form using the sigmoid transfer function, aligning it with the nature of the FS problem. It is then tested on various high-dimensional benchmark datasets. The effectiveness of BEGJO is evaluated by comparing it with well-known algorithms in terms of classification accuracy, feature dimension, and processing time. BEGJO outperforms other algorithms in terms of classification accuracy and feature dimension and ranks up to fourth in terms of processing time. Furthermore, the advantageous use of CE is demonstrated by comparing the performance of the proposed algorithm with traditional FS algorithms. Statistical evaluations are conducted to further validate the effectiveness and superiority of the proposed algorithm. The results confirm that BEGJO is an effective solution for high-dimensional FS problems.

Locating Multiple Equivalent Feature Subsets in Feature Selection for Imbalanced Classification

Feature selection can be used to solve imbalanced classification problems encountered in big data projects. There often exist multiple feature subsets achieving the same accuracy. These subsets tend to exhibit different acquisition difficulty and reliability, thus offering decision-makers with multiple choices if they can be well-identified. This work formulates feature selection as a Multimodal Multiobjective Problem (MMOP), where a point on Pareto front in objective space has multiple equivalent feature subsets in decision space. To seek more equivalent feature subsets, this work proposes a new multiobjective fireworks algorithm. It extends a latest single-objective fireworks algorithm to a multiobjective version such that it becomes suitable for solving MMOP. An adaptive strategy and special archive guidance are newly designed to improve its performance. A weighted extreme learning machine is chosen to classify datasets and return classification accuracy due to its fast learning speed. Experimental results show that the proposed algorithm outperforms its compared ones on 15 imbalanced classification datasets including 5 low-dimensional, 5 high-dimensional feature selection problems and 5 large-scale problems with larger imbalanced ratio, and its runtime is the least among them. Also, fault diagnosis in self-organizing cellular networks, as an important imbalance classification problem, is performed by the proposed algorithm and the results show that it can perform fault diagnosis well.

Multi-objective optimization algorithm based on clustering guided binary equilibrium optimizer and NSGA-III to solve high-dimensional feature selection problem

Feature selection (FS) is an indispensable activity in machine learning, whose purpose is to identify relevant predictive values from a high-dimensional feature space to improve performance and reduce model learning time. However, the large increase in feature dimensions poses a great challenge to FS methods. Therefore, a multi-objective optimization algorithm consisting of an Equilibrium Optimizer (EO) and NSGA-III was proposed to solve the FS problem with high-dimensional data. Through S-shaped, V-shaped and U-shaped transfer functions, the conversion from real number coding to binary coding is realized to solve discrete problems, and the influence of these three transfer functions on the effect of FS is compared. In addition, the algorithm optimizes the population in the binary search space by building an external archive, and realizes the selection and optimization of external archive individuals based on the clustering strategy. The KNN classifier was used to realize the classification progress. The simulation experiments are divided into two groups with 18 medium and high dimensional data. The first group analyzes the optimization effect of the proposed framework under three transfer functions. The second group of experiments selects the algorithm that wins in the first group of experiments and compares it with eleven classical multi-objective optimization algorithms. The evaluation criteria includes two optimization objectives of the FS problem and the optimization indices of HV and IGD. The first set of experiments showed that the U-shaped transfer function family performed best in the FS problem, with U3 being the most excellent, followed by V-shaped and S-shaped. Compared to other multi-objective optimization algorithms, the simulation results also confirm the effectiveness of the proposed strategy.

Binary dwarf mongoose optimizer for solving high-dimensional feature selection problems.

Selecting appropriate feature subsets is a vital task in machine learning. Its main goal is to remove noisy, irrelevant, and redundant feature subsets that could negatively impact the learning model’s accuracy and improve classification performance without information loss. Therefore, more advanced optimization methods have been employed to locate the optimal subset of features. This paper presents a binary version of the dwarf mongoose optimization called the BDMO algorithm to solve the high-dimensional feature selection problem. The effectiveness of this approach was validated using 18 high-dimensional datasets from the Arizona State University feature selection repository and compared the efficacy of the BDMO with other well-known feature selection techniques in the literature. The results show that the BDMO outperforms other methods producing the least average fitness value in 14 out of 18 datasets which means that it achieved 77.77% on the overall best fitness values. The result also shows BDMO demonstrating stability by returning the least standard deviation (SD) value in 13 of 18 datasets (72.22%). Furthermore, the study achieved higher validation accuracy in 15 of the 18 datasets (83.33%) over other methods. The proposed approach also yielded the highest validation accuracy attainable in the COIL20 and Leukemia datasets which vividly portray the superiority of the BDMO.

A Surrogate-Assisted Evolutionary Feature Selection Algorithm With Parallel Random Grouping for High-Dimensional Classification

Various evolutionary algorithms (EAs) have been proposed to address feature selection (FS) problems, in which a large number of fitness evaluations are needed. With the rapid growth of data scales, the fitness evaluation becomes time consuming, which makes FS problems expensive optimization problems. Surrogate-assisted EAs (SAEAs) have been widely used to solve expensive optimization problems. However, the SAEAs still face difficulties in solving expensive FS problems due to their high-dimensional discrete decision variables. To address this issue, we propose an SAEA with parallel random grouping for expensive FS problems, in which three main components consist. First, a constraint-based sampling strategy is proposed, which considers the influence of the constraint boundary and the number of selected features. Second, a high-dimensional FS problem is randomly divided into several low-dimensional subproblems. Surrogate models are then constructed in these low-dimensional decision spaces. After that, all the subproblems are optimized in parallel. The process of random grouping and parallel optimization continues until the termination condition is met. Finally, a final solution is chosen from the best solution in the historical search and the best solution in the last population using a random, distance-, or voting-based method. Experimental results show that the proposed algorithm generally outperforms traditional, ensemble, and evolutionary FS methods on 14 datasets with up to 10 000 features, especially when the required number of real fitness evaluations is limited.

An improved group teaching optimization algorithm based on local search and chaotic map for feature selection in high-dimensional data

The current study proposes a novel binary group teaching optimization algorithm with local search and chaos mapping (BGTOALC) as a wrapper-based feature selection method to solve high-dimensional feature selection problems. The local search and chaos mapping enhance the performance of the proposed algorithm. Also, two novel binary operators called Binary Teacher Phase Good Group (BTPGG) and Binary Teacher Phase Bad Group (BTPBG) are applied to the teacher’s phase for increasing the exploration and exploitation of the algorithm. Moreover, a new Binary Student Opposition-Based Learning (BSOBL) operator is introduced for the student phase, using an opposition-based strategy to achieve better exploitation. Finally, the teacher allocation phase is designed in a binary manner using the new Mean Binary Select (MBS) operator to increase the algorithm’s convergence rate. Subsequently, two other binary group teaching optimization algorithms, named BGTOAV and BGTOAS, are developed utilizing the S-shaped and V-shaped transfer functions to compare their performance with the BGTOALC algorithm. The proposed approaches are compared to other state-of-the-art binary algorithms on 30 datasets with different dimensions. Different experiments prove that the BGTOALC method outperforms the previous methods in terms of reducing the number of selected features and increasing the accuracy of the machine learning algorithm. Eventually, statistical analyses indicate the superiority of the BGTOALC method in terms of efficiency and convergence rate against other binary metaheuristic algorithms.

A self-adaptive level-based learning artificial bee colony algorithm for feature selection on high-dimensional classification

Feature selection is an important data preprocessing method in data mining and machine learning, yet it faces the challenge of “curse of dimensionality” when dealing with high-dimensional data. In this paper, a self-adaptive level-based learning artificial bee colony (SLLABC) algorithm is proposed for high-dimensional feature selection problem. The SLLABC algorithm includes three new mechanisms: (1) A novel level-based learning mechanism is introduced to accelerate the convergence of the basic artificial bee colony algorithm, which divides the population into several levels and the individuals on each level learn from the individuals on higher levels, especially, the individuals on the highest level learn from each other. (2) A self-adaptive method is proposed to keep the balance between exploration and exploitation abilities, which takes the diversity of population into account to determine the number of levels. The lower the diversity is, the fewer the levels are divided. (3) A new update mechanism is proposed to reduce the number of selected features. In this mechanism, if the error rate of an offspring is higher than or is equal to that of its parent but selects more features, then the offspring is discarded and the parent is retained, otherwise, the offspring replaces its parent. Further, we discuss and analyze the contribution of these novelties to the diversity of population and the performance of classification. Finally, the results, compared with 8 state-of-the-art algorithms on 12 high-dimensional datasets, confirm the competitive performance of the proposed SLLABC on both classification accuracy and the size of the feature subset.

Improved Reptile Search Optimization Algorithm Using Chaotic Map and Simulated Annealing for Feature Selection in Medical Field

The increased volume of medical datasets has produced high dimensional features, negatively affecting machine learning (ML) classifiers. In ML, the feature selection process is fundamental for selecting the most relevant features and reducing redundant and irrelevant ones. The optimization algorithms demonstrate its capability to solve feature selection problems. Reptile Search Algorithm (RSA) is a new nature-inspired optimization algorithm that stimulates Crocodiles’ encircling and hunting behavior. The unique search of the RSA algorithm obtains promising results compared to other optimization algorithms. However, when applied to high-dimensional feature selection problems, RSA suffers from population diversity and local optima limitations. An improved metaheuristic optimizer, namely the Improved Reptile Search Algorithm (IRSA), is proposed to overcome these limitations and adapt the RSA to solve the feature selection problem. Two main improvements adding value to the standard RSA; the first improvement is to apply the chaos theory at the initialization phase of RSA to enhance its exploration capabilities in the search space. The second improvement is to combine the Simulated Annealing (SA) algorithm with the exploitation search to avoid the local optima problem. The IRSA performance was evaluated over 20 medical benchmark datasets from the UCI machine learning repository. Also, IRSA is compared with the standard RSA and state-of-the-art optimization algorithms, including Particle Swarm Optimization (PSO), Genetic Algorithm (GA), Grasshopper Optimization algorithm (GOA) and Slime Mould Optimization (SMO). The evaluation metrics include the number of selected features, classification accuracy, fitness value, Wilcoxon statistical test (p-value), and convergence curve. Based on the results obtained, IRSA confirmed its superiority over the original RSA algorithm and other optimized algorithms on the majority of the medical datasets.

Two-stage improved Grey Wolf optimization algorithm for feature selection on high-dimensional classification

In recent years, evolutionary algorithms have shown great advantages in the field of feature selection because of their simplicity and potential global search capability. However, most of the existing feature selection algorithms based on evolutionary computation are wrapper methods, which are computationally expensive, especially for high-dimensional biomedical data. To significantly reduce the computational cost, it is essential to study an effective evaluation method. In this paper, a two-stage improved gray wolf optimization (IGWO) algorithm for feature selection on high-dimensional data is proposed. In the first stage, a multilayer perceptron (MLP) network with group lasso regularization terms is first trained to construct an integer optimization problem using the proposed algorithm for pre-selection of features and optimization of the hidden layer structure. The dataset is compressed using the feature subset obtained in the first stage. In the second stage, a multilayer perceptron network with group lasso regularization terms is retrained using the compressed dataset, and the proposed algorithm is employed to construct the discrete optimization problem for feature selection. Meanwhile, a rapid evaluation strategy is constructed to mitigate the evaluation cost and improve the evaluation efficiency in the feature selection process. The effectiveness of the algorithm was analyzed on ten gene expression datasets. The experimental results show that the proposed algorithm not only removes almost more than 95.7% of the features in all datasets, but also has better classification accuracy on the test set. In addition, the advantages of the proposed algorithm in terms of time consumption, classification accuracy and feature subset size become more and more prominent as the dimensionality of the feature selection problem increases. This indicates that the proposed algorithm is particularly suitable for solving high-dimensional feature selection problems.

Adaptive crossover operator based multi-objective binary genetic algorithm for feature selection in classification

Feature selection is a key pre-processing technique for classification which aims at removing irrelevant or redundant features from a given dataset. Generally speaking, feature selection can be considered as a multi-objective optimization problem, i.e, removing number of features and improving the classification accuracy. Genetic algorithms (GAs) have been widely used for feature selection problems. The crossover operator, as an important technique to search for new solutions in GAs, has a strong impact on the final optimization results. However, many crossover operators are problem-dependent and have different search abilities. Thus, it is a challenge to select the most efficient one to solve different feature selection problems, especially when the nature of feature selection problems is unknown in advance. In order to overcome this challenge, in this paper, a multi-objective binary genetic algorithm integrating an adaptive operator selection mechanism (MOBGA-AOS) is proposed. In MOBGA-AOS, five crossover operators with different search characteristics are used. Each of them is assigned a probability based on the performance in the evolution process. In different phases of evolution, the proper crossover operator is selected by roulette wheel selection according to the probabilities to produce new solutions for the next generation. The proposed algorithm is compared with five well-known evolutionary multi-objective algorithms on ten datasets. The experimental results reveal that MOBGA-AOS is capable of removing a large amount of features while ensuring a small classification error. Moreover, it obtains prominent advantages on large-scale datasets, which demonstrates that MOBGA-AOS is competent to solve high-dimensional feature selection problems.

Feature selection using bare-bones particle swarm optimization with mutual information

Feature selection (FS) is an important data processing method in pattern recognition and data mining. Due to not considering characteristics of the FS problem itself, traditional particle update mechanisms and swarm initialization strategies adopted in most particle swarm optimization (PSO) limit their performance on dealing with high-dimensional FS problems. Focused on it, this paper proposes a novel feature selection algorithm based on bare bones PSO (BBPSO) with mutual information. Firstly, an effective swarm initialization strategy based on label correlation is developed, making full use of the correlation between features and class labels to accelerate the convergence of swarm. Then, in order to enhance the exploitation performance of the algorithm, two local search operators, i.e., the supplementary operator and the deletion operator, are developed based on feature relevance-redundancy. Furthermore, an adaptive flip mutation operator is designed to help particles jump out of local optimal solutions. We apply the proposed algorithm to typical datasets based on the K-Nearest Neighbor classifier (K-NN), and compare it with eleven state-of-the-art algorithms, SFS, PTA, SGA, BPSO, PSO(4-2), HPSO-LS, Binary BPSO, NaFA, IBFA, KPLS-mRMR and SMBA-CSFS. The experimental results show that the proposed algorithm can achieve a feature subset with better performance, and is a highly competitive FS algorithm.

Feature selection based on hybridization of genetic algorithm and competitive swarm optimizer

Feature selection is one of the hottest machine learning topics in recent years. The main purposes of it are to simplify the original model, improve the readability of the model, and prevent over-fitting by searching for a suitable subset of features. There are many methods for this problem, including evolutionary algorithms and particle swarm optimization. Among them, the competitive swarm optimizer is a new optimization algorithm proposed in recent years, which is based on particle swarm optimization algorithm, and has achieved good results in high-dimensional feature selection problems, but it also has the problems of high computation time cost and easily being premature. Aiming at these problems, this paper proposes to add the crossover operator and mutation operator in the genetic algorithm to the competitive swarm optimization, so as to improve the generation speed of new individuals in the algorithm and prevent premature population. After testing on UC Irvine Machine Learning Repository, the new algorithm not only improves the computational efficiency, but also avoids the problem that the competitive swarm optimization algorithm is easy to fall into the local optimum, which greatly improves the calculation effect.

Dimensionality reduction in evolutionary algorithms-based feature selection for motor imagery brain-computer interface

For the classification of motor imagery brain-computer interface (BCI) based on electroencephalography (EEG), appropriate features are crucial to obtain a high classification accuracy. Considering the characteristics of the EEG signals, the time-frequency-space three-dimensional features are extracted. Due to a considerable number of the extracted features, the performance of a classifier will degrade. Therefore, it is necessary to implement feature selection. However, existing feature selection methods are easy to fall into a local optimum of a high-dimensional feature selection problem. In this paper, a dimensionality reduction mechanism (called DimReM) is proposed, which gradually reduces the dimension of the search space by removing some unimportant features. In principle, DimReM transforms a high-dimensional feature selection problem into a low-dimensional one. DimReM does not introduce any additional parameters and its implementation is simple. To verify its effectiveness, DimReM is combined with different evolutionary algorithms and different classifiers to select features on various kinds of datasets. Compared with evolutionary algorithms without dimensionality reduction, their augmented versions equipped with DimReM can find feature subsets with higher classification accuracies while smaller numbers of selected features.

A multi-objective feature selection method based on bacterial foraging optimization

Feature selection plays an important role in data preprocessing. The aim of feature selection is to recognize and remove redundant or irrelevant features. The key issue is to use as few features as possible to achieve the lowest classification error rate. This paper formulates feature selection as a multi-objective problem. In order to address feature selection problem, this paper uses the multi-objective bacterial foraging optimization algorithm to select the feature subsets and k-nearest neighbor algorithm as the evaluation algorithm. The wheel roulette mechanism is further introduced to remove duplicated features. Four information exchange mechanisms are integrated into the bacteria-inspired algorithm to avoid the individuals getting trapped into the local optima so as to achieve better results in solving high-dimensional feature selection problem. On six small datasets and ten high-dimensional datasets, comparative experiments with different conventional wrapper methods and several evolutionary algorithms demonstrate the superiority of the proposed bacteria-inspired based feature selection method.

Recovery of weak signal in high dimensional linear regression by data perturbation

How to recover weak signals (i.e., small nonzero regression coefficients) is a difficult task in high dimensional feature selection problems. Both convex and nonconvex regularization methods fail to fully recover the true model whenever there exist strong columnwise correlations in design matrices or small nonzero coefficients below some threshold. To address the two challenges, we propose a procedure, Perturbed LASSO (PLA), that weakens correlations in the design matrix and strengthens signals by adding random perturbations to the design matrix. Moreover, a quantitative relationship between the selection accuracy and computing cost of PLA is derived. We theoretically prove and demonstrate using simulations that PLA substantially improves the chance of recovering weak signals and outperforms comparable methods at a limited cost of computation.