Feature Selection For High-dimensional Data Research Articles

Roulette wheel-based level learning evolutionary algorithm for feature selection of high-dimensional data

Feature selection in high-dimensional data is a large-scale sparse and discrete optimization problem. Most evolutionary algorithms are designed to tackle continuous optimization problems. However, when dealing with high-dimensional feature selection tasks, they often suffer from poor population diversity and are computationally expensive. To address these challenges, this work introduces a roulette wheel-based level learning evolutionary algorithm (RWLLEA). RWLLEA integrates two key components. Firstly, it employs a leveled population mode. Individuals from higher levels provide guidance to those at lower levels during the evolutionary process, thereby exploring the potential combinatorial effects among features. Secondly, recognizing the characteristics associated with the high-dimensional feature selection task, a roulette wheel-based update method is devised to dynamically reduce search space and harmonize the algorithm's exploitation and exploration capacities across different stages. The performance of the proposed method was evaluated by comparing it with six other feature selection techniques across a range of fifteen diverse datasets. The experimental findings demonstrate that the proposed method can achieve a reduced feature set with a shorter runtime and exhibit superior classification accuracy.

Application of non parametric Bayesian methods in high dimensional data

With the development of technology and the widespread collection of data, high-dimensional data analysis has become a research hotspot in many fields. Traditional parameter methods often face problems such as dimensional disasters in high-dimensional data analysis. Non parametric methods have broad application prospects in high-dimensional data because they do not rely on specific parameter distribution assumptions. The Bayesian rule is more suitable for dealing with noise and outliers in high-dimensional data because it takes uncertainty into account. Therefore, it is of great significance to combine non parametric methods with Bayesian methods for application research in high-dimensional data analysis. In this paper, the nonparametric Bayesian method was applied to the analysis of high-dimensional data, and the Dirichlet process Mixture model was used to cluster high-dimensional data. The regression analysis of high-dimensional data was carried out through the prediction model of nonparametric Bayesian regression. In this paper, the nonparametric Bayesian method based on Bayesian sparse linear model was used for feature selection of high-dimensional data. In order to determine the superiority of nonparametric Bayesian methods in high-dimensional data analysis, this paper conducted experiments on nonparametric Bayesian methods and traditional parametric methods in high-dimensional data analysis from five aspects of cluster analysis, classification analysis, regression analysis, feature selection and anomaly detection, and evaluated them through multiple indicators. This article explored the application of non parametric Bayesian methods in high-dimensional data analysis from these aspects through simulation experiments. The experimental results show that the clustering accuracy of the non parametric Bayesian clustering algorithm was 0.93, and the accuracy of the non parametric Bayesian classification algorithm was between 0.93 and 0.99; the coefficient of determination of nonparametric Bayesian regression algorithm was 0.98; the F1 values of non parametric Bayesian methods in anomaly detection ranged from 0.86 to 0.91, which was superior to traditional methods. Non parametric Bayesian methods have broad application prospects in high-dimensional data analysis, and can be applied in multiple fields such as clustering, classification, regression, etc.

Feature selection in high-dimensional data: an enhanced RIME optimization with information entropy pruning and DBSCAN clustering

Feature selection in high-dimensional data: an enhanced RIME optimization with information entropy pruning and DBSCAN clustering

Development of a Java Library with Bacterial Foraging Optimization for Feature Selection of High-Dimensional Data

High-dimensional data allows researchers to conduct comprehensive analyses. However, such data often exhibits characteristics like small sample sizes, class imbalance, and high complexity, posing challenges for classification. One approach employed to tackle high-dimensional data is feature selection. This study uses the Bacterial Foraging Optimization (BFO) algorithm for feature selection. A dedicated BFO Java library is developed to extend the capabilities of WEKA for feature selection purposes. Experimental results confirm the successful integration of BFO. The outcomes of BFO's feature selection are then compared against those of other evolutionary algorithms, namely Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC), and Ant Colony Optimization (ACO). Comparison of algorithms conducted using the same datasets. The experimental results indicate that BFO effectively reduces features while maintaining consistent accuracy. In 4 out of 9 datasets, BFO outperforms other algorithms, showcasing superior processing time performance in 6 datasets. BFO is a favorable choice for selecting features in high-dimensional datasets, providing consistent accuracy and effective processing. The optimal fraction of features in the Ovarian Cancer dataset signifies that the dataset retains a minimal number of selected attributes. Consequently, the learning process gains speed due to the reduced feature set. Remarkably, accuracy substantially increased, rising from 0.868 before feature selection to 0.886 after feature selection. The classification processing time has also been significantly shortened, completing the task in just 0.3 seconds, marking a remarkable improvement from the previous 56.8 seconds.

Enhancing Feature Selection in High-Dimensional Data With Fuzzy Fitness-Integrated Memetic Algorithms

Enhancing Feature Selection in High-Dimensional Data With Fuzzy Fitness-Integrated Memetic Algorithms

A Novel Unsupervised Feature Selection for High-Dimensional Data Based on FCM and k -Nearest Neighbor Rough Sets.

Large amounts of high-dimensional unlabeled data typically contain only a small portion of truly effective information. Consequently, the issue of unsupervised feature selection methods has gained significant attention in research. However, current unsupervised feature selection approaches face limitations when dealing with datasets that exhibit uneven density, and they also require substantial computational time. To address this problem, this research article proposes a feature extraction technique that combines the Fuzzy C-Means (FCM) and k -nearest neighbor rough sets. FCM is a clustering algorithm grounded in fuzzy theory, which takes into account the inherent data structure and the correlations between different features. Consequently, FCM is particularly well-suited for datasets with uneven density. Our proposed method consists of three steps. First, the FCM algorithm is used to cluster the unlabeled data. Second, a measure that evaluates the importance of features is defined and sorted based on the clustering results. Finally, redundant features are filtered using k -nearest neighbor rough sets while retaining important features, significantly reducing the running time. In addition, we designed the feature selection algorithm (KND-UFS) and conducted experiments on 12 public datasets. We compared KND-UFS with eight existing algorithms in terms of running time, classification accuracy, and the number of selected features. The experimental results provided strong evidence supporting the superior performance of the KND-UFS algorithm.

Bird’s Eye View feature selection for high-dimensional data

In machine learning, an informative dataset is crucial for accurate predictions. However, high dimensional data often contains irrelevant features, outliers, and noise, which can negatively impact model performance and consume computational resources. To tackle this challenge, the Bird’s Eye View (BEV) feature selection technique is introduced. This approach is inspired by the natural world, where a bird searches for important features in a sparse dataset, similar to how a bird search for sustenance in a sprawling jungle. BEV incorporates elements of Evolutionary Algorithms with a Genetic Algorithm to maintain a population of top-performing agents, Dynamic Markov Chain to steer the movement of agents in the search space, and Reinforcement Learning to reward and penalize agents based on their progress. The proposed strategy in this paper leads to improved classification performance and a reduced number of features compared to conventional methods, as demonstrated by outperforming state-of-the-art feature selection techniques across multiple benchmark datasets.

Research on Computational Methods and Algorithms for Dimensionality Reduction and Feature Selection in High-Dimensional Data

Research on Computational Methods and Algorithms for Dimensionality Reduction and Feature Selection in High-Dimensional Data

Design of feature selection algorithm for high-dimensional network data based on supervised discriminant projection.

High dimension and complexity of network high-dimensional data lead to poor feature selection effect network high-dimensional data. To effectively solve this problem, feature selection algorithms for high-dimensional network data based on supervised discriminant projection (SDP) have been designed. The sparse representation problem of high-dimensional network data is transformed into an Lp norm optimization problem, and the sparse subspace clustering method is used to cluster high-dimensional network data. Dimensionless processing is carried out for the clustering processing results. Based on the linear projection matrix and the best transformation matrix, the dimensionless processing results are reduced by combining the SDP. The sparse constraint method is used to achieve feature selection of high-dimensional data in the network, and the relevant feature selection results are obtained. The experimental findings demonstrate that the suggested algorithm can effectively cluster seven different types of data and converges when the number of iterations approaches 24. The F1 value, recall, and precision are all kept at high levels. High-dimensional network data feature selection accuracy on average is 96.9%, and feature selection time on average is 65.1 milliseconds. The selection effect for network high-dimensional data features is good.

A high-dimensional feature selection method based on modified Gray Wolf Optimization

For data mining tasks on high-dimensional data, feature selection is a necessary pre-processing stage that plays an important role in removing redundant or irrelevant features and improving classifier performance. The Gray Wolf optimization algorithm is a global search mechanism with promising applications in feature selection, but tends to stagnate in high-dimensional problems with locally optimal solutions. In this paper, a modified gray wolf optimization algorithm is proposed for feature selection of high-dimensional data. The algorithm introduces ReliefF algorithm and Coupla entropy in the initialization process, which effectively improves the quality of the initial population. In addition, modified gray wolf optimization includes two new search strategies: first, a competitive guidance strategy is proposed to update individual positions, which make the algorithm’s search more flexible; second, a differential evolution-based leader wolf enhancement strategy is proposed to find a better position where the leader wolf may exist and replace it, which can prevent the algorithm from falling into local optimum. The results on 10 high-dimensional small-sample gene expression datasets demonstrate that the proposed algorithm selects less than 0.67% of the features, improves the classification accuracy while further reducing the number of features, and obtains very competitive results compared with some advanced feature selection methods. The comprehensive study analysis shows that proposed algorithm better balances the exploration and exploration balance, and the two search strategies are conducive to the improvement of gray wolf optimization search capability.

A filter feature selection for high-dimensional data

In a classification problem, before building a prediction model, it is very important to identify informative features rather than using tens or thousands which may penalize some learning methods and increase the risk of over-fitting. To overcome these problems, the best solution is to use feature selection. In this article, we propose a new filter method for feature selection, by combining the Relief filter algorithm and the multi-criteria decision-making method called TOPSIS (Technique for Order Preference by Similarity to Ideal Solution), we modeled the feature selection task as a multi-criteria decision problem. Exploiting the Relief methodology, a decision matrix is computed and delivered to Technique for Order Preference by Similarity to Ideal Solution in order to rank the features. The proposed method ends up giving a ranking to the features from the best to the mediocre. To evaluate the performances of the suggested approach, a simulation study including a set of experiments and case studies was conducted on three synthetic dataset scenarios. Finally, the obtained results approve the effectiveness of our proposed filter to detect the best informative features.

Framework of Meta-Heuristic Variable Length Searching for Feature Selection in High-Dimensional Data

Feature Selection in High Dimensional Space is a combinatory optimization problem with an NP-hard nature. Meta-heuristic searching with embedding information theory-based criteria in the fitness function for selecting the relevant features is used widely in current feature selection algorithms. However, the increase in the dimension of the solution space leads to a high computational cost and risk of convergence. In addition, sub-optimality might occur due to the assumption of a certain length of the optimal number of features. Alternatively, variable length searching enables searching within the variable length of the solution space, which leads to more optimality and less computational load. The literature contains various meta-heuristic algorithms with variable length searching. All of them enable searching in high dimensional problems. However, an uncertainty in their performance exists. In order to fill this gap, this article proposes a novel framework for comparing various variants of variable length-searching meta-heuristic algorithms in the application of feature selection. For this purpose, we implemented four types of variable length meta-heuristic searching algorithms, namely VLBHO-Fitness, VLBHO-Position, variable length particle swarm optimization (VLPSO) and genetic variable length (GAVL), and we compared them in terms of classification metrics. The evaluation showed the overall superiority of VLBHO over the other algorithms in terms of accomplishing lower fitness values when optimizing mathematical functions of the variable length type.

A Machine Learning-Based Approach to Discriminating Basaltic Tectonic Settings

The geochemical characteristics of magmatic rocks can distinguish the tectonic setting of magma formation and their geochemical signatures are discriminated by using the whole-rock geochemical data. As a new attempt of artificial intelligence technology in geochemistry, the machine learning discrimination method is gradually complementary to the classical discriminative graphical method. However, the feature selection of high-dimensional data and the determination of many unknown parameters are the two main factors affecting the classification accuracy of the algorithm. In this paper, a particle swarm optimized support vector machine (PSO-SVM) model is established to classify the tectonic environments of basaltic rocks in the GEOROC database. The model mainly relies on the powerful search capability of the particle swarm algorithm to find the best parameter combination selected by the SVM based on experience to improve the accuracy. In this study, based on the basalt samples in the database and the confusion matrix, the performance of PSO-SVM model is evaluated by simulation experiments. The results show that the model proposed in this paper is more effective in distinguishing the basaltic tectonic environments, with an accuracy of more than 90%. Therefore, compared with the traditional discriminant map method, the machine learning method based on the fusion of two algorithms performs better in the tectonic environment classification problems.

An improved group teaching optimization algorithm based on local search and chaotic map for feature selection in high-dimensional data

The current study proposes a novel binary group teaching optimization algorithm with local search and chaos mapping (BGTOALC) as a wrapper-based feature selection method to solve high-dimensional feature selection problems. The local search and chaos mapping enhance the performance of the proposed algorithm. Also, two novel binary operators called Binary Teacher Phase Good Group (BTPGG) and Binary Teacher Phase Bad Group (BTPBG) are applied to the teacher’s phase for increasing the exploration and exploitation of the algorithm. Moreover, a new Binary Student Opposition-Based Learning (BSOBL) operator is introduced for the student phase, using an opposition-based strategy to achieve better exploitation. Finally, the teacher allocation phase is designed in a binary manner using the new Mean Binary Select (MBS) operator to increase the algorithm’s convergence rate. Subsequently, two other binary group teaching optimization algorithms, named BGTOAV and BGTOAS, are developed utilizing the S-shaped and V-shaped transfer functions to compare their performance with the BGTOALC algorithm. The proposed approaches are compared to other state-of-the-art binary algorithms on 30 datasets with different dimensions. Different experiments prove that the BGTOALC method outperforms the previous methods in terms of reducing the number of selected features and increasing the accuracy of the machine learning algorithm. Eventually, statistical analyses indicate the superiority of the BGTOALC method in terms of efficiency and convergence rate against other binary metaheuristic algorithms.

A general framework of nonparametric feature selection in high-dimensional data.

Nonparametric feature selection for high-dimensional data is an important and challenging problem in the fields of statistics and machine learning. Most of the existing methods for feature selection focus on parametric or additive models which may suffer from model misspecification. In this paper, we propose a new framework to perform nonparametric feature selection for both regression and classification problems. Under this framework, we learn prediction functions through empirical risk minimization over a reproducing kernel Hilbert space. The space is generated by a novel tensor product kernel, which depends on a set of parameters that determines the importance of the features. Computationally, we minimize the empirical risk with a penalty to estimate the prediction and kernel parameters simultaneously. The solution can be obtained by iteratively solving convex optimization problems. We study the theoretical property of the kernel feature space and prove the oracle selection property and Fisher consistency of our proposed method. Finally, we demonstrate the superior performance of our approach compared to existing methods via extensive simulation studies and applications to two realstudies.

Regression with adaptive lasso and correlation based penalty

Feature selection for high-dimensional data is an important issue in machine learning, pattern recognition and bioinformatics fields. Feature selection algorithms are proposed to select the relevant feature subset from the original features. To adaptively identify the important highly correlated features from high-dimensional data which often beneficial to improve classification accuracy is a challenge. In this paper, we propose a regularized logistic regression with adaptive Lasso and correlation based penalty model to select informative highly correlated features adaptively. To incorporate significance of features into regression model, we first measure significance of each feature based on mutual information, and propose an adaptive weight construction strategy. Based on the adaptive weight construction strategy, the proposed adaptive logistic regression can impose a large amount of penalty on irrelevant features, and thus noise features are easily removed from the model and remain the informative features. The experimental results on the simulation and real-world datasets demonstrate the effectiveness and the superiority the proposed model by comparing it to existing competing regularized logistic regression models.

A Clustering-Guided Integer Brain Storm Optimizer for Feature Selection in High-Dimensional Data

For high-dimensional data with a large number of redundant features, existing feature selection algorithms still have the problem of “curse of dimensionality.” In view of this, the paper studies a new two-phase evolutionary feature selection algorithm, called clustering-guided integer brain storm optimization algorithm (IBSO-C). In the first phase, an importance-guided feature clustering method is proposed to group similar features, so that the search space in the second phase can be reduced obviously. The second phase applies oneself to finding optimal feature subset by using an improved integer brain storm optimization. Moreover, a new encoding strategy and a time-varying integer update method for individuals are proposed to improve the search performance of brain storm optimization in the second phase. Since the number of feature clusters is far smaller than the size of original features, IBSO-C can find an optimal feature subset fast. Compared with several existing algorithms on some real-world datasets, experimental results show that IBSO-C can find feature subset with high classification accuracy at less computation cost.

RHDSI: A novel dimensionality reduction based algorithm on high dimensional feature selection with interactions

Classical statistical learning techniques struggle to perform feature selection in high-dimensional data that includes interaction effects i.e., when independent feature/s influence the effect of another feature on study outcome. Methods like penalized regression and sparse partial least squares regression can help, but penalization restricts the handling of interaction terms. This study proposes a novel Dimensionality Reduction based algorithm on High Dimensional feature Selection with Interactions (RHDSI), a new feature selection method that integrates dimensionality reduction and machine learning. The method can handle high-dimensional data, incorporate interaction terms and perform statistically-interpretable feature selection; and enables existing classical statistical techniques to work on high-dimensional data. RHDSI performs feature selection in three steps. The first step is a coarse feature selection through dimensionality reduction and statistical modeling on multiple resampled datasets and features, along with their interaction terms. The second step uses pooled results for unsupervised statistical learning-based feature refinement. Finally, supervised statistical learning-based feature selection is performed on the refined feature set to identify the final features with interactions. We evaluate the performance of this algorithm on simulated data and real studies. RHDSI shows better or par performance compared to standard feature selection algorithms like LASSO, subset selection, and sparse PLS.

Ensemble learning-based filter-centric hybrid feature selection framework for high-dimensional imbalanced data

In recent years, research on feature selection for high-dimensional imbalanced data has attracted a considerable amount of attention. The filter-wrapper hybrid method, which is a conventional method of feature selection for high-dimensional data, aims to reduce excessive computational time. On the other hand, ensemble learning-based feature selection, even though it has a high level of computational complexity, focuses exclusively on the discovery of robust features. From this perspective, combining these two feature selection methods is not easy. However, a combined method is essential to advancing machine learning research that addresses real-world problems. We propose an filter-centric hybrid method based on ensemble-learning that can select the best feature subset for high-dimensional imbalanced data. The basic concept of the proposed method is to design a feature evaluation scheme based on the filter method and to apply ensemble learning with reasonable computational time. To achieve this objective, our innovative method utilizes predictions produced by multiple classifiers as inputs of the feature evaluation function. As a result, it can reflect the predictive performance of the classifiers and overcome the low performance of selected features by filter methods. In addition, it can find robust features simultaneously. To demonstrate the superiority of the proposed method, we perform various experiments using 14 experimental datasets that consist of low-dimensional balanced, high-dimensional balanced, and high-dimensional imbalanced datasets. Finally, we compare the proposed method with state-of-the-art feature selection methods.

An Asymmetric Chaotic Competitive Swarm Optimization Algorithm for Feature Selection in High-Dimensional Data

This paper presents a method for feature selection in a high-dimensional classification context. The proposed method finds a candidate solution based on quality criteria using subset searching. In this study, the competitive swarm optimization (CSO) algorithm was implemented to solve feature selection problems in high-dimensional data. A new asymmetric chaotic function was proposed and used to generate the population and search for a CSO solution. Its histogram is right-skewed. The proposed method is named an asymmetric chaotic competitive swarm optimization algorithm (ACCSO). According to the asymmetrical property of the proposed chaotic map, ACCSO prefers zero than one. Therefore, the solution is very compact and can achieve high classification accuracy with a minimal feature subset for high-dimensional datasets. The proposed method was evaluated on 12 datasets, with dimensions ranging from 4 to 10,304. ACCSO was compared to the original CSO algorithm and other metaheuristic algorithms. Experimental results show that the proposed method can increase accuracy and it reduces the number of selected features. Compared to different optimization algorithms with other wrappers, the proposed method exhibits excellent performance.