Traditional Feature Selection Methods Research Articles

Enhancing Gearbox Fault Diagnosis through Advanced Feature Engineering and Data Segmentation Techniques

Efficient gearbox fault diagnosis is crucial for the cost-effective maintenance and reliable operation of rotating machinery. Despite extensive research, effective fault diagnosis remains challenging due to the multitude of features available for classification. Traditional feature selection methods often fail to achieve optimal performance in fault classification tasks. This study introduces diverse ranking methods for selecting the relevant features and utilizes data segmentation techniques such as sliding, windowing, and bootstrapping to strengthen predictive model performance and scalability. A comparative analysis of these methods was conducted to identify the potential causes and future solutions. An evaluation of the impact of enhanced feature engineering and data segmentation on predictive maintenance in gearboxes revealed promising outcomes, with decision trees, SVM, and KNN models outperforming others. Additionally, within a fully connected network, windowing emerged as a more robust and efficient segmentation method compared to bootstrapping. Further research is necessary to assess the performance of these techniques across diverse datasets and applications, offering comprehensive insights for future studies in fault diagnosis and predictive maintenance.

Comparative Performance Analysis of Metaheuristic Feature Selection Methods for Speech Emotion Recognition

Abstract Emotion recognition systems from speech signals are realized with the help of acoustic or spectral features. Acoustic analysis is the extraction of digital features from speech files using digital signal processing methods. Another method is the analysis of time-frequency images of speech using image processing. The size of the features obtained by acoustic analysis is in the thousands. Therefore, classification complexity increases and causes variation in classification accuracy. In feature selection, features unrelated to emotions are extracted from the feature space and are expected to contribute to the classifier performance. Traditional feature selection methods are mostly based on statistical analysis. Another feature selection method is the use of metaheuristic algorithms to detect and remove irrelevant features from the feature set. In this study, we compare the performance of metaheuristic feature selection algorithms for speech emotion recognition. For this purpose, a comparative analysis was performed on four different datasets, eight metaheuristics and three different classifiers. The results of the analysis show that the classification accuracy increases when the feature size is reduced. For all datasets, the highest accuracy was achieved with the support vector machine. The highest accuracy for the EMO-DB, EMOVA, eNTERFACE’05 and SAVEE datasets is 88.1%, 73.8%, 73.3% and 75.7%, respectively.

An improved binary particle swarm optimization algorithm for clinical cancer biomarker identification in microarray data

Background and ObjectiveThe limited number of samples and high-dimensional features in microarray data make selecting a small number of features for disease diagnosis a challenging problem. Traditional feature selection methods based on evolutionary algorithms are difficult to search for the optimal set of features in a limited time when dealing with the high-dimensional feature selection problem. New solutions are proposed to solve the above problems. MethodsIn this paper, we propose a hybrid feature selection method (C-IFBPFE) for biomarker identification in microarray data, which combines clustering and improved binary particle swarm optimization while incorporating an embedded feature elimination strategy. Firstly, an adaptive redundant feature judgment method based on correlation clustering is proposed for feature screening to reduce the search space in the subsequent stage. Secondly, we propose an improved flipping probability-based binary particle swarm optimization (IFBPSO), better applicable to the binary particle swarm optimization problem. Finally, we also design a new feature elimination (FE) strategy embedded in the binary particle swarm optimization algorithm. This strategy gradually removes poorer features during iterations to reduce the number of features and improve accuracy. ResultsWe compared C-IFBPFE with other published hybrid feature selection methods on eight public datasets and analyzed the impact of each improvement. The proposed method outperforms other current state-of-the-art feature selection methods in terms of accuracy, number of features, sensitivity, and specificity. The ablation study of this method validates the efficacy of each component, especially the proposed feature elimination strategy significantly improves the performance of the algorithm. ConclusionsThe hybrid feature selection method proposed in this paper helps address the issue of high-dimensional microarray data with few samples. It can select a small subset of features and achieve high classification accuracy on microarray datasets. Additionally, independent validation of the selected features shows that those chosen by C-IFBPFE have strong correlations with disease phenotypes and can identify important biomarkers from data related to biomedical problems.

Improved discrimination of COVID-19 based on data enhancement technology and an information balance feature selection (INB) method

The coronavirus disease (COVID-19) ravaged the world in late 2019 and posed a serious threat to human life and property destruction on a global scale. In this paper, the Wasserstein generative adversarial network with gradient penalty (WGAN-GP) method was selected for balancing the data sample, and an information balance feature selection (INB) method was first proposed to realize the accurate discrimination of COVID-19 saliva samples based on the attenuated total reflection Fourier transform infrared (ATR-FTIR) spectroscopy. The results of the experiment showed that the INB method obtained higher classification accuracy than the traditional feature selection methods in both the original spectrum and the second-order derivative spectrum, especially in the second-order derivative spectrum where all the indexes reached about 85 %. In addition, the combination of WGAN_GP data augmentation and the INB method resulted in an accuracy of 88.7 % for the original spectrum and even 90.6 % for the second-order derivative spectrum. According to these findings, classification research using the WGAN_GP data enhancement model may increase classification accuracy. Additionally, the ability to successfully separate COVID-19 indicates that the INB method to identify spectral data features is a workable method, which also offers a fresh viewpoint on feature selection.

Enhancing Default Prediction in P2P Lending using Random Forest and Grey Wolf Optimization-based Feature Selection

Online lending services such as Peer to Peer (P2P) loans provide convenience for lenders to transact directly without involving banks as intermediaries. Identifying potential loan recipients who are at risk of default is a crucial step in preventing financial losses, as lenders are responsible for default risk. However, predicting default risk becomes a challenge when P2P lending datasets have various complex features. Some features in P2P lending are redundant, while others do not significantly contribute to an effective solution. Therefore, feature selection is an important process to choose a relevant subset of features from input or target data. Traditional feature selection methods often fail to provide optimal results. A better approach is to use heuristic search algorithms capable of finding suboptimal feature subsets. We employ the Grey Wolf Optimization (GWO) technique, inspired by the hierarchy of leadership and grey wolf hunting mechanisms. Combined with Random Forest (RF), which has limitations in classifying data with very high dimensions, our GWO+RF combination has proven to enhance classification performance better than previous research. It achieves an accuracy score of 97.31%, compared to previous research with scores of only 67.72% for RBM+RF, 64% for Binary PSO+ERT, and 92% for GA+RF.

Evaluation of deep learning-based feature selection for single-cell RNA sequencing data analysis

BackgroundFeature selection is an essential task in single-cell RNA-seq (scRNA-seq) data analysis and can be critical for gene dimension reduction and downstream analyses, such as gene marker identification and cell type classification. Most popular methods for feature selection from scRNA-seq data are based on the concept of differential distribution wherein a statistical model is used to detect changes in gene expression among cell types. Recent development of deep learning-based feature selection methods provides an alternative approach compared to traditional differential distribution-based methods in that the importance of a gene is determined by neural networks.ResultsIn this work, we explore the utility of various deep learning-based feature selection methods for scRNA-seq data analysis. We sample from Tabula Muris and Tabula Sapiens atlases to create scRNA-seq datasets with a range of data properties and evaluate the performance of traditional and deep learning-based feature selection methods for cell type classification, feature selection reproducibility and diversity, and computational time.ConclusionsOur study provides a reference for future development and application of deep learning-based feature selection methods for single-cell omics data analyses.

Feature selection under budget constraint in medical applications: analysis of penalized empirical risk minimization methods

Feature selection is a crucial step when building supervised predictive models. In many medical applications, features are associated with costs. For example, the diagnostic value extracted by a clinical test is associated with its own cost. Costs can also refer to a non-financial aspects, such as a decision between an invasive exploratory surgery and a simple blood test. Traditional feature selection methods, which ignore costs, aim to choose a subset of features that maximize the accuracy of the corresponding model. However, such a model can be impractical as the total cost of making a prediction may exceed the assumed user-specified budget. In cost-constrained methods, it is necessary to take into account both the relevance of the feature and its cost. We focus on embedded feature selection methods based on a very general penalized empirical risk minimization framework that includes various loss functions. The most natural ℓ0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _0$$\\end{document}-type penalty is computationally intractable and therefore we analyze other penalties such as: the cost-sensitive lasso and adaptive lasso, non-convex penalties and the method based on knockoffs. The experiments performed on real medical datasets, including large database MIMIC, indicate that non-convex penalties give promising results, in particular they allow to achieve high accuracy, especially when the assumed budget is low. Our model achieved AUC 0.88 for the MIMIC-II dataset in which we predict the occurrence of liver diseases based on clinical features with a budget equal to (5%) of the cost of all available features, which is significantly better than the AUC for the traditional method. Moreover, the fraction of feature cost wasted for noisy features in our method is usually lower than for cost-sensitive lasso.

Intelligent classification and analysis of regional landforms based on automatic feature selection

AbstractTerrain features are an important basis for realizing high‐precision landform classification, and feature selection is a key step of machine learning and knowledge mining. However, the feature selection process is facing challenges due to the multidimensionality and correlation of multisource terrain feature datasets and factors. Traditional feature selection methods lack enough consideration for the interpretability and transparency of feature factors, but the transparent decision‐making process of feature selection precisely determines the modelling effect and the reliability of model application results. Current research urgently needs to work out the black holes of visual representation during feature selection. In the process of intelligent landform classification, multiple and effective terrain feature is an essential factor in enhancing the performance and generalisation ability of the network. Therefore, we initially selected 40 terrain feature parameters, including basic terrain factors and digital elevation model (DEM) terrain textures, to calculate the feature contribution degree and sort the parameter importance based on the SHapley Additive exPlanations (SHAP) method, then reserved 10%, 20%, 30%, 40% and 50% terrain features in turn for constructing the landform classification dataset. Because the traditional UNet network cannot completely capture abrupt landform features, the convolutional block attention module (CBAM) was integrated into the UNet, and a deep learning model was established for the fine‐grained classification of regional landforms. Considering the calculation rate, even though there are large regional spatial differences and genetic mechanisms, it is appropriate to retain 20% of terrain features for intelligent landform classification. The classification accuracy of typical regions, namely, the Hanzhong Basin, North China Plain, Yunnan–Guizhou Plateau and Tibetan Plateau, reached 98.76%, 97.36%, 96.3% and 92.78%, respectively, and what's more, some accuracies went up to a higher level under other feature combinations. Meanwhile, given the different feature combinations corresponding to regional landform types, the combinative stability and spatial orderliness characteristics were explored to explain the accuracy variation trend.

Robust Feature Selection Method Based on Joint L2,1 Norm Minimization for Sparse Regression

Feature selection methods are widely used in machine learning tasks to reduce the dimensionality and improve the performance of the models. However, traditional feature selection methods based on regression often suffer from a lack of robustness and generalization ability and are easily affected by outliers in the data. To address this problem, we propose a robust feature selection method based on sparse regression. This method uses a non-square form of the L2,1 norm as both the loss function and regularization term, which can effectively enhance the model’s resistance to outliers and achieve feature selection simultaneously. Furthermore, to improve the model’s robustness and prevent overfitting, we add an elastic variable to the loss function. We design two efficient convergent iterative processes to solve the optimization problem based on the L2,1 norm and propose a robust joint sparse regression algorithm. Extensive experimental results on three public datasets show that our feature selection method outperforms other comparison methods.

A quantum feature selection framework via ground state preparation

Traditional feature selection methods face the challenges of increasing time complexity and local optima. In previous works, many classical feature selection methods were accelerated through quantum algorithms. However, these approaches still inherit the constraints of these classical methods as they do not address the issue of local minima. Here, we propose a novel quantum feature selection framework based on the classifier’s result, which utilizes Hamiltonian encoding and a ground state preparation algorithm. Numerical experiments are conducted on real-world datasets from the finance and medicine domains. Moreover, the results demonstrate that the proposed method produces the same or better classification accuracy on the classifier than the original data without feature selection. Overall, our approach presents a promising solution to feature selection using quantum computing.

A data-driven evidential regression model for building hourly energy consumption prediction with feature selection and parameters learning

Building energy consumption prediction is critical for building energy management and energy policy formulation, and its inherent uncertainty can significantly affect the utilization of current energy market benefits for market participants. To capture the uncertainty of energy consumption and enhance the predictive capability of the model, in this study, a data-driven evidential regression (EVREG) model with integrated feature selection function is proposed based on Dempster–Shafer theory and mutual information, which can perform point prediction and interval prediction for building hourly energy consumption to describe its fluctuation and uncertainty. Different from the traditional EVREG model, this method enables simultaneous feature selection and model parameters learning instead of treating feature selection as a separate data pre-processing step. Specifically, an evaluation function is defined to describe the significance of a candidate feature, taking into account the predictive power of regression model and the redundancy between the candidate feature and already selected features. According to a search strategy, features with high significance are selected to minimize the objective function. A real dataset from a commercial building is used to evaluate the performance of the proposed method. The results demonstrate that the proposed method can select fewer features while achieving better prediction performance compared to traditional feature selection methods used as data preprocessing. The proposed method also achieves better or comparable performance compared to commonly applied point prediction and interval prediction methods.

MiRDM-rfGA: Genetic algorithm-based identification of a miRNA set for detecting type 2 diabetes

BackgroundType 2 diabetes mellitus (T2DM) affects approximately 451 million adults globally. In this study, we identified the optimal combination of marker candidates for detecting T2DM using miRNA-Seq data from 95 samples including T2DM and healthy individuals.MethodsWe utilized the genetic algorithm (GA) in the discovery of an optimal miRNA biomarker set. We discovered miRNA subsets consisting of three miRNAs for detecting T2DM by random forest-based GA (miRDM-rfGA) as a feature selection algorithm and created six GA parameter settings and three settings using traditional feature selection methods (F-test and Lasso). We then evaluated the prediction performance to detect T2DM in the miRNA subsets derived from each setting.ResultsThe miRNA subset in setting 5 using miRDM-rfGA performed the best in detecting T2DM (mean AUROC = 0.92). Target mRNA identification and functional enrichment analysis of the best miRNA subset (hsa-miR-125b-5p, hsa-miR-7-5p, and hsa-let-7b-5p) validated that this combination was involved in T2DM. We also confirmed that the targeted genes were negatively correlated with the clinical variables related to T2DM in the BxD mouse genetic reference population database.ConclusionsUsing GA in miRNA-Seq data, we identified the optimal miRNA biomarker set for T2DM detection. GA can be a useful tool for biomarker discovery and drug-target identification.

Application of the Fuzzy Approach for Evaluating and Selecting Relevant Objects, Features, and Their Ranges.

Relevant attribute selection in machine learning is a key aspect aimed at simplifying the problem, reducing its dimensionality, and consequently accelerating computation. This paper proposes new algorithms for selecting relevant features and evaluating and selecting a subset of relevant objects in a dataset. Both algorithms are mainly based on the use of a fuzzy approach. The research presented here yielded preliminary results of a new approach to the problem of selecting relevant attributes and objects and selecting appropriate ranges of their values. Detailed results obtained on the Sonar dataset show the positive effects of this approach. Moreover, the observed results may suggest the effectiveness of the proposed method in terms of identifying a subset of truly relevant attributes from among those identified by traditional feature selection methods.

Lithological Classification by Hyperspectral Images Based on a Two-Layer XGBoost Model, Combined with a Greedy Algorithm

Lithology classification is important in mineral resource exploration, engineering geological exploration, and disaster monitoring. Traditional laboratory methods for the qualitative analysis of rocks are limited by sampling conditions and analytical techniques, resulting in high costs, low efficiency, and the inability to quickly obtain large-scale geological information. Hyperspectral remote sensing technology can classify and identify lithology using the spectral characteristics of rock, and is characterized by fast detection, large coverage area, and environmental friendliness, which provide the application potential for lithological mapping at a large regional scale. In this study, ZY1-02D hyperspectral images were used as data sources to construct a new two-layer extreme gradient boosting (XGBoost) lithology classification model based on the XGBoost decision tree and an improved greedy search algorithm. A total of 153 spectral bands of the preprocessed hyperspectral images were input into the first layer of the XGBoost model. Based on the tree traversal structural characteristics of the leaf nodes in the XGBoost model, three built-in XGBoost importance indexes were split and combined. The improved greedy search algorithm was used to extract the spectral band variables, which were imported into the second layer of the XGBoost model, and the bat algorithm was used to optimize the modeling parameters of XGBoost. The extraction model of rock classification information was constructed, and the classification map of regional surface rock types was drawn. Field verification was performed for the two-layer XGBoost rock classification model, and its accuracy and reliability were evaluated based on four indexes, namely, accuracy, precision, recall, and F1 score. The results showed that the two-layer XGBoost model had a good lithological classification effect, robustness, and adaptability to small sample datasets. Compared with the traditional machine learning model, the two-layer XGBoost model shows superior performance. The accuracy, precision, recall, and F1 score of the verification set were 0.8343, 0.8406, 0.8350, and 0.8157, respectively. The variable extraction ability of the constructed two-layer XGBoost model was significantly improved. Compared with traditional feature selection methods, the GREED-GFC method, when applied to the two-layer XGBoost model, contributes to more stable rock classification performance and higher lithology prediction accuracy, and the smallest number of extracted features. The lithological distribution information identified by the model was in good agreement with the lithology information verified in the field.

Clustering-Fusion Feature Selection Method in Identifying Major Depressive Disorder Based on Resting State EEG Signals

Depression is a heterogeneous syndrome with certain individual differences among subjects. Exploring a feature selection method that can effectively mine the commonness intra-groups and the differences inter-groups in depression recognition is therefore of great significance. This study proposed a new clustering-fusion feature selection method. Hierarchical clustering (HC) algorithm was used to capture the heterogeneity distribution of subjects. Average and similarity network fusion (SNF) algorithms were adopted to characterize the brain network atlas of different populations. Differences analysis was also utilized to obtain the features with discriminant performance. Experiments showed that compared with traditional feature selection methods, HCSNF method yielded the optimal classification results of depression recognition in both sensor and source layers of electroencephalography (EEG) data. Especially in the beta band of EEG data at sensor layer, the classification performance was improved by more than 6%. Moreover, the long-distance connections between parietal-occipital lobe and other brain regions not only have high discriminative power, but also significantly correlate with depressive symptoms, indicating the important role of these features in depression recognition. Therefore, this study may provide methodological guidance for the discovery of reproducible electrophysiological biomarkers and new insights into common neuropathological mechanisms of heterogeneous depression diseases.

A two-stage hybrid biomarker selection method based on ensemble filter and binary differential evolution incorporating binary African vultures optimization

BackgroundIn the field of genomics and personalized medicine, it is a key issue to find biomarkers directly related to the diagnosis of specific diseases from high-throughput gene microarray data. Feature selection technology can discover biomarkers with disease classification information.ResultsWe use support vector machines as classifiers and use the five-fold cross-validation average classification accuracy, recall, precision and F1 score as evaluation metrics to evaluate the identified biomarkers. Experimental results show classification accuracy above 0.93, recall above 0.92, precision above 0.91, and F1 score above 0.94 on eight microarray datasets.MethodThis paper proposes a two-stage hybrid biomarker selection method based on ensemble filter and binary differential evolution incorporating binary African vultures optimization (EF-BDBA), which can effectively reduce the dimension of microarray data and obtain optimal biomarkers. In the first stage, we propose an ensemble filter feature selection method. The method combines an improved fast correlation-based filter algorithm with Fisher score. obviously redundant and irrelevant features can be filtered out to initially reduce the dimensionality of the microarray data. In the second stage, the optimal feature subset is selected using an improved binary differential evolution incorporating an improved binary African vultures optimization algorithm. The African vultures optimization algorithm has excellent global optimization ability. It has not been systematically applied to feature selection problems, especially for gene microarray data. We combine it with a differential evolution algorithm to improve population diversity.ConclusionCompared with traditional feature selection methods and advanced hybrid methods, the proposed method achieves higher classification accuracy and identifies excellent biomarkers while retaining fewer features. The experimental results demonstrate the effectiveness and advancement of our proposed algorithmic model.

Deep feature screening: Feature selection for ultra high-dimensional data via deep neural networks

The applications of traditional statistical feature selection methods to high-dimension, low-sample-size data often struggle and encounter challenging problems, such as overfitting, curse of dimensionality, computational infeasibility, and strong model assumptions. In this paper, we propose a novel two-step nonparametric approach called Deep Feature Screening (DeepFS) that can overcome these problems and identify significant features with high precision for ultra high-dimensional, low-sample-size data. This approach first extracts a low-dimensional representation of input data and then applies feature screening on the original input feature space based on multivariate rank distance correlation recently developed by Deb and Sen (2021). This approach combines the strengths of both deep neural networks and feature screening, thereby having the following appealing features in addition to its ability of handling ultra high-dimensional data with small number of samples: (1) it is model free and distribution free; (2) it can be used for both supervised and unsupervised feature selection; and (3) it is capable of recovering the original input data. The superiority of DeepFS is demonstrated via extensive simulation studies and real data analyses.

Semisupervised Bacterial Heuristic Feature Selection Algorithm for High-Dimensional Classification with Missing Labels

Feature selection is a crucial method for discovering relevant features in high-dimensional data. However, most studies primarily focus on completely labeled data, ignoring the frequent occurrence of missing labels in real-world problems. To address high-dimensional and label-missing problems in data classification simultaneously, we proposed a semisupervised bacterial heuristic feature selection algorithm. To track the label-missing problem, a k-nearest neighbor semisupervised learning strategy is designed to reconstruct missing labels. In addition, the bacterial heuristic algorithm is improved using hierarchical population initialization, dynamic learning, and elite population evolution strategies to enhance the search capacity for various feature combinations. To verify the effectiveness of the proposed algorithm, three groups of comparison experiments based on eight datasets are employed, including two traditional feature selection methods, four bacterial heuristic feature selection algorithms, and two swarm-based heuristic feature selection algorithms. Experimental results demonstrate that the proposed algorithm has obvious advantages in terms of classification accuracy and selected feature numbers.

ECM-EFS: An ensemble feature selection based on enhanced co-association matrix

Currently, feature selection faces a huge challenge that no single feature selection method can effectively deal with various data sets for all real cases. Ensemble learning is a potential promising solution to address this problem. We propose an ensemble feature selection method based on enhanced co-association matrix (ECM-EFS). Positive-co-association matrix (PCM), negative-co-association matrix (NCM), and relative-co-association matrix (RCM) are first introduced to discover the relationship among features by ensembling the results in multiple feature selection methods. To further produce a more stable feature selection result, “Feature Kernel” is also introduced and used as a starting point for feature selection. Comparative experiments with four state-of-the-art methods have confirmed that the ECM-EFS can provide more robust results. Moreover, compared with traditional ensemble feature selection methods, our method can compensate information loss and reduce computational cost significantly.

Fast orthogonal locality-preserving projections for unsupervised feature selection

Graph-based sparsity learning is one of the most successful unsupervised feature selection methods that has been widely adopted in many real-world applications. However, traditional graph-based unsupervised feature selection methods have several drawbacks: (1) being time-consuming and unable to deal with large-scale problems; (2) having difficulty tuning the regularization parameter with the sparsity regularization term; and (3) being unable to find explicit solutions owing to the limitation of sparsity, that is, feature selection with the ℓ2,1-norm constrained problem. Thus, this paper proposes OLPPFS, a method to preserve the local geometric structure within the feature subspace by imposing the ℓ2,0-norm constraint. First, the linear mapping capability of the proposed model is enhanced using locality-preserving projections (LPPs), whichpreserve the local and global geometric manifold structure of the data while enhancing the ability to reconstruct data. Second, the graph-embedding learning method can accelerate the construction of a sparsity affinity graph and describe the intrinsic structure of the dataset well. More importantly, we propose a method for solving a projection matrix with the ℓ2,0-norm constrained, which can accurately select a explicit group of discriminative feature subsets. This method can yield a more accurate sparse projection matrix than the ℓ2,1-norm. We also adopt FOLPPFS, an effective anchor-based strategy to further accelerate our model with two flexible options. Extensive experiments on eight datasets demonstrate that the proposed method is superior to the other methods and can preserve a better local geometric structure of the dataset with less time consumption.