Recursive Feature Elimination Strategy Research Articles

A SYSTEMATIC LITERATURE REVIEW: RECURSIVE FEATURE ELIMINATION ALGORITHMS

Recursive feature elimination (RFE) is a feature selection algorithm that works by gradually eliminating unimportant features. RFE has become a popular method for feature selection in various machine learning applications, such as classification and prediction. However, there is no systematic literature review (SLR) that discusses recursive feature elimination algorithms. This article conducts a SLR on RFE algorithms. The goal is to provide an overview of the current state of the RFE algorithm. This SLR uses IEEE Xplore, ScienceDirect, Springer, and Scopus (publish and publish) databases from 2018 to 2023. This SLR received 76 relevant papers with 49% standard RFEs, 43% strategy RFEs, and 8% modified RFEs. Research using RFE continues to increase every year, from 2018 to 2023. The feature selection method used simultaneously or for comparison is based on a filter approach, namely Pearson correlation, and an embedded approach, namely random forest. The most widely used machine learning algorithms are support vector machines and random forests, with 19.5% and 16.7%, respectively. Strategy RFE and modified RFE can be referred to as hybrid RFEs. Based on relevant papers, it is found that the RFE strategy is broadly divided into two categories: using RFE after other feature selection methods and using RFE simultaneously with other methods. Modification of the RFE is done by modifying the flow of the RFE. The modification process is divided into two categories: before the process of calculating the smallest weight criteria and after calculating the smallest weight criteria. Calculating the smallest weight criteria in this RFE modification is still a challenge at this time to obtain optimal results.

A Novel Multiobjective Genetic Programming Approach to High-Dimensional Data Classification.

The development of data sensing technology has generated a vast amount of high-dimensional data, posing great challenges for machine learning models. Over the past decades, despite demonstrating its effectiveness in data classification, genetic programming (GP) has still encountered three major challenges when dealing with high-dimensional data: 1) solution diversity; 2) multiclass imbalance; and 3) large feature space. In this article, we have developed a problem-specific multiobjective GP framework (PS-MOGP) for handling classification tasks with high-dimensional data. To reduce the large solution space caused by high dimensionality, we incorporate the recursive feature elimination strategy based on mining the archive of evolved GP solutions. A progressive domination Pareto archive evolution strategy (PD-PAES), which optimizes the objectives in a specific order according to their objectives, is proposed to evaluate the GP individuals and maintain a better diversity of solutions. Besides, to address the seriously imbalanced class issue caused by traditional binary decomposition (BD) one versus rest (OVR) for multiclass classification problems, we design a method named BD with a similar positive and negative class size (BD-SPNCS) to generate a set of auxiliary classifiers. Experimental results on benchmark and real-world datasets demonstrate that our proposed PS-MOGP outperforms state-of-the-art traditional and evolutionary classification methods in the context of high-dimensional data classification.

Identification and classification of arrhythmic heartbeats from electrocardiogram signals using feature induced optimal extreme gradient boosting algorithm

Arrhythmic heartbeat classification has gained a lot of attention to accelerate the detection of cardiovascular diseases and mitigating the potential cause of one-third of deaths worldwide. In this article, a computer-aided diagnostic (CAD) approach has been proposed for the automated identification and classification of arrhythmic heartbeats from electrocardiogram (ECG) signals using multiple features aided supervised learning model. For proper diagnosis of arrhythmic heartbeats, MIT-BIH Arrhythmia database has been used to train and test the proposed approach. The ECG signals, extracted from sensor leads, have undergone pre-processing via discrete wavelet transform. Three sets of features, i.e. statistical, temporal, and spectral, are extracted from the processed ECG signals followed by random forest aided recursive feature elimination strategy to select the prominent features for proper classification of arrhythmic heartbeats by the proposed optimal extreme gradient boosting (O-XGBoost) classifier. Hyperparameters such as learning rate, tree-specific parameters, and regularization parameters have been optimized to improve the performance of the XGBoost classifier. Moreover, the synthetic minority over-sampling technique has been employed for balancing the dataset in order to improve the classification performance. Quantitative results reveal the remarkable performance over state-of-the-art methods. The proposed model can be implemented in any computer-aided diagnostic system with similar topological structures.

6mA-Finder: a novel online tool for predicting DNA N6-methyladenine sites in genomes.

DNA N6-methyladenine (6 mA) has recently been found as an essential epigenetic modification, playing its roles in a variety of cellular processes. The abnormal status of DNA 6 mA modification has been reported in cancer and other disease. The annotation of 6 mA marks in genome is the first crucial step to explore the underlying molecular mechanisms including its regulatory roles. We present a novel online DNA 6 mA site tool, 6 mA-Finder, by incorporating seven sequence-derived information and three physicochemical-based features through recursive feature elimination strategy. Our multiple cross-validations indicate the promising accuracy and robustness of our model. 6 mA-Finder outperforms its peer tools in general and species-specific 6 mA site prediction, suggesting it can provide a useful resource for further experimental investigation of DNA 6 mA modification. https://bioinfo.uth.edu/6mA_Finder. Supplementary data are available at Bioinformatics online.

Combining Electrodermal Activity and Speech Analysis towards a more Accurate Emotion Recognition System.

Current research in the emotion recognition field is exploring the possibility of merging the information from physiological signals, behavioural data, and speech. Electrodermal activity (EDA) is amongst the main psychophysiological arousal indicators. Nonetheless, it is quite difficult to be analyzed in ecological scenarios, like, for instance, when the subject is speaking. On the other hand, speech carries relevant information of subject emotional state and its potential in the field of affective computing is still to be fully exploited. In this work, we aim at exploring the possibility of merging the information from electrodermal activity (EDA) and speech to improve the recognition of human arousal level during the pronunciation of single affective words. Unlike the majority of studies in the literature, we focus on speakers' arousal rather than the emotion conveyed by the spoken word. Specifically, a support vector machine with recursive feature elimination strategy (SVM-RFE) is trained and tested on three datasets, i.e using the two channels (i.e., speech and EDA) separately and then jointly. The results show that the merging of EDA and speech information significantly improves the marginal classifier (+11.64%). The six selected features by the RFE procedure will be used for the development of a future multivariate model of emotions.

Efficient feature selection and classification for microarray data.

Feature selection and classification are the main topics in microarray data analysis. Although many feature selection methods have been proposed and developed in this field, SVM-RFE (Support Vector Machine based on Recursive Feature Elimination) is proved as one of the best feature selection methods, which ranks the features (genes) by training support vector machine classification model and selects key genes combining with recursive feature elimination strategy. The principal drawback of SVM-RFE is the huge time consumption. To overcome this limitation, we introduce a more efficient implementation of linear support vector machines and improve the recursive feature elimination strategy and then combine them together to select informative genes. Besides, we propose a simple resampling method to preprocess the datasets, which makes the information distribution of different kinds of samples balanced and the classification results more credible. Moreover, the applicability of four common classifiers is also studied in this paper. Extensive experiments are conducted on six most frequently used microarray datasets in this field, and the results show that the proposed methods have not only reduced the time consumption greatly but also obtained comparable classification performance.

Optimizing a machine learning based glioma grading system using multi-parametric MRI histogram and texture features.

Current machine learning techniques provide the opportunity to develop noninvasive and automated glioma grading tools, by utilizing quantitative parameters derived from multi-modal magnetic resonance imaging (MRI) data. However, the efficacies of different machine learning methods in glioma grading have not been investigated.A comprehensive comparison of varied machine learning methods in differentiating low-grade gliomas (LGGs) and high-grade gliomas (HGGs) as well as WHO grade II, III and IV gliomas based on multi-parametric MRI images was proposed in the current study. The parametric histogram and image texture attributes of 120 glioma patients were extracted from the perfusion, diffusion and permeability parametric maps of preoperative MRI. Then, 25 commonly used machine learning classifiers combined with 8 independent attribute selection methods were applied and evaluated using leave-one-out cross validation (LOOCV) strategy. Besides, the influences of parameter selection on the classifying performances were investigated. We found that support vector machine (SVM) exhibited superior performance to other classifiers. By combining all tumor attributes with synthetic minority over-sampling technique (SMOTE), the highest classifying accuracy of 0.945 or 0.961 for LGG and HGG or grade II, III and IV gliomas was achieved. Application of Recursive Feature Elimination (RFE) attribute selection strategy further improved the classifying accuracies. Besides, the performances of LibSVM, SMO, IBk classifiers were influenced by some key parameters such as kernel type, c, gama, K, etc. SVM is a promising tool in developing automated preoperative glioma grading system, especially when being combined with RFE strategy. Model parameters should be considered in glioma grading model optimization.

Predicting the future performance of soccer players

We propose a multitask, regression‐based approach for predicting future performances of soccer players. The multitask approach allows us to simultaneously learn individual player models as offsets to a general model. We devise multitask variants of ridge regression and ε‐support vector regression. Together with a hashed joint feature space, the generalized models can be optimized using standard techniques. Relevant features for the prediction are identified by a modified recursive feature elimination strategy. We report on extensive empirical results using real data from the German Bundesliga. © 2016 Wiley Periodicals, Inc. Statistical Analysis and Data Mining: The ASA Data Science Journal, 2016

Discriminative analysis of skull morphology in adolescent idiopathic scoliosis patients: Comparative study with normal controls

Statistical morphometric analysis is used in this paper to assess the skull morphological difference in adolescent idiopathic scoliosis (AIS) patients as compared with normal controls. The skull morphology analysis starts from a 3-D segmentation of the skull in MRI data using the adaptive thresholding and mathematical morphological operations. The skull vault is then separated from the whole skull surface using a demarcation plane determined by three anthropometrically meaningful landmarks. Since the segmented skull surface is not sufficiently smooth, which will introduce errors in the subsequent shape description, we devise a surface smoothing method to improve the quality of the skull surface. This mesh smoothing task is formulated as a quadratic optimization problem, in which the objective function is designed to be a linear combination of the shape term and the smoothing term, thus it overcomes the shrinkage problem in the traditional Laplacian smoothing. A spherically uniform sampling on the smoothed skull surface is performed afterwards. The scale-normalized distances from the skull vault centroid to the sample points are defined as features, on which the recursive feature elimination strategy is applied to select the features that contribute most to the classification of AIS patients and normal controls using the support vector machine. The results of this study quantify the skull vault surface changes and provide new evidence of bone formation abnormality in AIS patients.