Feature Subset Selection Algorithm Research Articles

Evolution of the random subset feature selection algorithm for classification problem

Datasets often include excessive or irrelevant data that affect the performance and complexity of the machine learning model. Feature selection is one of the most effective dimension-reduction technics in datasets to solve the problem. Feature reduction can lead to increased classification accuracy and decreased computational costs. Wrapper feature selection algorithms are a popular and effective category of these methods which considering the learning method's feedback. One of the well-known algorithms in this area is the random subset feature selection algorithm (RSFS). This study proposes an improved version that has higher convergence speed, lower feature selection rate, and higher classification accuracy. Improvements include considering the features' interaction in the selective subsets, continuous evaluation of the selected features to avoid being stuck in the local optima and enhancing the method's evolution phase. To comprehensively evaluate the proposed algorithm, its performance was applied to 20 standard and well-known datasets from public resources and compared with various recent related methods. Experimental results demonstrate that the proposed method reduced classification errors using fewer features compared to other methods, achieving the highest ranking in the Friedman and Wilcoxon rank-sum tests.

A machine learning-based approach for flames classification in industrial Heavy Oil-Fire Boilers

The burner combustion tuning is a complex problem that has been studied through flame monitoring and characterization. It has been observed that the flame electromagnetic spectrum and flickering contain specific flame information in combustion processes. This information is helpful for combustion stoichiometry tuning on burners. This paper described a method for selecting the best flame feature subset that can be computed from the scanner signal, in order to get the flame index and induce combustion stoichiometry on burners under specific combustion conditions. We propose a method for selecting a reduced subset with only the useful flame features for flame index classification. To extract the most relevant flame features we use a feature subset selection (FSS) algorithm and to determine the combustion state in burners, five flame indices were defined that represent the most common flame states in oil fuel-fired boilers. FSS includes complete, sequential, and random searches in order to eliminate redundant and noisy flame features to decrease the flame feature set dimension. A probabilistic neural network (PNN) algorithm was implemented for flame feature clustering. Signals from the actual flame scanner system and relevant variables from the boiler data acquisition system were used by the algorithms to calculate the burner flame index. A set of parametric tests was done in a heavy oil-fired boiler under well-known flame and index conditions to train and test the flame classifier. The results showed that only the four more relevant features are enough to classify flames with a good performance (92.3% accuracy), which is useful for burner combustion monitoring and optimization.

Detection of Android Malware in the Internet of Things through the K-Nearest Neighbor Algorithm.

Predicting attacks in Android malware devices using machine learning for recommender systems-based IoT can be a challenging task. However, it is possible to use various machine-learning techniques to achieve this goal. An internet-based framework is used to predict and recommend Android malware on IoT devices. As the prevalence of Android devices grows, the malware creates new viruses on a regular basis, posing a threat to the central system's security and the privacy of the users. The suggested system uses static analysis to predict the malware in Android apps used by consumer devices. The training of the presented system is used to predict and recommend malicious devices to block them from transmitting the data to the cloud server. By taking into account various machine-learning methods, feature selection is performed and the K-Nearest Neighbor (KNN) machine-learning model is proposed. Testing was carried out on more than 10,000 Android applications to check malicious nodes and recommend that the cloud server block them. The developed model contemplated all four machine-learning algorithms in parallel, i.e., naive Bayes, decision tree, support vector machine, and the K-Nearest Neighbor approach and static analysis as a feature subset selection algorithm, and it achieved the highest prediction rate of 93% to predict the malware in real-world applications of consumer devices to minimize the utilization of energy. The experimental results show that KNN achieves 93%, 95%, 90%, and 92% accuracy, precision, recall and f1 measures, respectively.

AutoFe-Sel: A Meta-learning based methodology for Recommending Feature Subset Selection Algorithms

AutoFe-Sel: A Meta-learning based methodology for Recommending Feature Subset Selection Algorithms

Feature subset selection for data and feature streams: a review

Real-world problems are commonly characterized by a high feature dimensionality, which hinders the modelling and descriptive analysis of the data. However, some of these data may be irrelevant or redundant for the learning process. Different approaches can be used to reduce this information, improving not only the speed of building models but also their performance and interpretability. In this review, we focus on feature subset selection (FSS) techniques, which select a subset of the original feature set without making any transformation on the attributes. Traditional batch FSS algorithms may not be adequate to efficiently handle large volumes of data, either because memory problems arise or data are received in a sequential manner. Thus, this article aims to survey the state of the art of incremental FSS algorithms, which can perform more efficiently under these circumstances. Different strategies are described, such as incrementally updating feature weights, applying information theory or using rough set-based FSS, as well as multiple supervised and unsupervised learning tasks where the application of FSS is interesting.

Feature subset selection algorithm based on symmetric uncertainty and interaction factor

Feature subset selection algorithm based on symmetric uncertainty and interaction factor

TFSFB: Two-stage feature selection via fusing fuzzy multi-neighborhood rough set with binary whale optimization for imbalanced data

Obtaining informative features is crucial in imbalanced classification. However, existing neighborhood rough set-based feature selection approaches easily overlook the diversity and complexity of data distributions, and it is difficult to obtain this global optimal feature subset from imbalanced and high-dimensional datasets. To tackle these limitations, we construct a new two-stage feature subset selection scheme by fusing the fuzzy multi-neighborhood rough set (FMRS) with the binary whale optimization algorithm (BWOA) for imbalanced data. First, to evaluate those distributions of different features, this standard deviation coefficient is introduced to construct a fuzzy multi-neighborhood radius set. Then, the fuzzy multi-neighborhood granule and fuzzy membership degree are presented to establish the novel FMRS, and the feature significance measure in the view of algebra is developed to balance the approximate properties and influences of different features in the negative and positive classes. Second, fuzzy multi-neighborhood conditional entropy is defined to maximize information quantity of class- imbalanced data from the information view, and then by fusing the two evaluation perspectives above, this mixed metric is provided to fully assess this uncertainty of class-imbalanced datasets. These internal and external significant metrics are designed to obtain this preselected candidate set of features based on the filter FMRS model at this first stage. Third, a control factor can be developed to dominate the whale position update, and a novel fitness function will be constructed when fusing the dependency degree and entropy measure with the reduction ratio to evaluate this optimal subset of features. Adopting population partitioning and local interference schemes can prevent the BWOA from becoming trapped within a local optimum. To reduce this search space of evolution, the dynamic bitmask is used to improve the BWOA, and then an optimal subset of features is acquired through continuous iterations of this wrapper BWOA at this second stage. Finally, a new two-stage algorithm for feature subset selection by fusing FMRS and BWOA is provided to process class-imbalanced data, where this particle swarm optimization algorithm confirms those optimized parameters. Experiments on 31 datasets show that our algorithm is efficient and can achieve excellent classification efficiency for binary and multiclass imbalanced data.

Quantum-Inspired Owl Search Algorithm with Ensembles of Filter Methods for Gene Subset Selection from Microarray Data

Finding the optimum subset of genes for microarray classification is laborious because microarray data are often high-dimensional and contain many irrelevant and redundant genes. To overcome this problem, we have proposed a two-step technique. In the first step, to reduce the vast number of genes or features, an ensemble of popular rank-based feature selection algorithms with filter evaluation metrics are used to select a group of top-ranking genes. In the next step, the quantum-inspired owl search algorithm ([Formula: see text]), a new filter fitness function-based metaheuristic search technique incorporating concepts from quantum computing, is developed to identify the best subset of genes from the predetermined list. The experimental findings reveal that the ensemble approach in the first step can select more dominant groups of genes than each of the individual filters. Furthermore, it has been found that [Formula: see text] can reduce the cardinality of the selected optimum gene subset with comparable classification accuracy and requires lesser computational time than our earlier proposed QIOSA-based wrapper approach (i.e. [Formula: see text]). Besides, compared with three popular evolutionary feature subset selection algorithms, [Formula: see text] efficiently reduces the optimum cardinality of the gene subset while maintaining acceptable classification accuracy.

Intelligent Intrusion Detection System for Industrial Internet of Things Environment

Rapid increase in the large quantity of industrial data, Industry 4.0/5.0 poses several challenging issues such as heterogeneous data generation, data sensing and collection, real-time data processing, and high request arrival rates. The classical intrusion detection system (IDS) is not a practical solution to the Industry 4.0 environment owing to the resource limitations and complexity. To resolve these issues, this paper designs a new Chaotic Cuckoo Search Optimization Algorithm (CCSOA) with optimal wavelet kernel extreme learning machine (OWKELM) named CCSOA-OWKELM technique for IDS on the Industry 4.0 platform. The CCSOA-OWKELM technique focuses on the design of feature selection with classification approach to achieve minimum computation complexity and maximum detection accuracy. The CCSOA-OWKELM technique involves the design of CCSOA based feature selection technique, which incorporates the concepts of chaotic maps with CSOA. Besides, the OWKELM technique is applied for the intrusion detection and classification process. In addition, the OWKELM technique is derived by the hyperparameter tuning of the WKELM technique by the use of sunflower optimization (SFO) algorithm. The utilization of CCSOA for feature subset selection and SFO algorithm based hyperparameter tuning leads to better performance. In order to guarantee the supreme performance of the CCSOA-OWKELM technique, a wide range of experiments take place on two benchmark datasets and the experimental outcomes demonstrate the promising performance of the CCSOA-OWKELM technique over the recent state of art techniques.

3D Geophysical Predictive Modeling by Spectral Feature Subset Selection in Mineral Exploration

Several technical challenges are related to data collection, inverse modeling, model fusion, and integrated interpretations in the exploration of geophysics. A fundamental problem in integrated geophysical interpretation is the proper geological understanding of multiple inverted physical property images. Tackling this problem requires high-dimensional techniques for extracting geological information from modeled physical property images. In this study, we developed a 3D statistical tool to extract geological features from inverted physical property models based on a synergy between independent component analysis and continuous wavelet transform. An automated interpretation of multiple 3D geophysical images is also presented through a hybrid spectral feature subset selection (SFSS) algorithm based on a generalized supervised neural network algorithm to rebuild limited geological targets from 3D geophysical images. Our self-proposed algorithm is tested on an Au/Ag epithermal system in British Columbia (Canada), where layered volcano-sedimentary sequences, particularly felsic volcanic rocks, are associated with mineralization. Geophysical images of the epithermal system were obtained from 3D cooperative inversion of aeromagnetic, direct current resistivity, and induced polarization data sets. The recovered cooperative susceptibilities allowed locating a magnetite destructive zone associated with porphyritic intrusions and felsic volcanoes (Au host rocks). The practical implementation of the SFSS algorithm in the study area shows that the proposed spectral learning scheme can efficiently learn the lithotypes and Au grade patterns and makes predictions based on 3D physical property inputs. The SFSS also minimizes the number of extracted spectral features and tries to pick the best representative features for each target learning case. This approach allows interpreters to understand the relevant and irrelevant spectral features in addition to the 3D predictive models. Compared to conventional 3D interpolation methods, the 3D lithology and Au grade models recovered with SFSS add predictive value to the geological understanding of the deposit in places without access to prior geological and borehole information.

Analyzing factors and interaction terms affecting urban fatal crash types based on a hybrid framework of econometric model and machine learning approaches

The discrete outcome model is an important method for analyzing the factors affecting crash outcomes. However, the lack of effective approaches for discretizing continuous variables and mining interaction terms are two important problems confronted by such models. To address the above issues, this paper proposes a hybrid approach combining machine learning and econometric modelling to investigate fatal crash types in Shenzhen, China. First, the fatal crash data were collected from 2014 to 2016 in Shenzhen. Second, the minimum description length principle (MDLP), an outstanding representative of supervised discretization algorithms, was used for the discretization of continuous variables in the data. This algorithm selects the proper cut-point through the minimization of the entropy for the given interval. Subsequently, the feature subset selection algorithm based on association rule mining (FEAST), which has advantages over other interaction-mining algorithms in terms of structure freedom and the global search capability, was employed to mine the interaction effects between variables. Finally, the discretized continuous variables and the interaction terms were incorporated into the random parameters logit (RPL) model. Results reveal that the goodness of fit of the MDLP-FEAST-RPL model proposed in this paper is significantly better than that of the equal width discretization (EWD)-RPL, MDLP-RPL, and EWD-FEAST-RPL models. In addition, a total of eleven factors and interaction terms are associated with urban fatal crash types. These findings will facilitate the development of cost-effective policies or countermeasures for targeted crash types in large cities of developing countries.

Optimized Machine Learning-Based Intrusion Detection System for Fog and Edge Computing Environment

As a new paradigm, fog computing (FC) has several characteristics that set it apart from the cloud computing (CC) environment. Fog nodes and edge computing (EC) hosts have limited resources, exposing them to cyberattacks while processing large streams and sending them directly to the cloud. Intrusion detection systems (IDS) can be used to protect against cyberattacks in FC and EC environments, while the large-dimensional features in networking data make processing the massive amount of data difficult, causing lower intrusion detection efficiency. Feature selection is typically used to alleviate the curse of dimensionality and has no discernible effect on classification outcomes. This is the first study to present an Effective Seeker Optimization model in conjunction with a Machine Learning-Enabled Intrusion Detection System (ESOML-IDS) model for the FC and EC environments. The ESOML-IDS model primarily designs a new ESO-based feature selection (FS) approach to choose an optimal subset of features to identify the occurrence of intrusions in the FC and EC environment. We also applied a comprehensive learning particle swarm optimization (CLPSO) with Denoising Autoencoder (DAE) for the detection of intrusions. The development of the ESO algorithm for feature subset selection and the DAE algorithm for parameter optimization results in improved detection efficiency and effectiveness. The experimental results demonstrated the improved outcomes of the ESOML-IDS model over recent approaches.

Scalable feature subset selection for big data using parallel hybrid evolutionary algorithm based wrapper under apache spark environment.

Extant sequential wrapper-based feature subset selection (FSS) algorithms are not scalable and yield poor performance when applied to big datasets. Hence, to circumvent these challenges, we propose parallel and distributed hybrid evolutionary algorithms (EAs) based wrappers under Apache Spark. We propose two hybrid EAs based on the Binary Differential Evolution (BDE), and Binary Threshold Accepting (BTA), namely, (i) Parallel Binary Differential Evolution and Threshold Accepting (PB-DETA), where BDE and BTA work in tandem in every iteration, and (ii) its ablation variant, Parallel Binary Threshold Accepting and Differential Evolution (PB-TADE). Here, BTA is invoked to enhance the search capability and avoid premature convergence of BDE. For comparison purposes, we also parallelized two state-of-the-art algorithms: adaptive DE (ADE) and permutation based DE (DE-FSPM), and named them PB-ADE and P-DE-FSPM respectively. Throughout, logistic regression (LR) is employed to compute the fitness function, namely, area under the receiver operator characteristic curve (AUC). The effectiveness of the proposed algorithms is tested over the five big datasets of varying dimensions. It is noteworthy that the PB-TADE turned out to be statistically significant than the rest. All the algorithms have shown the repeatability property. The proposed parallel model attained a speedup of 2.2–2.9. We also reported feature subset with high AUC and least cardinality.

ETCD: An effective machine learning based technique for cardiac disease prediction with optimal feature subset selection

Cardiac disease is the leading cause of death worldwide. The early diagnosis and prognosis can help patients live longer by lowering mortality and boosting survival rates. The paucity of radiologists and doctors in various nations, due to a variety of factors, is a substantial barrier to early diagnosis. Computational intelligence is an emerging concept in the field of medical imaging to identify, prognosticate, and diagnose disease, among numerous initiatives to construct decision support systems. It relieves radiologists and doctors from being overworked and reduces the time it takes to diagnose patients promptly. In this work, an effective technique for cardiac disease (ETCD) prediction based on machine intelligence has been proposed. To ensure the success of our proposed model, we used effective Data Collection, Data Pre-processing, and feature selection process to generate accurate data for the training model. ETCD utilizes the optimal feature subset selection algorithm (OFSSA) to extract features from different datasets (Cleveland, Hungarian, Combined dataset, and Z_Alizadeh Saini datasets) having varying properties available at the UCI machine learning repository. With ETCD, the average accuracy performance for considered datasets gets increased with SVM, KNN, DT, NB, and RF classifiers by 6.227%, 2.72%, 7.345%, 14.084%, and 18.921% respectively. Further, the results of the experiments demonstrate that ETCD outperformed several contemporary baseline approaches in terms of accuracy and was comparable in terms of sensitivity, specificity, precision, and F_Score. ETCD returns the best feasible solution among all input predictive models considering performance criteria and improves the efficacy of the system, hence can assist doctors and radiologists in a better way to diagnose cardiac patients.

MABUSE: A margin optimization based feature subset selection algorithm using boosting principles

Feature subset selection is one of the most common procedures in machine learning tasks. In a broad sense, feature selection methods can be classified into three major groups, embedded, filter and wrapper methods. Although wrappers might attain superior classification performances, they suffer from scalability issues as they are more computationally expensive than the other methods. Filters are typically faster, and sometimes they are the only applicable methods when the datasets are large. In the field of classification, margin optimization has been proven to be an efficient approach for improving the generalization performance of many classification models. Although margins have been used as criteria for feature selection, in most cases, the most advanced methods are wrappers, which suffer from high computational costs and do not outperform the faster algorithms. In this paper, we propose MABUSE, which is a feature selection method that optimizes margins from a filter perspective. We consider a nearest-neighbor margin definition and, borrowing from the strategy of classifier ensemble construction using boosting, we develop a new method that uses a simple heuristic search. Extensive experimental validation demonstrates that our proposed approach outperforms the state-of-the-art algorithms in both classification and reduction, and has a computational cost that is similar to previous algorithms.

Classification of Right/Left Hand Motor Imagery by Effective Connectivity Based on Transfer Entropy in Electroencephalogram Signal.

The right and left-hand motor imagery (MI) analysis based on the electroencephalogram (EEG) signal can directly link the central nervous system to a computer or a device. This study aims to identify a set of robust and nonlinear effective brain connectivity features quantified by transfer entropy (TE) to characterize the relationship between brain regions from EEG signals and create a hierarchical feature selection and classification for discrimination of right and lefthand MI tasks. TE is calculated among EEG channels as the distinctive, effective connectivity features. TE is a model-free method that can measure nonlinear effective connectivity and analyze multivariate dependent directed information flow among neural EEG channels. Then four feature subset selection methods namely relief-F, Fisher, Laplacian, and local learningbased clustering (LLCFS) algorithms are used to choose the most significant effective connectivity features and reduce redundant information. Finally, support vector machine (SVM) and linear discriminant analysis (LDA) methods are used for classification. Results show that the best performance in 29 healthy subjects and 60 trials is achieved using the TE method via the Relief-F algorithm as feature selection and support vector machine (SVM) classification with 91.02% accuracy. The TE index and a hierarchical feature selection and classification can be useful for the discrimination of right- and left-hand MI tasks from multichannel EEG signals. Effective connectivity features were extracted from electroencephalogram (EEG) to analyze relationships between regions.Four feature selection methods used to select most significant effective features.Support vector machine (SVM) used for discrimination of right and left hand motor imagery (MI) task. In this study, we investigated brain activity using effective connectivity during MI task based on EEG signals. The motor imagery task can accomplish the same goal as motor execution, since they are both activated by the same brain area. Transfer entropy, coherence, and Granger casualty were employed to extract the features. Differential patterns of activity between the left vs. right MI task showed activity around the motor area rather than other areas. In order to reduce redundant information and select the most significant effective connectivity features, four feature subset selection algorithms are used: Relief-F, Fisher, Laplacian, and learning-based clustering feature selection (LLCFS). Then, support vector machine (SVM) and linear discriminant analysis (LDA) are used to classify left and right hand MI task. Comparison of three different connectivity methods showed that TE index had the highest classification accuracy, and could be useful for the discrimination of right and left hand MI tasks from multichannel EEG signals.

A novel feature selection algorithm based on damping oscillation theory.

Feature selection is an important task in big data analysis and information retrieval processing. It reduces the number of features by removing noise, extraneous data. In this paper, one feature subset selection algorithm based on damping oscillation theory and support vector machine classifier is proposed. This algorithm is called the Maximum Kendall coefficient Maximum Euclidean Distance Improved Gray Wolf Optimization algorithm (MKMDIGWO). In MKMDIGWO, first, a filter model based on Kendall coefficient and Euclidean distance is proposed, which is used to measure the correlation and redundancy of the candidate feature subset. Second, the wrapper model is an improved grey wolf optimization algorithm, in which its position update formula has been improved in order to achieve optimal results. Third, the filter model and the wrapper model are dynamically adjusted by the damping oscillation theory to achieve the effect of finding an optimal feature subset. Therefore, MKMDIGWO achieves both the efficiency of the filter model and the high precision of the wrapper model. Experimental results on five UCI public data sets and two microarray data sets have demonstrated the higher classification accuracy of the MKMDIGWO algorithm than that of other four state-of-the-art algorithms. The maximum ACC value of the MKMDIGWO algorithm is at least 0.5% higher than other algorithms on 10 data sets.

Predicting MCI to AD Conversation Using Integrated sMRI and rs-fMRI: Machine Learning and Graph Theory Approach.

BackgroundGraph theory and machine learning have been shown to be effective ways of classifying different stages of Alzheimer’s disease (AD). Most previous studies have only focused on inter-subject classification with single-mode neuroimaging data. However, whether this classification can truly reflect the changes in the structure and function of the brain region in disease progression remains unverified. In the current study, we aimed to evaluate the classification framework, which combines structural Magnetic Resonance Imaging (sMRI) and resting-state functional Magnetic Resonance Imaging (rs-fMRI) metrics, to distinguish mild cognitive impairment non-converters (MCInc)/AD from MCI converters (MCIc) by using graph theory and machine learning.MethodsWith the intra-subject (MCInc vs. MCIc) and inter-subject (MCIc vs. AD) design, we employed cortical thickness features, structural brain network features, and sub-frequency (full-band, slow-4, slow-5) functional brain network features for classification. Three feature selection methods [random subset feature selection algorithm (RSFS), minimal redundancy maximal relevance (mRMR), and sparse linear regression feature selection algorithm based on stationary selection (SS-LR)] were used respectively to select discriminative features in the iterative combinations of MRI and network measures. Then support vector machine (SVM) classifier with nested cross-validation was employed for classification. We also compared the performance of multiple classifiers (Random Forest, K-nearest neighbor, Adaboost, SVM) and verified the reliability of our results by upsampling.ResultsWe found that in the classifications of MCIc vs. MCInc, and MCIc vs. AD, the proposed RSFS algorithm achieved the best accuracies (84.71, 89.80%) than the other algorithms. And the high-sensitivity brain regions found with the two classification groups were inconsistent. Specifically, in MCIc vs. MCInc, the high-sensitivity brain regions associated with both structural and functional features included frontal, temporal, caudate, entorhinal, parahippocampal, and calcarine fissure and surrounding cortex. While in MCIc vs. AD, the high-sensitivity brain regions associated only with functional features included frontal, temporal, thalamus, olfactory, and angular.ConclusionsThese results suggest that our proposed method could effectively predict the conversion of MCI to AD, and the inconsistency of specific brain regions provides a novel insight for clinical AD diagnosis.

PiEnPred: a bi-layered discriminative model for enhancers and their subtypes via novel cascade multi-level subset feature selection algorithm

piEnPred: a bi-layered discriminative model for enhancers and their subtypes via novel cascade multi-level subset feature selection algorithm

DeepSGP: Deep Learning for Gene Selection and Survival Group Prediction in Glioblastoma

Glioblastoma Multiforme (GBM) is an aggressive form of glioma, exhibiting very poor survival. Genomic input, in the form of RNA sequencing data (RNA-seq), is expected to provide vital information about the characteristics of the genes that affect the Overall Survival (OS) of patients. This could have a significant impact on treatment planning. We present a new Autoencoder (AE)-based strategy for the prediction of survival (low or high) of GBM patients, using the RNA-seq data of 129 GBM samples from The Cancer Genome Atlas (TCGA). This is a novel interdisciplinary approach to integrating genomics with deep learning towards survival prediction. First, the Differentially Expressed Genes (DEGs) were selected using EdgeR. These were further reduced using correlation-based analysis. This was followed by the application of ranking with different feature subset selection and feature extraction algorithms, including the AE. In each case, fifty features were selected/extracted, for subsequent prediction with different classifiers. An exhaustive study for survival group prediction, using eight different classifiers with the accuracy and Area Under the Curve (AUC), established the superiority of the AE-based feature extraction method, called DeepSGP. It produced a very high accuracy (0.83) and AUC (0.90). Of the eight classifiers, using the extracted features by DeepSGP, the MLP was the best at Overall Survival (OS) prediction with an accuracy of 0.89 and an AUC of 0.97. The biological significance of the genes extracted by the AE were also analyzed to establish their importance. Finally, the statistical significance of the predicted output of the DeepSGP algorithm was established using the concordance index.