Effective Feature Selection Research Articles

Effective feature learning of multi‐modal genetic and neuroimaging data for prediction of future conversion to Alzheimer’s disease: A machine learning based study

AbstractBackgroundDue to the high dimensionality of Single Nucleotide Polymorphism (SNP) data, current imaging genetic studies of Dementia of Alzheimer’s Type (DAT) usually only select a limited number of candidate SNPs that are reported in the literature, while there might be potentially important AD‐related risk factors in the remaining genes. In this study, we harness a robust feature selection technique that utilizes all available SNPs in the human genome to identify the relevant MRI and genetic features while overcoming feature redundancy.MethodA total of 543 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) that had both MRI and genetic data available are included in the study. These subjects are divided into seven novel stratified groups [Table 1] based on their longitudinal clinical diagnosis. A subject is categorized as DAT+ if it has a follow‐up diagnosis of DAT, and is categorized as DAT‐ otherwise. We utilize a 2‐stage probabilistic multi‐kernel classifier. In the first stage, using 10‐fold cross‐validation: In each fold we select discriminative features by applying Fisher's Exact test on SNP data and Welch's t‐test on MRI data using subjects in the subgroups with the most certain longitudinal diagnosis (sNC in DAT‐ and sDAT in DAT+). In the second stage, using only the most frequently selected features above, we retrain on 80% of sNC and sDAT, validate on the remaining 20%, and test on subjects in the other subgroups with milder longitudinal diagnosis results.ResultOur study showed that although genetic features have a lower prediction power than MRI features, combining both modalities can improve the prediction of future conversion to AD. For subjects in pNC group, MRI features fail to indicate potential risk of developing AD while genetic data can accurately identify the risk [Figure 1]. Our feature selection method has successfully identified significant AD risk factors for both modalities.ConclusionOur study demonstrated that using effective feature selection methods eliminate the need for heuristic selection of AD‐related genes used for machine‐learning studies for image genetics. Our proposed method reveals genetic risk factors already mentioned in literature as well as some novel risk factors that could advance clinical diagnostics in the future.

Enabling Micro AI for Securing Edge Devices at Hardware Level

Emerging embedded systems and Internet-of-Things (IoT) devices, which account for a wide range of applications are often highly resource-constrained that are challenging the software-based methods traditionally adopted for detecting and containing cyber-attacks (e.g., malware) in general-purpose computing systems. In addition to the complexity and cost (computing and storage), the software-based detection methods mainly rely on the static signature analysis of the running programs, requiring continuous software update which is not affordable for embedded systems and edge devices with limited computing and communication bandwidth. To address these challenges, this work proposes an accurate and cost-efficient micro AI enabled countermeasure for securing modern edge devices against emerging cyber-attacks, i.e., malware and Side-Channel Attacks (SCAs) at the hardware level by monitoring applications’ Hardware Performance Counter (HPC) features. To realize a run-time ML-based solution that relies on limited available HPCs in modern edge processors, we first identify the most prominent HPC events for accurate attack detection with the aid of an effective feature selection method. Next, various standard machine learning classifiers are implemented for effective and accurate run-time hardware-assisted malware and side-channel attacks detection. They are compared and characterized in terms of detection accuracy, F-measure, robustness, latency, power consumption, and hardware overheads. Experimental results demonstrate that the J48 classifier achieves the highest detection rate (F-measure) for both malware and SCAs detection with 0.917 and 0.987, respectively, with relatively negligible latency and area overhead as compared to complex models making it a suitable algorithm for enabling an efficient hardware-assisted micro AI countermeasure in edge devices.

An Ensemble Classification Method for High-Dimensional Data Using Neighborhood Rough Set

Mining useful knowledge from high-dimensional data is a hot research topic. Efficient and effective sample classification and feature selection are challenging tasks due to high dimensionality and small sample size of microarray data. Feature selection is necessary in the process of constructing the model to reduce time and space consumption. Therefore, a feature selection model based on prior knowledge and rough set is proposed. Pathway knowledge is used to select feature subsets, and rough set based on intersection neighborhood is then used to select important feature in each subset, since it can select features without redundancy and deals with numerical features directly. In order to improve the diversity among base classifiers and the efficiency of classification, it is necessary to select part of base classifiers. Classifiers are grouped into several clusters by k-means clustering using the proposed combination distance of Kappa-based diversity and accuracy. The base classifier with the best classification performance in each cluster will be selected to generate the final ensemble model. Experimental results on three Arabidopsis thaliana stress response datasets showed that the proposed method achieved better classification performance than existing ensemble models.

A combined forecasting system based on multi-objective optimization and feature extraction strategy for hourly PM2.5 concentration

Accurate hourly PM2.5 concentration prediction plays a key role in air quality monitoring and controlling system, especially when severe haze occurs frequently. A PM2.5 hourly prediction system is developed in this paper, based on an advanced data processing strategy, an effective feature selection technology and a novel optimization algorithm. First, the collected original sequence is decomposed into a group of filters with different wave frequencies and each filter is weighted and reconstructed to mitigate the negative impact of noisy fluctuations. Then mRMR is introduced for extracting interaction information between pollutants, further determining the input of artificial intelligence models. Whereafter, a five-component combined system is taken shape, in which BPNN, ELM, GRNN and BiLSTM are employed as foundation models while Multi-objective Water Cycle Algorithm (MOWCA) is the weight optimization model. The results of hourly PM2.5 concentration prediction simulation experiment in the Bei–Shang–Guang–Shen area make clear that the developed system with minimum forecasting error, excellent generalization capability and robust prediction performance shows a definite latent capacity and future to deal with early warning problems and to design suitable abatement strategies.

Online streaming feature selection based on neighborhood rough set

Feature selection is considered as a necessary and significant pre-processing step in many fields, especially in machine learning. However, in some real problems, in which features flow one by one, many existing approaches do not work well on the online streaming features, and most online streaming feature selection (OSFS) methods face the challenge of requiring domain knowledge before setting optimal parameters in advance. Therefore, an effective feature selection method for online streaming features, named OFS-Gapknn, is proposed in this paper. A new neighborhood rough set relation is firstly defined, which combines the advantages of both the k-nearest and the Gap neighborhood. The proposed neighborhood relation can not only work well on the unevenly distributed sample space, but also need not any parameters and domain knowledge. Then, the relevance and redundancy features are analyzed by using the dependency based on the neighborhood rough set. Finally, one of the optimal feature subsets is obtained. To validate the effectiveness of the proposed algorithm, four traditional methods and three OSFS methods are compared with it on 11 datasets. Experimental results indicate the dominance and significance of the proposed method.

Biomarker Identification From Gene Expression Based on Symmetrical Uncertainty

In this paper, the authors present an effective information theoretic feature selection method, symmetrical uncertainty, to classify gene expression microarray data and detect biomarkers from it. Here, information gain and symmetrical uncertainty contribute for ranking the features. Based on computed values of symmetrical uncertainty, features were sorted from most informative to least informative ones. Then, the top features from the sorted list are passed to random forest, logistic regression, and other well-known classifiers with leave-one-out cross validation to construct the best classification model(s) and accordingly select the most important genes from microarray datasets. Obtained results in terms of classification accuracy, running time, root mean square error, and other parameters computed on leukemia and colon cancer datasets demonstrate the effectiveness of the proposed approach. The proposed method is relatively much faster than many other wrapper or ensemble methods.

Prediction of outcomes after heart transplantation by machine learning models

Abstract Background Models based on traditional statistics for the prediction of outcomes after heart transplantation (HT) have moderate accuracy. We sought to develop and validate state-of-the-art machine learning (ML) models to predict mortality and acute rejection after contemporary HT. Methods We included adult HT recipients from the UNOS database between 2010–2018 using solely pre-transplant clinical and laboratory variables. The study cohort was randomly split in a derivation and a validation cohort with a 3:1 ratio. An effective feature selection algorithm was used to identify strong predictors of 1-year mortality and rejection in the training cohort. Results were used to train the ML models, which were then internally tested using the validation cohort. LIME explainability analysis was used for the best performing ML model. A similar subgroup analysis was performed for 3- and 5-year survival. Results The study cohort comprised of 18,625 patients (53±13 years, 73% males). At 1-year after cardiac transplant, there were 2,334 (12.5%) deaths. Out of a total of 134 pre-transplant variables, 39 and 27 were selected as highly predictive of 1-year mortality and acute rejection respectively, and were used in the ML models. Areas under the curve for the prediction of 1-year survival were 0.689, 0.642, 0.649, 0.637, 0.526 for the Adaboost, Logistic Regression, Decision Tree, Support Vector Machine and K-nearest neighbor models respectively, whereas the IMPACT score had an AUC of 0.569. For the prediction of 1-year acute rejection, Adaboost achieved the highest predictive performance (AUC 0.629). LIME explainability analysis identified the relative impact of the 10 strongest predictors of 1-year mortality and acute rejection. Subgroup analysis using a similar methodology for 3- and 5-year survival yielded AUC of 0.609 and 0.610 using 31 and 91 selected variables respectively. Conclusion ML models created and validated using a contemporary cohort of the UNOS database showed improved accuracy in predicting survival and acute rejection after HT. Funding Acknowledgement Type of funding sources: None.

Onboard Cloud-enabled Parkinson Disease Identification System Using Enhanced Grey Wolf Optimization

Parkinson disease is a rigorous neurodegenerative disorder characterized by the cognitive behavior ending with disability problems. Especially, the elderly people should be given more care and spend more time duration to diagnose when they are at risk. It is more important to identify and diagnose Parkinson disease at an earlier stage rather than spending too much of cost later stages. Different ways of diagnosing the disease ranging from gene analysis to gait behavior, speech, writing test and olfactory models were used in the conventional testing process. In order to increase the patient’s quality of life and minimize the cost of healthcare utilization, an Onboard Cloud-Enabled Parkinson Disease Identification System (OCPDIS) is proposed. An enhanced grey wolf optimization is explored along with the differential evolution techniques to form an effective hybrid feature selection method. Using this feature selection method in the enhanced k-Nearest Neighbor (k-NN) classifier model could substantially improve the prediction time and prediction accuracy.

An Efficient Feature Engineering Method for Fraud Detection in E-commerce

With the speedy expansion of e-commerce, credit cards have also become rising vogue, and that makes online transactions sleek and suitable. In conjunction with rising in online transactions, credit card fraud also increasing, which contributes to losses incurred yearly. As a result, many deep and machine learning methods are produced to fix such as problems like Logistic Regression (LR), Support Vector Machine (SVM), Naïve Bayes (NB), and other algorithms, but the current models are still not accurate. Moreover, sometimes the used datasets still need further preprocessing, since that has been approved the important role of feature engineering in performance optimization. In this paper, effective feature engineering and feature selection methods have been produced for preprocessing the raw dataset, which was transformed with Exploratory Data Analysis (EDA). Then LightGBM, XGboost, and Random forest classifiers are used for fraud detection. Experiments show that the LightGBM and XGboost models achieved the best accuracy with 100% after applying further preprocessing on the dataset. Index Terms— Fraud detection, Feature engineering, Machine Learning, LightGBM, Random forest, XGboost.

Effective feature selection based on Fisher Ratio for snoring recognition using different validation methods

Snoring is an important characteristic of obstructive sleep apnea-hypopnea syndrome (OSAHS). Snoring sound can be used to develop a non-invasive approach for automatically screening OSAHS. In this paper, a snoring detection algorithm based on acoustic features and the XGBoost was proposed. The Fisher Ratio (FR) method was used for feature selection and the 23-dimensional feature set was extracted from the original 58-dimensional feature to reduce computational complexity. A variety of training and test data split methods were used to evaluate the performance of the proposed model. In addition, the proposed method was compared with the deep learning method on the independent subjects training set and test set. The accuracy of the reduced feature set by the proposed model was 87.22% and the deep learning model 87.26% on independent subjects training set and test set, the prediction time of the deep learning model on the test set was around 238 ms, the prediction time of XGBoost was around 58 ms, and the computing efficiency of XGBoost is higher than that of the deep learning model. The results show that the proposed method achieves a good balance between computational cost and performance. These results provide an important method for fast and real-time classification and evaluation of snoring for OSAHS.

An efficient Pareto-based feature selection algorithm for multi-label classification

Multi-label learning algorithms have significant challenges due to high-dimensional feature space and noises in multi-label datasets. Feature selection methods are effective techniques to deal with these problems. ParetoCluster is an effective multi-label feature selection algorithm based on Pareto dominance and cluster analysis concepts which considers each label an objective function. This algorithm loses its effectiveness to differentiate features when dealing with high labeled datasets and makes most features incomparable (e.g., when most features fall into the first layer). Thus, a cluster analysis criterion in ParetoCluster will play a decisive role in determining the most relevant features. Bearing this in mind, in this paper, we have modeled the multi-label feature selection problem into a bi-objective optimization problem regarding the relevancy and redundancy degree of the features. We then handle it using Pareto dominance in this bi-objective domain. To illustrate the optimality and efficiency of the proposed method, we have compared our approach against some similar techniques.

Autism Spectrum Disorder Studies Using fMRI Data and Machine Learning: A Review

Machine learning methods have been frequently applied in the field of cognitive neuroscience in the last decade. A great deal of attention has been attracted to introduce machine learning methods to study the autism spectrum disorder (ASD) in order to find out its neurophysiological underpinnings. In this paper, we presented a comprehensive review about the previous studies since 2011, which applied machine learning methods to analyze the functional magnetic resonance imaging (fMRI) data of autistic individuals and the typical controls (TCs). The all-round process was covered, including feature construction from raw fMRI data, feature selection methods, machine learning methods, factors for high classification accuracy, and critical conclusions. Applying different machine learning methods and fMRI data acquired from different sites, classification accuracies were obtained ranging from 48.3% up to 97%, and informative brain regions and networks were located. Through thorough analysis, high classification accuracies were found to usually occur in the studies which involved task-based fMRI data, single dataset for some selection principle, effective feature selection methods, or advanced machine learning methods. Advanced deep learning together with the multi-site Autism Brain Imaging Data Exchange (ABIDE) dataset became research trends especially in the recent 4 years. In the future, advanced feature selection and machine learning methods combined with multi-site dataset or easily operated task-based fMRI data may appear to have the potentiality to serve as a promising diagnostic tool for ASD.

Automated detection of bioimages using novel deep feature fusion algorithm and effective high-dimensional feature selection approach

The classification of bioimages plays an important role in several biological studies, such as subcellular localisation, phenotype identification and other types of histopathological examinations. The objective of the present study was to develop a computer-aided bioimage classification method for the classification of bioimages across nine diverse benchmark datasets. A novel algorithm was developed, which systematically fused the features extracted from nine different convolution neural network architectures. A systematic fusion of features boosts the performance of a classifier but at the cost of the high dimensionality of the fused feature set. Therefore, non-discriminatory and redundant features need to be removed from a high-dimensional fused feature set to improve the classification performance and reduce the time complexity. To achieve this aim, a method based on analysis of variance and evolutionary feature selection was developed to select an optimal set of discriminatory features from the fused feature set. The proposed method was evaluated on nine different benchmark datasets. The experimental results showed that the proposed method achieved superior performance, with a significant reduction in the dimensionality of the fused feature set for most bioimage datasets. The performance of the proposed feature selection method was better than that of some of the most recent and classical methods used for feature selection. Thus, the proposed method was desirable because of its superior performance and high compression ratio, which significantly reduced the computational complexity.

A high-accuracy hybrid method for short-term wind power forecasting

In this article, a high-accuracy hybrid approach for short-term wind power forecasting is proposed using historical data of wind farm and Numerical Weather Prediction (NWP) data. The power forecasting is carried out in three stages: wind direction forecasting, wind speed forecasting, and wind power forecasting. In all three phases, the same hybrid method is used, and the only difference is in the input data set. The main steps of the proposed method are constituted of outlier detection, decomposition of time series using wavelet transform, effective feature selection and prediction of each time series decomposed using Multilayer Perceptron (MLP) neural network. The combination of automatic clustering and T2 statistic is employed for outlier detection. Effective feature selection is also carried out with the assistance of the Non-dominated Sorting Genetic Algorithm II (NSGA- ӀӀ) and the Radial Basis Function (RBF) Neural network. The evaluation of the proposed method using the data of Sotavento wind farm located in Spain demonstrates the very high accuracy of the proposed approach.

Random forest-integrated analysis in AD and LATE brain transcriptome-wide data to identify disease-specific gene expression.

Alzheimer’s disease (AD) is a complex neurodegenerative disorder that affects thinking, memory, and behavior. Limbic-predominant age-related TDP-43 encephalopathy (LATE) is a recently identified common neurodegenerative disease that mimics the clinical symptoms of AD. The development of drugs to prevent or treat these neurodegenerative diseases has been slow, partly because the genes associated with these diseases are incompletely understood. A notable hindrance from data analysis perspective is that, usually, the clinical samples for patients and controls are highly imbalanced, thus rendering it challenging to apply most existing machine learning algorithms to directly analyze such datasets. Meeting this data analysis challenge is critical, as more specific disease-associated gene identification may enable new insights into underlying disease-driving mechanisms and help find biomarkers and, in turn, improve prospects for effective treatment strategies. In order to detect disease-associated genes based on imbalanced transcriptome-wide data, we proposed an integrated multiple random forests (IMRF) algorithm. IMRF is effective in differentiating putative genes associated with subjects having LATE and/or AD from controls based on transcriptome-wide data, thereby enabling effective discrimination between these samples. Various forms of validations, such as cross-domain verification of our method over other datasets, improved and competitive classification performance by using identified genes, effectiveness of testing data with a classifier that is completely independent from decision trees and random forests, and relationships with prior AD and LATE studies on the genes linked to neurodegeneration, all testify to the effectiveness of IMRF in identifying genes with altered expression in LATE and/or AD. We conclude that IMRF, as an effective feature selection algorithm for imbalanced data, is promising to facilitate the development of new gene biomarkers as well as targets for effective strategies of disease prevention and treatment.

An effective genetic algorithm-based feature selection method for intrusion detection systems

Availability of suitable and validated data is a key issue in multiple domains for implementing machine learning methods. Higher data dimensionality has adverse effects on the learning algorithm's performance. This work aims to design a method that preserves most of the unique information related to the data with minimum number of features. Addressing the feature selection problem in the domain of network security and intrusion detection, this work contributes an enhanced Genetic Algorithm (GA)-based feature selection method, named as GA-based Feature Selection (GbFS), to increase the classifiers’ accuracy. Securing a network from the cyber-attacks is a critical task and needs to be strengthened. Machine learning, due to its proven results, is widely used in developing firewalls and Intrusion Detection Systems (IDSs) to identify new kinds of attacks. Utilizing machine learning algorithms, IDSs are able to detect the intruder by analyzing the network traffic passing through it. This work presents parameter tuning for the GA-based feature selection along with a novel fitness function. The present work develops an enhanced GA-based feature selection method which is tested over three benchmark network traffic datasets, namely, CIRA-CIC-DOHBrw-2020, UNSW-NB15, and Bot-IoT. A comparison is also performed with the standard feature selection methods. Results show that the accuracies improve using GbFS by achieving a maximum accuracy of 99.80%.

A Multi-Objective Multi-Label Feature Selection Algorithm Based on Shapley Value.

Multi-label learning is dedicated to learning functions so that each sample is labeled with a true label set. With the increase of data knowledge, the feature dimensionality is increasing. However, high-dimensional information may contain noisy data, making the process of multi-label learning difficult. Feature selection is a technical approach that can effectively reduce the data dimension. In the study of feature selection, the multi-objective optimization algorithm has shown an excellent global optimization performance. The Pareto relationship can handle contradictory objectives in the multi-objective problem well. Therefore, a Shapley value-fused feature selection algorithm for multi-label learning (SHAPFS-ML) is proposed. The method takes multi-label criteria as the optimization objectives and the proposed crossover and mutation operators based on Shapley value are conducive to identifying relevant, redundant and irrelevant features. The comparison of experimental results on real-world datasets reveals that SHAPFS-ML is an effective feature selection method for multi-label classification, which can reduce the classification algorithm’s computational complexity and improve the classification accuracy.

Effective feature selection with fuzzy entropy and similarity classifier for chatter vibration diagnosis

Feature selection represents the main challenge against the classification strategies for several applications of signal processing. Besides, the high computational speed and accuracy represent a critical requirement of chatter diagnosis. In this paper, a new fuzzy entropy measure with a similarity classifier is developed for feature selection vibration diagnosis in CNC machines. The measured signals are filtered using ensemble empirical mode decomposition and analyzed using Hilbert-Huang transform approach. The proposed feature selection technique can decrease noise and this strategy improves the classification accuracy. The proposed fuzzy entropy approach is compared with different feature selection and machine learning methods in the literature. The experimental results and comparative analyses confirm the superiority of the proposed fuzzy entropy method for chatter vibration identification with a low computational burden and a high accuracy of 98% that is significantly greater than other existing intelligent diagnosis techniques.

A hybrid ensemble-filter wrapper feature selection approach for medical data classification

Background and objectiveMedical data plays a decisive role in disease diagnosis. The classification accuracy of high-dimensional datasets is often diminished by several redundant and irrelevant features. In this context, feature selection becomes an indispensable process. Feature selection primarily intends to identify a feature subspace which retains the classification accuracy while reducing the high computational cost of learning model as well as eliminating noise. The suitability of an appropriate feature selection approach heavily depends upon the capability of that approach to match the problem framework and to discover the intrinsic patterns within the data. The prime objective of this paper is to develop an ensemble-filter based hybrid feature selection model for disease detection. MethodsIn this paper, a four-step hybrid ensemble feature selection algorithm has been introduced. Firstly, the dataset is partitioned using the cross-validation procedure. Secondly, in the filter step, various filter methods based on weighted scores were ensembled to generate a ranking of features, and thirdly sequential forward selection algorithm is utilized as a wrapper technique to obtain an optimal subset of features. Finally, the resulting optimal subset is processed for subsequent classification tasks. ResultsExperiments have been performed on twenty benchmark medical datasets with different dimensionalities. The performance of the proposed hybrid approach is compared with fourteen state-of-the-art feature selection algorithms on four benchmark classifiers namely, Naïve Bayes, Support Vector Machine with Radial Basis Function, Random Forest and k-Nearest Neighbor. The empirical results clearly demonstrate the superiority of the proposed hybrid methodology over the competing methods with respect to accuracy, sensitivity, specificity, f1-score, area under curve evaluation measures and number of selected features. The statistical analysis of the obtained results shows the outperformance and the competitiveness of the proposed hybrid method with respect to various state-of-the-art algorithms. ConclusionsThis study concludes that the proposed hybrid approach proves to be more effective and reliable feature selection technique in selecting highly discriminative features. The framework can be utilized as a promising tool by both clinicians and researchers in enhancing the classification performance of medical datasets.

An efficient feature selection framework based on information theory for high dimensional data

Feature selection plays a vital role in many fields, particularly in pattern recognition and bioinformatics, for selecting informative and relevant features from high dimensional datasets. The increase in dimensionality of data along with the existence of redundant and irrelevant features leads to challenging performance issues when processing and analysing the data. In this paper, an effective feature selection technique called mutual information and Monte Carlo based feature selection (MIMCFS) is proposed. It comprises of two stages. The first stage aims to select predominant features from the high dimensional data. The second stage involves elimination of redundant features that were selected in the first stage. For the purpose of implementing the first stage, a new feature selection strategy based on the approximate Markov blanket and the concept of mutual information is proposed to find out irrelevant and redundant features. In second stage, to avoid misjudgement of redundant features as relevant features, a new strategy based on Monte Carlo tree search technique is proposed in order to completely eradicate redundant features and to improve feature interaction. For experimental evaluation, eight benchmark microarray datasets including imbalanced ones pertaining to cancer analysis are used. Further, in order to compare and justify the performance of the proposed feature selection method, seven state-of-art feature selection techniques namely CFS, Relief, DISR, JMI, CMIM and CMI are employed. The outputs from these feature selection techniques are provided to three standard classifiers namely Naive Bayes, SVM and C4.5 in order to assess the significance of the selected features in building classification models. 10-fold cross validation is adopted to evaluate the classifiers. Accuracy, precision, recall, f-measure, standard deviation, statistical significance metrics are measured to quantify the classifier performance. Experimental results demonstrate the outstanding performance of the proposed algorithm when compared to that of the standard existing methods.