Effective Feature Selection Method Research Articles

Series Arc Fault Detection Based on Dual Filtering Feature Selection and Improved Hierarchical Clustering Sensitive Component Selection

Affected by the downstream load of the line, the low-voltage AC series arc fault signal presents nonlinear, non-stationary and random characteristics, which bring great difficulties to the extraction of arc fault features and lead to low arc detection accuracy. To solve this problem, a low-voltage AC series arc fault detection method based on strong discriminative features and sensitive components is proposed. In this work, firstly, the arc fault current signal is collected based on the self-built experimental platform, and the arc fault current components are obtained by the complete ensemble empirical mode decomposition with the adaptive noise (CEEMDAN) method. Secondly, 16 feature indexes based on time window are used to describe the component signals, and an effective feature selection method based on the maximum mutual information coefficient and the significance of feature changes is proposed to realize the mining of highly identifiable features. Then, an improved hierarchical clustering algorithm and a sensitive component selection strategy combining the saliency of feature changes are proposed to eliminate the interference of redundant components. Finally, a strong discriminative feature library of sensitive components of current signal is constructed, and series arc fault detection is realized based on support vector machine. Experimental results show that the proposed method is feasible and effective in arc current feature extraction and fault detection, and provides a reference for low-voltage AC series arc fault detection.

IOFS-SA: An interactive online feature selection tool for survival analysis

Background:Survival analysis is a primary problem before clinical treatments to cancer patients after their operations. In order to make this kind of analysis simple, many corresponding tools have been proposed. Though these tools are easy to use, there exist still two fatal flaws. One is that sample grouping is commonly empirical and wrongly based on original gene expressions or survival time. The other is that their feature selection methods mostly depend univariate semi-supervised regression or the multivariate one without considering the small sample size compared with the high dimension. Objective:In order to solve the two problems, we design an automatic feature selection web tool which can also satisfy interactive sample grouping. Methods:An automatic feature selection is performed on user-defined data or TCGA data. users can also perform manual feature selection. Then, hierarchical clustering is used and an automatic re-clustering strategy is proposed after interactive risk score split. Kaplan–Meier survival curve and log-rank test are utilized as the measurement. Results:Experimental results on 53 datasets from TCGA demonstrate the effectiveness of our method. The tree view, heat map and scatter map can intuitively display the result of the selected genes to the doctors for further research. Conclusions:This method is suitable for survival analysis of high-dimensional small sample data sets. At the same time, it also provides a platform for researchers to analyze custom data. It solves the problems of the existing web tools and provides an effective feature selection method for survival analysis. Availability:The full code package is freely available and can be downloaded at https://github.com/Yuan-23/IOFS-SA-ecp-data-main, and the online version at https://bioinfor.nefu.edu.cn/IOFS-SA/ is ready for use freely.

Multi-dimensional variables and feature parameter selection for aboveground biomass estimation of potato based on UAV multispectral imagery.

Aboveground biomass (AGB) is an essential assessment of plant development and guiding agricultural production management in the field. Therefore, efficient and accurate access to crop AGB information can provide a timely and precise yield estimation, which is strong evidence for securing food supply and trade. In this study, the spectral, texture, geometric, and frequency-domain variables were extracted through multispectral imagery of drones, and each variable importance for different dimensional parameter combinations was computed by three feature parameter selection methods. The selected variables from the different combinations were used to perform potato AGB estimation. The results showed that compared with no feature parameter selection, the accuracy and robustness of the AGB prediction models were significantly improved after parameter selection. The random forest based on out-of-bag (RF-OOB) method was proved to be the most effective feature selection method, and in combination with RF regression, the coefficient of determination (R2) of the AGB validation model could reach 0.90, with root mean square error (RMSE), mean absolute error (MAE), and normalized RMSE (nRMSE) of 71.68 g/m2, 51.27 g/m2, and 11.56%, respectively. Meanwhile, the regression models of the RF-OOB method provided a good solution to the problem that high AGB values were underestimated with the variables of four dimensions. Moreover, the precision of AGB estimates was improved as the dimensionality of parameters increased. This present work can contribute to a rapid, efficient, and non-destructive means of obtaining AGB information for crops as well as provide technical support for high-throughput plant phenotypes screening.

Drug-target interaction prediction using reliable negative samples and effective feature selection methods

Machine learning-based approaches in the field of drug discovery have dramatically reduced the time and cost of the laboratory process of detecting potential drug-target interactions (DTIs). Standard binary classifiers require both positive and negative samples in the training and validation phases. One of the major challenges in the DTI context is the lack of access to non-interacting pairs as negative samples in the learning process. Many recent studies in this field have randomly selected negative samples from unlabeled drug-target pairs. Therefore, due to the probability of the presence of unknown positive samples in a set considered as negative samples, the model results may be affected and appear with a high rate of false positive. In this study, an algorithm called Reliable Non-Interacting Drug-Target Pairs (RNIDTP) is proposed to select reliable negative samples and an efficient algorithm to select relevant features for drug-target interaction prediction. To validate the performance of the proposed RNIDTP algorithm in the selection of negative samples, a benchmark drug-target interactions dataset is used. The results demonstrate the superiority of the proposed algorithm compared with other algorithms in most cases. The results also indicate that by using an appropriate algorithm for the selection of negative samples, the performance of the learning process is significantly increased compared to random selection.

Effective Feature Selection Methods to Detect IoT DDoS Attack in 5G Core Network.

The 5G networks aim to realize a massive Internet of Things (IoT) environment with low latency. IoT devices with weak security can cause Tbps-level Distributed Denial of Service (DDoS) attacks on 5G mobile networks. Therefore, interest in automatic network intrusion detection using machine learning (ML) technology in 5G networks is increasing. ML-based DDoS attack detection in a 5G environment should provide ultra-low latency. To this end, utilizing a feature-selection process that reduces computational complexity and improves performance by identifying features important for learning in large datasets is possible. Existing ML-based DDoS detection technology mostly focuses on DDoS detection learning models on the wired Internet. In addition, studies on feature engineering related to 5G traffic are relatively insufficient. Therefore, this study performed feature selection experiments to reduce the time complexity of detecting and analyzing large-capacity DDoS attacks in real time based on ML in a 5G core network environment. The results of the experiment showed that the performance was maintained and improved when the feature selection process was used. In particular, as the size of the dataset increased, the difference in time complexity increased rapidly. The experiments show that the real-time detection of large-scale DDoS attacks in 5G core networks is possible using the feature selection process. This demonstrates the importance of the feature selection process for removing noisy features before training and detection. As this study conducted a feature study to detect network traffic passing through the 5G core with low latency using ML, it is expected to contribute to improving the performance of the 5G network DDoS attack automation detection technology using AI technology.

İstatistiksel olarak Anlamlı EEG Zaman Alanı ve Frekans Alanı Öznitelikleri ile Motor Hayali Görev Sınıflandırılması

Motor Imaginary (MI) electroencephalography (EEG) signals are obtained when a subject imagines a task without essentially applying it. The accurate decoding of MI EEG signals plays an important role in the design of brain-computer interface (BCI) systems due to the use of these signals in the rehabilitation process of paralyzed patients in recent studies. In this study, two different MI tasks were tried to be differentiated by extracting time-domain and frequency-domain features from 22 channel EEG signals and determining best combination of important and distinctive features based on statistical significance. MI EEG signals were supplied from BCI Competition IV Dataset-IIa. These features were differentiated using 25 different classification algorithms and 5-fold cross-validation method. The repeatability of the results was examined testing each algorithm 10 times. As a result, the highest average accuracy rate of 60.69% was calculated in the Quadratic Support Vector Machine (SVM) using all features and 62.52% in the Ensemble Subspace Discriminant (ESD) algorithm using only the selected features by the independent t-test. The results showed that the independent t-test based feature selection increased the performance in 20 classifiers, and decreased the performance in 5 classifiers. Also, the effectiveness of the feature selection method examined using the paired-sample t-test which is known as repeated measures t-test. The significance value, p-value was found as 0.04. Therefore, the independent t-test based feature selection method is an effective feature selection method and is providing the significant improvement in classifier performance.

Design and Implementation of ML Model for Early Diagnosis of Parkinson’s Disease using Gait Data Analysis in IoT Environment

Parkinson’s disease (PD) is the world’s second most neurodegenerative disorder that results in a steady loss of movement. The symptoms in patients occur slowly with the passage of time are and very hard to identify in its initial stage. So, early diagnosis of PD is the foremost need for timely treatment to people. The introduction of smart technologies like the Internet of Things (IoT) and wearable sensors in the healthcare domain offers a smart way of identifying the symptoms of PD patients. In which smart sensors are worn on the patient’s body which continuously monitor the symptoms in patients and track their possible health status. The major objective of this work is to propose a machine learning-based healthcare model that best classifies the subjects into healthy and Parkinson's patients by extracting the most important features. A step regression-based feature selection method is followed to improve the classification of PD. A Shapiro Wilk test is adopted to check the normality of the gait dataset. This model is implemented on three publicly available Parkinson’s datasets collected from three different studies available on Psyionet. All these data sets contain VGRF recordings obtained from eight different sensors placed under each foot. Experimentation is done on the Jupyter notebook by utilizing Python as a programming language. Experimental results revealed that our proposed model with effective pre-processing, feature extraction, and feature selection method achieved the highest accuracy result of 95.54%, 98.80%, and 94.52% respectively when applied to three datasets. Our research inducts knowledge about significant characteristics of a patient suffering from PD and may help to diagnose and cure at an early stage.

Effective feature learning of multi‐modal genetic and neuroimaging data for prediction of future conversion to Alzheimer’s disease: A machine learning based study

AbstractBackgroundDue to the high dimensionality of Single Nucleotide Polymorphism (SNP) data, current imaging genetic studies of Dementia of Alzheimer’s Type (DAT) usually only select a limited number of candidate SNPs that are reported in the literature, while there might be potentially important AD‐related risk factors in the remaining genes. In this study, we harness a robust feature selection technique that utilizes all available SNPs in the human genome to identify the relevant MRI and genetic features while overcoming feature redundancy.MethodA total of 543 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) that had both MRI and genetic data available are included in the study. These subjects are divided into seven novel stratified groups [Table 1] based on their longitudinal clinical diagnosis. A subject is categorized as DAT+ if it has a follow‐up diagnosis of DAT, and is categorized as DAT‐ otherwise. We utilize a 2‐stage probabilistic multi‐kernel classifier. In the first stage, using 10‐fold cross‐validation: In each fold we select discriminative features by applying Fisher's Exact test on SNP data and Welch's t‐test on MRI data using subjects in the subgroups with the most certain longitudinal diagnosis (sNC in DAT‐ and sDAT in DAT+). In the second stage, using only the most frequently selected features above, we retrain on 80% of sNC and sDAT, validate on the remaining 20%, and test on subjects in the other subgroups with milder longitudinal diagnosis results.ResultOur study showed that although genetic features have a lower prediction power than MRI features, combining both modalities can improve the prediction of future conversion to AD. For subjects in pNC group, MRI features fail to indicate potential risk of developing AD while genetic data can accurately identify the risk [Figure 1]. Our feature selection method has successfully identified significant AD risk factors for both modalities.ConclusionOur study demonstrated that using effective feature selection methods eliminate the need for heuristic selection of AD‐related genes used for machine‐learning studies for image genetics. Our proposed method reveals genetic risk factors already mentioned in literature as well as some novel risk factors that could advance clinical diagnostics in the future.

Enabling Micro AI for Securing Edge Devices at Hardware Level

Emerging embedded systems and Internet-of-Things (IoT) devices, which account for a wide range of applications are often highly resource-constrained that are challenging the software-based methods traditionally adopted for detecting and containing cyber-attacks (e.g., malware) in general-purpose computing systems. In addition to the complexity and cost (computing and storage), the software-based detection methods mainly rely on the static signature analysis of the running programs, requiring continuous software update which is not affordable for embedded systems and edge devices with limited computing and communication bandwidth. To address these challenges, this work proposes an accurate and cost-efficient micro AI enabled countermeasure for securing modern edge devices against emerging cyber-attacks, i.e., malware and Side-Channel Attacks (SCAs) at the hardware level by monitoring applications’ Hardware Performance Counter (HPC) features. To realize a run-time ML-based solution that relies on limited available HPCs in modern edge processors, we first identify the most prominent HPC events for accurate attack detection with the aid of an effective feature selection method. Next, various standard machine learning classifiers are implemented for effective and accurate run-time hardware-assisted malware and side-channel attacks detection. They are compared and characterized in terms of detection accuracy, F-measure, robustness, latency, power consumption, and hardware overheads. Experimental results demonstrate that the J48 classifier achieves the highest detection rate (F-measure) for both malware and SCAs detection with 0.917 and 0.987, respectively, with relatively negligible latency and area overhead as compared to complex models making it a suitable algorithm for enabling an efficient hardware-assisted micro AI countermeasure in edge devices.

Online streaming feature selection based on neighborhood rough set

Feature selection is considered as a necessary and significant pre-processing step in many fields, especially in machine learning. However, in some real problems, in which features flow one by one, many existing approaches do not work well on the online streaming features, and most online streaming feature selection (OSFS) methods face the challenge of requiring domain knowledge before setting optimal parameters in advance. Therefore, an effective feature selection method for online streaming features, named OFS-Gapknn, is proposed in this paper. A new neighborhood rough set relation is firstly defined, which combines the advantages of both the k-nearest and the Gap neighborhood. The proposed neighborhood relation can not only work well on the unevenly distributed sample space, but also need not any parameters and domain knowledge. Then, the relevance and redundancy features are analyzed by using the dependency based on the neighborhood rough set. Finally, one of the optimal feature subsets is obtained. To validate the effectiveness of the proposed algorithm, four traditional methods and three OSFS methods are compared with it on 11 datasets. Experimental results indicate the dominance and significance of the proposed method.

Biomarker Identification From Gene Expression Based on Symmetrical Uncertainty

In this paper, the authors present an effective information theoretic feature selection method, symmetrical uncertainty, to classify gene expression microarray data and detect biomarkers from it. Here, information gain and symmetrical uncertainty contribute for ranking the features. Based on computed values of symmetrical uncertainty, features were sorted from most informative to least informative ones. Then, the top features from the sorted list are passed to random forest, logistic regression, and other well-known classifiers with leave-one-out cross validation to construct the best classification model(s) and accordingly select the most important genes from microarray datasets. Obtained results in terms of classification accuracy, running time, root mean square error, and other parameters computed on leukemia and colon cancer datasets demonstrate the effectiveness of the proposed approach. The proposed method is relatively much faster than many other wrapper or ensemble methods.

Onboard Cloud-enabled Parkinson Disease Identification System Using Enhanced Grey Wolf Optimization

Parkinson disease is a rigorous neurodegenerative disorder characterized by the cognitive behavior ending with disability problems. Especially, the elderly people should be given more care and spend more time duration to diagnose when they are at risk. It is more important to identify and diagnose Parkinson disease at an earlier stage rather than spending too much of cost later stages. Different ways of diagnosing the disease ranging from gene analysis to gait behavior, speech, writing test and olfactory models were used in the conventional testing process. In order to increase the patient’s quality of life and minimize the cost of healthcare utilization, an Onboard Cloud-Enabled Parkinson Disease Identification System (OCPDIS) is proposed. An enhanced grey wolf optimization is explored along with the differential evolution techniques to form an effective hybrid feature selection method. Using this feature selection method in the enhanced k-Nearest Neighbor (k-NN) classifier model could substantially improve the prediction time and prediction accuracy.

An Efficient Feature Engineering Method for Fraud Detection in E-commerce

With the speedy expansion of e-commerce, credit cards have also become rising vogue, and that makes online transactions sleek and suitable. In conjunction with rising in online transactions, credit card fraud also increasing, which contributes to losses incurred yearly. As a result, many deep and machine learning methods are produced to fix such as problems like Logistic Regression (LR), Support Vector Machine (SVM), Naïve Bayes (NB), and other algorithms, but the current models are still not accurate. Moreover, sometimes the used datasets still need further preprocessing, since that has been approved the important role of feature engineering in performance optimization. In this paper, effective feature engineering and feature selection methods have been produced for preprocessing the raw dataset, which was transformed with Exploratory Data Analysis (EDA). Then LightGBM, XGboost, and Random forest classifiers are used for fraud detection. Experiments show that the LightGBM and XGboost models achieved the best accuracy with 100% after applying further preprocessing on the dataset. Index Terms— Fraud detection, Feature engineering, Machine Learning, LightGBM, Random forest, XGboost.

Autism Spectrum Disorder Studies Using fMRI Data and Machine Learning: A Review

Machine learning methods have been frequently applied in the field of cognitive neuroscience in the last decade. A great deal of attention has been attracted to introduce machine learning methods to study the autism spectrum disorder (ASD) in order to find out its neurophysiological underpinnings. In this paper, we presented a comprehensive review about the previous studies since 2011, which applied machine learning methods to analyze the functional magnetic resonance imaging (fMRI) data of autistic individuals and the typical controls (TCs). The all-round process was covered, including feature construction from raw fMRI data, feature selection methods, machine learning methods, factors for high classification accuracy, and critical conclusions. Applying different machine learning methods and fMRI data acquired from different sites, classification accuracies were obtained ranging from 48.3% up to 97%, and informative brain regions and networks were located. Through thorough analysis, high classification accuracies were found to usually occur in the studies which involved task-based fMRI data, single dataset for some selection principle, effective feature selection methods, or advanced machine learning methods. Advanced deep learning together with the multi-site Autism Brain Imaging Data Exchange (ABIDE) dataset became research trends especially in the recent 4 years. In the future, advanced feature selection and machine learning methods combined with multi-site dataset or easily operated task-based fMRI data may appear to have the potentiality to serve as a promising diagnostic tool for ASD.

An effective genetic algorithm-based feature selection method for intrusion detection systems

Availability of suitable and validated data is a key issue in multiple domains for implementing machine learning methods. Higher data dimensionality has adverse effects on the learning algorithm's performance. This work aims to design a method that preserves most of the unique information related to the data with minimum number of features. Addressing the feature selection problem in the domain of network security and intrusion detection, this work contributes an enhanced Genetic Algorithm (GA)-based feature selection method, named as GA-based Feature Selection (GbFS), to increase the classifiers’ accuracy. Securing a network from the cyber-attacks is a critical task and needs to be strengthened. Machine learning, due to its proven results, is widely used in developing firewalls and Intrusion Detection Systems (IDSs) to identify new kinds of attacks. Utilizing machine learning algorithms, IDSs are able to detect the intruder by analyzing the network traffic passing through it. This work presents parameter tuning for the GA-based feature selection along with a novel fitness function. The present work develops an enhanced GA-based feature selection method which is tested over three benchmark network traffic datasets, namely, CIRA-CIC-DOHBrw-2020, UNSW-NB15, and Bot-IoT. A comparison is also performed with the standard feature selection methods. Results show that the accuracies improve using GbFS by achieving a maximum accuracy of 99.80%.

A Multi-Objective Multi-Label Feature Selection Algorithm Based on Shapley Value.

Multi-label learning is dedicated to learning functions so that each sample is labeled with a true label set. With the increase of data knowledge, the feature dimensionality is increasing. However, high-dimensional information may contain noisy data, making the process of multi-label learning difficult. Feature selection is a technical approach that can effectively reduce the data dimension. In the study of feature selection, the multi-objective optimization algorithm has shown an excellent global optimization performance. The Pareto relationship can handle contradictory objectives in the multi-objective problem well. Therefore, a Shapley value-fused feature selection algorithm for multi-label learning (SHAPFS-ML) is proposed. The method takes multi-label criteria as the optimization objectives and the proposed crossover and mutation operators based on Shapley value are conducive to identifying relevant, redundant and irrelevant features. The comparison of experimental results on real-world datasets reveals that SHAPFS-ML is an effective feature selection method for multi-label classification, which can reduce the classification algorithm’s computational complexity and improve the classification accuracy.

An Enhanced Binary Particle Swarm Optimization for Optimal Feature Selection in Bearing Fault Diagnosis of Electrical Machines

This study proposes an effective bearing fault diagnosis model based on an optimized approach for feature selection. The measured signal of the electric motor is processed by envelope analysis and Hilbert-Huang transform techniques to extract the potential features. An enhancement of the binary particle swarm optimization algorithm through population initialization strategy based on feature weights, new updating mechanism, and the screening and replacing process create a new and effective feature selection method that improves classification accuracy and reduces data size. The optimal feature subset is provided separately for artificial neural networks, and support vector machine classifier for the final recognition task. In multiple case studies, the proposed feature selection method is evaluated against the benchmark data sets and shows performance comparable to that of other peer competitors. The effectiveness of the proposed bearing fault diagnosis model is verified on different testbeds and achieves high accuracy and robustness under noise conditions. In addition, experimental results are compared with existing fault diagnostic models, showing the high possibility of the proposed bearing fault diagnosis model.

Opinion classification on a social network by a novel feature selection technique

Most of the opinion comments on social networks are short and ambiguous. In general, opinion classification on the comments is difficult because of lacking dominant features. A feature extraction technique is therefore necessary for improving accuracy of the classification and computational time. This paper proposes an effective feature selection method for opinion classification on a social network. The proposed method selects features based on the concept of a filter model, together with association rules. Support and confidence are used to calculate the weights of features. The features with high weight are selected for classification. Unlike supports in association rules, supports in our method are normalized to 0-1 to remove outlier supports. Moreover, a tuning parameter is used to emphasize the degree of support or confidence. The experimental results show that the proposed method provides high classification efficiency. The proposed method outperforms Information Gain, Chi-Square, and Gini Index in both computational time and accuracy.

Evaluating and selecting features via information theoretic lower bounds of feature inner correlations for high-dimensional data

Feature selection is an important preprocessing and interpretable method in the fields where big data plays an essential role. In this paper, we first reformulate and analyze some representative information theoretic feature selection methods from the perspective of approximations of feature inner correlations, and indicate that many of these methods cannot guarantee any theoretical bounds of feature inner correlations. We thus introduce two lower bounds that have very simple forms for feature redundancy and complementarity, and verify that they are closer to the optima than the existing lower bounds applied by some state-of-the-art information theoretic methods. A simple and effective feature selection method based on the proposed lower bounds is then proposed and empirically verified with a wide scope of real-world datasets. The experimental results show that the proposed method achieves promising improvement on feature selection, indicating the effectiveness of the feature criterion consisting of the proposed lower bounds of redundancy and complementarity.

Semi-supervised neighborhood discrimination index for feature selection

Neighborhood discriminant index (NDI) is an effective feature selection method for supervised learning. In reality, it is easy to obtain unlabeled data and is costly to tag them all. Thus, the given dataset commonly has only a small amount of tagged samples and a large amount of unlabeled ones, which cannot be handled by supervised learning methods. For this situation, we propose a semi-supervised feature selection method called semi-supervised neighborhood discriminant index (SSNDI) that combines NDI and the Laplacian score method to effectively deal with both labeled and unlabeled samples. The goal of SSNDI is to find an optimal feature subset that has a good ability to keep local geometrical structure and to distinguish samples belonging to different classes. In SSNDI, the classical Laplacian score method is modified to cooperate the iterative form of NDI. In each iteration, SSNDI picks up an important feature according to the new criterion that is a mixture of NDI and the modified Laplacian score. Extensive experiments are conducted on UCI and microarray gene datasets. Experimental results confirm that SSNDI can achieve a better performance than NDI and the other state-of-the-art semi-supervised methods.