Effective Feature Selection Research Articles

Predicting length of fatigue cracks by means of machine learning algorithms in the small-data regime

In this paper several statistical learning algorithms are used to predict the maximal length of fatigue cracks based on a sample composed of 31 observations. The small-data regime is still a problem for many professionals, especially in the areas where failures occur rarely. The analyzed object is a high-pressure Nozzle of a heavy-duty gas turbine. Operating parameters of the engines are used for the regression analysis. The following algorithms are used in this work: multiple linear and polynomial regression, random forest, kernel-based methods, AdaBoost and extreme gradient boosting and artificial neural networks. A substantial part of the paper provides advice on the effective selection of features. The paper explains how to process the dataset in order to reduce uncertainty; thus, simplifying the analysis of the results. The proposed loss and cost functions are custom and promote solutions accurately predicting the longest cracks. The obtained results confirm that some of the algorithms can accurately predict maximal lengths of the fatigue cracks, even if the sample is small.

Machine learning for the early prediction of head-up tilt testing outcome

ObjectiveHead-up Tilt Testing (HUTT) is a widely used medical tool for the diagnosis of unexplained syncope. Current HUTT protocols, however, are time-consuming. This study aims to investigate the feasibility of using hemodynamic monitoring and machine learning techniques to achieve early prediction of HUTT outcome for syncope patients. MethodsA total of 209 subjects participated in this study from June 2016 to November 2019, and hemodynamic signals were collected via a Finometer device (Finapres Medical Systems BV, The Netherlands). We extracted features with a total dimension of 4,313 from the early 18 min (5 min of supine position and 13 min of tilting position). A genetic algorithm (GA) was introduced for feature selection, and an index called the selection ratio (SR) was proposed to further analyze the GA selection result. Four machine learning models were established for this classification task, and their performance results were compared. ResultsThe maximum tilting duration was shortened from 35 min to 13 min, and a best area under receiver operating characteristic curve of 0.94 via 5-fold cross-validation was obtained by the SVR model, with a sensitivity of 0.86 and a specificity of 0.82. The performance of all algorithms improved after feature selection by GA. ConclusionThe proposed approach is a promising method to shorten the diagnosis time compared to the existing diagnosis process. The GA introduced in this study is an effective feature selection tool to improve model performance. The proposed SR index effectively contributes to the usability and interpretability of the model.

Metaheuristic Search Based Feature Selection Methods for Classification of Cancer

Cancer is a cluster of diseases caused due to unusual cell growth. This paper aims to discover cancer prediction from the microarray gene expression data using the selected features. The metaheuristic search algorithms select the global and local optimal features using population and neighbourhood based algorithms. Although the ant colony optimization and genetic algorithm search for the global optimal features from the dataset entails enhanced classification, sometimes there occur some challenges in the selection of neighbourhood features. Against this background, two feature selection algorithms are proposed to hybridize tabu search, a neighbourhood based search algorithm with global optimal feature selection algorithm. Those are (1) Ant Colony Optimization and Tabu search with Fuzzy Rough set for Optimal feature selection (ACTFRO) algorithm, (2) Genetic algorithm and Tabu search with Fuzzy Rough set for Optimal feature selection (GATFRO) algorithm. The performance of proposed feature selection algorithms is assessed through a fuzzy rough nearest neighbour classifier using ten-fold cross validation. Four cancer medical datasets and one non-medical dataset are used to analyse the performance of the proposed algorithms in terms of classification accuracy, computation time, sensitivity, specificity, f-measure, receiver operation characteristics and positive predicted value. Results derived from the different performance metrics confirm that the proposed algorithms evidence effective global and local feature selection hybridization with improved results.

User-Level Ultra-Short-Term Load Forecasting Model Based on Optimal Feature Selection and Bahdanau Attention Mechanism

Accurate ultra-short-term load forecasting is of great significance for real-time power generation scheduling and development of power cyber physical systems (Power CPS). However, in order to forecast the future load using the current high-dimensional, diverse and heterogeneous electric power consumption information, new challenges have been raised to the effective feature selection and the accurate load forecasting algorithms. However, very limited existing works consider the feature selection for the electric power consumption information and impacts to the thereafter load forecasting model. In view of this point, features that are critical to the load forecasting are selected using an embedded feature selection algorithm based on LightGBM to form an optimal feature set, with which a sequence to sequence (S2S) and gated recurrent unit (GRU)-based ultra-short-term load forecasting model that incorporates Bahdanau attention (BA) mechanism is presented. The S2S-GRU model is based on an encoding–decoding framework that is compatible to the input and output data series with variable lengths. By introducing the BA mechanism, loss of previous information issue of GRU can be solved. Experimental results show that first the presented feature selection algorithm can help to improve the performance of the load forecasting model. Second, the presented load forecasting model can find a compromise between the forecasting efficiency and accuracy.

Deep self-representative subspace clustering network

Deep learning based subspace clustering networks have been a significant technique for motion segmentation, unsupervised image segmentation, image representation and compression, and face clustering by separating the high-dimensional data points into their representative low-dimensional linear subspaces. Effective feature selection is critical to remove redundant samples and select the representative feature subset from high-dimensional data space; hence deriving the number of subspaces, their dimensions, data segmentation, and a basis for each subspace. The effective self-representative feature selection and emphasis by scaling the feature map in the learned embedded space is required for deep learning based subspace clustering to reduce the number of parameters and dimension of the self-representative layer. In this paper, we propose a self-representative feature extraction deep neural network for unsupervised subspace clustering to improve representativeness and clustering ability. The extensive relevant results on various data demonstrate that deep subspace clustering employing self-representative features from high-dimensional data can effectively reduce the dimension of the self-representative layer while improving performance.

Brain tumor grade identification using deep Elman neural network with adaptive fuzzy clustering-based segmentation approach

A brain tumor is a collection of anomalous evolution of harmful and unnecessary nerve cells in the brain. For advanced treatment, it is essential to categorize the tumors from the MRI (magnetic resonance imaging). In this paper, develop the adaptive fuzzy Tsallis entropy (FTE) clustering with the improved cuckoo search optimization (ICS) procedure, and the effective feature selection mechanism is proposed in this paper. Initially, the input MRI is pre-processed using the anisotropic diffusion filter and non-parametric region-based techniques to eliminate noise removal and skull stripping. After that, the tumor regions are segmented using the adaptive FTE clustering method with the ICS algorithm. The robust features are attained using the first-order statistical, discrete wavelet transform (DWT), the histogram of oriented gradients (HOG), and intensity histogram. After that, the set of optimal features are selected from the mined features using the black widow optimization (BWO) algorithm in the feature selection (FS) process. Finally, a deep Elman neural network (DENN) structure is utilized to categorize the grade of brain tumor. The suggested approach is simulated using the Matlab environment using the BRATS 2012, BRATS 2019 and BRATS 2020 datasets. The different performances are evaluated, and it is related to other existing classifiers and other approaches. The simulation results verified that the accuracy of developed approach is 99.8% for BRATS 2012, 98.7% for BRATS 2019 and 98% for BRATS 2020 as compared to other classifiers and other existing approaches.

Battering Review Spam Through Ensemble Learning in Imbalanced Datasets

Abstract Nowadays, people’s buying or availing services decisions are subject to online available reviews/opinions. The authenticity of these reviews/opinions is dubious, as there exist many fake reviews posted to attain monetary benefits by promoting their own or demoting the competitor’s products or services known as review spam. Although the number of spam is relatively less than that of normal reviews in real-life, this class imbalance is a critical concern in review spam detection. The performance degrades when the classifier skew towards the majority class. Moreover, efficient feature selection is essentially needed for this issue. The purpose of this study is to develop a framework based on different effective feature selection along with data balancing techniques. Validation results show that our proposed framework commendably copes up with the review spam issue and a higher precision on the real-life dataset. Further, we tested the sensitivity of our proposed framework using both parametric and non-parametric tests and found it significant.

IoT-Based Epidemic Monitoring via Improved Gated Recurrent Unit Model

During the Coronavirus Disease 2019 (COVID-19) pandemic, non-contact health monitoring and human activity detection by various sensors have attracted tremendous attention. Robot monitoring will result in minimizing the life threat to health providers during the COVID-19 pandemic period. How to improve the performance and generalization of the monitoring model is a critical but challenging task. This paper constructs an epidemic monitoring architecture based on multi-sensor information fusion and applies it in medical robots’ services, such as patient-care, disinfection, garbage disposal, etc. We propose a gated recurrent unit model based on a genetic algorithm (GA-GRU)to realize the effective feature selection and improve the effectiveness and accuracy of the localization, navigation, and activity monitoring for indoor wireless sensor networks (WSNs). By using two GRU layers in the GA-GRU, we improve the generalization capability in multiple WSNs. All these advantages of GA-GRU make it outperform other representative algorithms in a variety of evaluation metrics. The experiments on the WSNs verify that the proposed GA-GRU leads to successful runs and provides optimal performances. These results suggest the GA-GRU method may be preferable for epidemic monitoring in medicine and allied areas with particular relation to the control of the epidemic or pandemic such as COVID-19 pandemic.

Effective combining of feature selection techniques for machine learning-enabled IoT intrusion detection

The rapid advancement of technologies has enabled businesses to carryout their activities seamlessly and revolutionised communications across the globe. There is a significant growth in the amount and complexity of Internet of Things devices that are deployed in a wider range of environments. These devices mostly communicate through Wi-Fi networks and particularly in smart environments. Besides the benefits, these devices also introduce security challenges. In this paper, we investigate and leverage effective feature selection techniques to improve intrusion detection using machine learning methods. The proposed approach is based on a centralised intrusion detection system, which uses the deep feature abstraction, feature selection and classification to train the model for detecting the malicious and anomalous actions in the traffic. The deep feature abstraction uses deep learning techniques of artificial neural network in the form of unsupervised autoencoder to construct more features for the traffic. Based on the availability of cumulative features, the system then employs a variety of wrapper-based feature selection techniques ranging from SVM and decision tree to Naive Bayes for selecting high-ranked features, which are then combined and fed into an artificial neural network classifier for distinguishing attack and normal behaviors. The experimental results reveal the effectiveness of the proposed method on Aegean Wi-Fi Intrusion Dataset, which achieves high detection accuracy of up to 99.95%, relatively competitive to the existing machine learning works for the same dataset.

Effective Feature Selection for Classification of Malware Families

Effective Feature Selection for Classification of Malware Families

Assessment of Outliers and Detection of Artifactual Network Segments Using Univariate and Multivariate Dispersion Entropy on Physiological Signals.

Network physiology has emerged as a promising paradigm for the extraction of clinically relevant information from physiological signals by moving from univariate to multivariate analysis, allowing for the inspection of interdependencies between organ systems. However, for its successful implementation, the disruptive effects of artifactual outliers, which are a common occurrence in physiological recordings, have to be studied, quantified, and addressed. Within the scope of this study, we utilize Dispersion Entropy (DisEn) to initially quantify the capacity of outlier samples to disrupt the values of univariate and multivariate features extracted with DisEn from physiological network segments consisting of synchronised, electroencephalogram, nasal respiratory, blood pressure, and electrocardiogram signals. The DisEn algorithm is selected due to its efficient computation and good performance in the detection of changes in signals for both univariate and multivariate time-series. The extracted features are then utilised for the training and testing of a logistic regression classifier in univariate and multivariate configurations in an effort to partially automate the detection of artifactual network segments. Our results indicate that outlier samples cause significant disruption in the values of extracted features with multivariate features displaying a certain level of robustness based on the number of signals formulating the network segments from which they are extracted. Furthermore, the deployed classifiers achieve noteworthy performance, where the percentage of correct network segment classification surpasses 95% in a number of experimental setups, with the effectiveness of each configuration being affected by the signal in which outliers are located. Finally, due to the increase in the number of features extracted within the framework of network physiology and the observed impact of artifactual samples in the accuracy of their values, the implementation of algorithmic steps capable of effective feature selection is highlighted as an important area for future research.

Unsupervised soft-label feature selection

Unsupervised feature selection is an important task in various research fields. It is difficult to select the discriminative features under unsupervised scenario due to the absence of label guidance. Recent works employ the pseudo labels to guide feature selection. However, they generate pseudo labels from the original feature space, where noises, redundancies and outliers may degrade the quality of pseudo labels. Besides, they ignore data fuzziness and use hard-labels as the semantic supervision of feature selection, thus the selected features suffer from significant information loss and semantic shortage. To tackle these problems, we propose an effective Unsupervised Soft-label Feature Selection (USFS) model, which performs soft-label learning and simultaneously guides the unsupervised feature selection process with the learned soft-labels. Specifically, we transform the data to low-dimensional subspace where the affinity matrix with sparse constraint is learned based on the local distances. The affinity matrix is determined as the soft-label matrix and further employed to guide the ultimate feature selection process. A simple yet efficient optimization method is derived to iteratively solve the formulated problem. Promising experimental results on widely tested benchmarks demonstrate the superiority of the proposed method compared with state-of-the-art approaches. For the purpose of reproducibility, we provide the code and testing datasets at https://github.com/wang-feifei/USFS-code.

Identification of Gene Signatures and Expression Patterns During Epithelial-to-Mesenchymal Transition From Single-Cell Expression Atlas.

Cancer, which refers to abnormal cell proliferative diseases with systematic pathogenic potential, is one of the leading threats to human health. The final causes for patients’ deaths are usually cancer recurrence, metastasis, and drug resistance against continuing therapy. Epithelial-to-mesenchymal transition (EMT), which is the transformation of tumor cells (TCs), is a prerequisite for pathogenic cancer recurrence, metastasis, and drug resistance. Conventional biomarkers can only define and recognize large tissues with obvious EMT markers but cannot accurately monitor detailed EMT processes. In this study, a systematic workflow was established integrating effective feature selection, multiple machine learning models [Random forest (RF), Support vector machine (SVM)], rule learning, and functional enrichment analyses to find new biomarkers and their functional implications for distinguishing single-cell isolated TCs with unique epithelial or mesenchymal markers using public single-cell expression profiling. Our discovered signatures may provide an effective and precise transcriptomic reference to monitor EMT progression at the single-cell level and contribute to the exploration of detailed tumorigenesis mechanisms during EMT.

Komparasi Information Gain, Gain Ratio, CFs-Bestfirst dan CFs-PSO Search Terhadap Performa Deteksi Anomali

Large data dimensionality is one of the issues in anomaly detection. One approach used to overcome large data dimensions is feature selection. An effective feature selection technique will produce the most relevant features and can improve the classification algorithm to detect attacks. There have been many studies on feature selection techniques, each using different methods and strategies to find the best and relevant features. In this study, a comparison of Information Gain, Gain Ratio, CFs-BestFirst and CFs-PSO Search techniques was compared. The selection features of the four techniques were further validated by the Naive Bayes classification algorithm, k-NN and J48. This study uses the ISCX CICIDS-2017 dataset. Based on the test results the feature selection techniques affect the performance of the Naive Bayes algorithm, k-NN and J48. Increasingly relevant and important features can improve detection performance. The test results also show that the number of features influences the processing / computing time. CFs-BestFirst produces a smaller number of features compared to CFs-PSO Search, Information Gain and Gain Ratio so it requires lower processing time. In addition, k-NN requires a higher processing time than Naive Bayes and J48

An automated breast cancer diagnosis using feature selection and parameter optimization in ANN

Detecting and treating breast cancer at earlier stages is highly proved to improve the survival rate of breast cancer patients as breast cancer is considered a major cause of death worldwide. Classical methods for diagnosing breast cancer depend on human expertise and they incur huge amounts of labor, time and are subject to human error. An Integrated Artificial Immune system and Artificial Bee Colony based breast cancer diagnosis (IAIS-ABC-CDS) is proposed for parallel processing of effective feature selection and parameter optimization in an Artificial Neural Network (ANN). The IAIS-ABC-CDS with Momentum-based Gradient Descent Backpropagation (MBGD) that uses the advantages of Simulated Annealing (SA) for enhancing local search process is compared to the benchmark diagnosis schemes of IAIS-ABC-CDS with Resilient Back-Propagation Techniques (RBPT) and Genetic Algorithm based ANN with Multilayer Perceptron (GA-ANN-MLP) schemes. The proposed IAIS-ABC-CDS is confirmed to produce a mean classification of 99.34% and 99.11% in ANN under the Wisconsin dataset.

An Effective Feature Generation and Selection Approach for Lymph Disease Recognition

An Effective Feature Generation and Selection Approach for Lymph Disease Recognition

Toward a Fair Evaluation and Analysis of Feature Selection for Music Tag Classification

Accurate music tag classification of music clips has been attracting great attention recently, because it allows one to provide various music excerpts, including unpopular ones, to users based on the clips’ acoustic similarities. Given a user’s preferred music, acoustic features are extracted and then fed into the classifier, which outputs the related tag to recommend new music. Furthermore, the accuracy of the tag classifiers can be improved by selecting the best feature subset based on the domain to which the tag belongs. However, recent studies have struggled to evaluate the superiority of various classifiers because they utilize different feature extractors. In this study, to conduct a direct comparison of existing methods of classification, we create 20 music datasets with the same acoustic feature structure. In addition, we propose an effective evolutionary feature selection algorithm to evaluate the effectiveness of feature selection for tag classification. Our experiments demonstrate that the proposed method improves the accuracy of tag classification, and the analysis with multiple datasets provides valuable insights, such as the important features for general music tag classification in target domains.

Intelligent Hybrid Feature Selection for Textual Sentiment Classification

Sentiment Analysis (SA) aims to extract useful information from online Unstructured User-Generated Contents (UUGC) and classify them into positive and negative classes. State-of-the-art techniques for SA suffer a high dimensional feature space because of noisy and irrelevant features from the UUGC. Researchers have also proposed feature extraction and selection techniques to reduce high dimensional feature space, but they fall short in extracting and selecting the most effective sentiment features for sentiment model learning. Effective feature extraction and selection are significant for the SA because they can boost the learning algorithm’s predictive performance while reducing the high-dimensional feature space. To address these concerns, we propose an Intelligent Hybrid Feature Selection for Sentiment Analysis (IHFSSA) based on ensemble learning methods. IHFSSA first identifies sentiment features in the review text utilizing Penn Treebank part-of-speech tagset and integrated Wide Coverage Sentiment Lexicons (WCSL). The sentiment features subset is then selected employing a fast and simple rank-based ensemble of multiple filters feature selection method. The selected sentiment features are further refined by applying a wrapper-based backward feature selection method. Finally, for textual sentiment classification, the well-known classification algorithms Support Vector Machine (SVM), Naive Bayes (NB), Generalized Linear Model (GLM) are trained in the ensemble model on the refined sentiment feature set. The in-depth evaluation using heterogeneous domain benchmark datasets demonstrates that IHFSSA outperforms existing SA techniques.

An Enhanced Binary Particle Swarm Optimization for Optimal Feature Selection in Bearing Fault Diagnosis of Electrical Machines

This study proposes an effective bearing fault diagnosis model based on an optimized approach for feature selection. The measured signal of the electric motor is processed by envelope analysis and Hilbert-Huang transform techniques to extract the potential features. An enhancement of the binary particle swarm optimization algorithm through population initialization strategy based on feature weights, new updating mechanism, and the screening and replacing process create a new and effective feature selection method that improves classification accuracy and reduces data size. The optimal feature subset is provided separately for artificial neural networks, and support vector machine classifier for the final recognition task. In multiple case studies, the proposed feature selection method is evaluated against the benchmark data sets and shows performance comparable to that of other peer competitors. The effectiveness of the proposed bearing fault diagnosis model is verified on different testbeds and achieves high accuracy and robustness under noise conditions. In addition, experimental results are compared with existing fault diagnostic models, showing the high possibility of the proposed bearing fault diagnosis model.

Minimizing the Overlapping Degree to Improve Class-Imbalanced Learning Under Sparse Feature Selection: Application to Fraud Detection

In recent years, the classification of class-imbalanced data has obtained increasing attention across different scientific areas such as fraud detection, metabolomics, Cancer diagnosis, etc. This interest comes after proving the negative effect of overlapping on the performance of class-imbalanced learning. Based on augmented R-value, our proposed strategy aims to select features that make data achieve the minimal overlap degree, so improving the performance of classification as well. In this context, we present three feature selection algorithms RONS (Reduce Overlapping with No-sampling), ROS (Reduce Overlapping with SMOTE), and ROA (Reduce Overlapping with ADASYN), which are built through sparse feature selection to minimize the overlapping and perform binary classification. Also, a re-sampling process has been included in both ROS and ROA. Simulation results show that our proposed algorithms as feature selection methods manage the variation of false discovery rate during the selection of main features for the process modeling. For the experiment, four credit card datasets have been selected to test the performance of our algorithms. Using F-measure and Gmean evaluation metrics, the results reveal that our proposed algorithms are considerably recommended compared with classical feature selection methods. Besides, this effective feature selection strategy can be extended as an alternative to deal with class-imbalanced learning problems that involve overlapping.