ReliefF Algorithms Research Articles

Multilabel Feature Selection Using Mutual Information and ML-ReliefF for Multilabel Classification

Recently, multilabel classification algorithms play an increasingly significant role in data mining and machine learning. However, some existing mutual information-based algorithms ignore the influence of the proportions of labels on the correlation degree between features and label sets. Besides, the correlation degree of label sets cannot be accurately measured in most traditional ReliefF algorithms, and the repeated calculation arises from the division of heterogeneous neighbors. To overcome these shortcomings, this paper proposes a multilabel feature selection method using mutual information and improved multilabel ReliefF (ML-ReliefF). First, the proportion of each label is calculated in label space and combined with the mutual information of features and labels to construct a novel correlation degree between features and label sets to preprocess multilabel datasets, which is used to reduce runtime of ML-ReliefF. Second, the mutual information of label sets is introduced into improving accuracy of the correlation degree among label sets. Furthermore, two types of correlation degree for label sets based on ML-ReliefF are developed to divide similar and heterogeneous samples more clearly. Third, a divided method of heterogeneous neighbors is presented to effectively avoid the repeated calculation in ML-ReliefF, and a novel method of feature weighting based on ML-ReliefF is constructed to evaluate the importance of features. Finally, a multilabel feature selection algorithm based on mutual information and ML-ReliefF for multilabel classification is designed to improve the performance of multilabel classification. Experiments under fourteen multilabel datasets show the effectiveness of our algorithm and improve the classification performance for multilabel datasets.

An efficient hybrid filter and evolutionary wrapper approach for sentiment analysis of various topics on Twitter

Sentiment Analysis is currently considered as one of the most attractive research topics in Natural Language Processing (NLP) field. The main objective of sentiment analysis is to identify the opinions and emotions of the users through written contents. While there are different studies that have approached this field using various techniques, it is still considered a challenging topic with many difficulties that are yet to be solved, such as having modern accents, slang words, spelling and grammatical mistakes, and other issues that cannot be overcome with traditional methods and sentiment lexicons. In this work, we propose a hybrid machine learning approach to enhance sentiment analysis; as we build a classification model based on three classes, which are positive, neutral, and negative emotions, using Support Vector Machines (SVM) classifier, while combining two feature selection techniques using the ReliefF and Multi-Verse Optimizer (MVO) algorithms. We also extract more than 6900 tweets from Twitter social network to test our work. Our hybrid method is compared against other classifiers and methods in terms of accuracy. Results show that our proposed method outperforms other techniques and classifiers, by obtaining better results in most of the datasets while reducing the number of features by up to 96.85% from the original feature set. We also categorize the extracted features into Objective, Subjective and Emoticon words to analyze them during the first and the final feature selection processes and find any existing relations. Very similar results are obtained by both feature selection techniques; due to a number of factors that are explained in this paper.

Feature selection with the Fisher score followed by the Maximal Clique Centrality algorithm can accurately identify the hub genes of hepatocellular carcinoma

This study aimed to select the feature genes of hepatocellular carcinoma (HCC) with the Fisher score algorithm and to identify hub genes with the Maximal Clique Centrality (MCC) algorithm. Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) enrichment analysis was performed to examine the enrichment of terms. Gene set enrichment analysis (GSEA) was used to identify the classes of genes that are overrepresented. Following the construction of a protein-protein interaction network with the feature genes, hub genes were identified with the MCC algorithm. The Kaplan–Meier plotter was utilized to assess the prognosis of patients based on expression of the hub genes. The feature genes were closely associated with cancer and the cell cycle, as revealed by GO, KEGG and GSEA enrichment analyses. Survival analysis showed that the overexpression of the Fisher score–selected hub genes was associated with decreased survival time (P < 0.05). Weighted gene co-expression network analysis (WGCNA), Lasso, ReliefF and random forest were used for comparison with the Fisher score algorithm. The comparison among these approaches showed that the Fisher score algorithm is superior to the Lasso and ReliefF algorithms in terms of hub gene identification and has similar performance to the WGCNA and random forest algorithms. Our results demonstrated that the Fisher score followed by the application of the MCC algorithm can accurately identify hub genes in HCC.

Control Oriented Prediction of Driver Brake Intention and Intensity Using a Composite Machine Learning Approach

Driver perception, decision, and control behaviors are easily affected by traffic conditions and driving style, showing the tendency of randomness and personalization. Brake intention and intensity are integrated and control-oriented parameters that are crucial to the development of an intelligent braking system. In this paper, a composite machine learning approach was proposed to predict driver brake intention and intensity with a proper prediction horizon. Various driving data were collected from Controller Area Network (CAN) bus under a real driving condition, which mainly contained urban and rural road types. ReliefF and RReliefF (they don’t have abbreviations) algorithms were employed as feature subset selection methods and applied in a prepossessing step before the training. The rank importance of selected predictors exhibited different trends or even negative trends when predicting brake intention and intensity. A soft clustering algorithm, Fuzzy C-means, was adopted to label the brake intention into categories, namely slight, medium, intensive, and emergency braking. Data sets with misplaced labels were used for training of an ensemble machine learning method, random forest. It was validated that brake intention could be accurately predicted 0.5 s ahead. An open-loop nonlinear autoregressive with external input (NARX) network was capable of learning the long-term dependencies in comparison to the static neural network and was suggested for online recognition and prediction of brake intensity 1 s in advance. As system redundancy and fault tolerance, a close-loop NARX network could be adopted for brake intensity prediction in the case of possible sensor failure and loss of CAN message.

A hybrid measure for the discrimination of the acoustic signals: Feature matrix (FMx)

We introduce a new feature matrix (FMx) to discriminate the acoustic signals with the help of their hybrid characteristics. The FMx has hybrid domain characteristics consisting of feature values such as distributional area (polygonal area), maximum values of the histogram and fundamental frequency of the difference-difference (d2d) vector. To show the performance of the FMx, three different datasets are used together with quadratic discriminant analysis (QDA), multiclass support vector machines (M-SVMs) and convolutional neural networks (CNN). The simulation results show that FMx provides effective and useful information for the discrimination of the signals into subclasses with the help of ReliefF and forward sequential algorithms. In simulations, the test accuracies with QDA, M-SVMs and CNN were obtained as 94.20%, 100% and 100% respectively. So, the results of the simulations support the effectiveness of the FMx for the acoustic signal classification with three different datasets compared to the previous studies.

Diagnosis of mesothelioma with deep learning.

Malignant mesothelioma (MM) is a rare but aggressive cancer. The definitive diagnosis of MM is critical for effective treatment and has important medicolegal significance. However, the definitive diagnosis of MM is challenging due to its composite epithelial/mesenchymal pattern. The aim of the current study was to develop a deep learning method to automatically diagnose MM. A retrospective analysis of 324 participants with or without MM was performed. Significant features were selected using a genetic algorithm (GA) or a ReliefF algorithm performed in MATLAB software. Subsequently, the current study constructed and trained several models based on a backpropagation (BP) algorithm, extreme learning machine algorithm and stacked sparse autoencoder (SSAE) to diagnose MM. A confusion matrix, F-measure and a receiver operating characteristic (ROC) curve were used to evaluate the performance of each model. A total of 34 potential variables were analyzed, while the GA and ReliefF algorithms selected 19 and 5 effective features, respectively. The selected features were used as the inputs of the three models. SSAE and GA+SSAE demonstrated the highest performance in terms of classification accuracy, specificity, F-measure and the area under the ROC curve. Overall, the GA+SSAE model was the preferred model since it required a shorter CPU time and fewer variables. Therefore, the SSAE with GA feature selection was selected as the most accurate model for the diagnosis of MM. The deep learning methods developed based on the GA+SSAE model may assist physicians with the diagnosis of MM.

The effect of feature selection on multivariate pattern analysis of structural brain MR images

Clinical predictions performed using structural magnetic resonance (MR) images are crucial in neuroimaging studies and can be used as a successful complementary method for clinical decision making. Multivariate pattern analysis (MVPA) is a significant tool that helps correct predictions by exhibiting a compound relationship between disease-related features. In this study, the effectiveness of determining the most relevant features for MVPA of the brain MR images are examined using ReliefF and minimum Redundancy Maximum Relevance (mRMR) algorithms to predict the Alzheimer’s disease (AD), schizophrenia, autism, and attention deficit and hyperactivity disorder (ADHD). Three state-of-the-art MVPA algorithms namely support vector machines (SVM), k-nearest neighbor (kNN) and backpropagation neural network (BP-NN) are employed to analyze the images from five different datasets that include 1390 subjects in total. Feature selection is performed on structural brain features such as volumes and thickness of anatomical structures and selected features are used to compare the effect of feature selection on different MVPA algorithms. Selecting the most relevant features for differentiating images of healthy controls from the diseased subjects using both ReliefF and mRMR methods significantly increased the performance. The most successful MVPA method was SVM for all classification tasks.

Feature selection for elderly faller classification based on wearable\xa0sensors

BackgroundWearable sensors can be used to derive numerous gait pattern features for elderly fall risk and faller classification; however, an appropriate feature set is required to avoid high computational costs and the inclusion of irrelevant features. The objectives of this study were to identify and evaluate smaller feature sets for faller classification from large feature sets derived from wearable accelerometer and pressure-sensing insole gait data.MethodsA convenience sample of 100 older adults (75.5 ± 6.7 years; 76 non-fallers, 24 fallers based on 6 month retrospective fall occurrence) walked 7.62 m while wearing pressure-sensing insoles and tri-axial accelerometers at the head, pelvis, left and right shanks. Feature selection was performed using correlation-based feature selection (CFS), fast correlation based filter (FCBF), and Relief-F algorithms. Faller classification was performed using multi-layer perceptron neural network, naïve Bayesian, and support vector machine classifiers, with 75:25 single stratified holdout and repeated random sampling.ResultsThe best performing model was a support vector machine with 78% accuracy, 26% sensitivity, 95% specificity, 0.36 F1 score, and 0.31 MCC and one posterior pelvis accelerometer input feature (left acceleration standard deviation). The second best model achieved better sensitivity (44%) and used a support vector machine with 74% accuracy, 83% specificity, 0.44 F1 score, and 0.29 MCC. This model had ten input features: maximum, mean and standard deviation posterior acceleration; maximum, mean and standard deviation anterior acceleration; mean superior acceleration; and three impulse features. The best multi-sensor model sensitivity (56%) was achieved using posterior pelvis and both shank accelerometers and a naïve Bayesian classifier. The best single-sensor model sensitivity (41%) was achieved using the posterior pelvis accelerometer and a naïve Bayesian classifier.ConclusionsFeature selection provided models with smaller feature sets and improved faller classification compared to faller classification without feature selection. CFS and FCBF provided the best feature subset (one posterior pelvis accelerometer feature) for faller classification. However, better sensitivity was achieved by the second best model based on a Relief-F feature subset with three pressure-sensing insole features and seven head accelerometer features. Feature selection should be considered as an important step in faller classification using wearable sensors.

A Cancer Gene Selection Algorithm Based on the K-S Test and CFS.

Background To address the challenging problem of selecting distinguished genes from cancer gene expression datasets, this paper presents a gene subset selection algorithm based on the Kolmogorov-Smirnov (K-S) test and correlation-based feature selection (CFS) principles. The algorithm selects distinguished genes first using the K-S test, and then, it uses CFS to select genes from those selected by the K-S test. Results We adopted support vector machines (SVM) as the classification tool and used the criteria of accuracy to evaluate the performance of the classifiers on the selected gene subsets. This approach compared the proposed gene subset selection algorithm with the K-S test, CFS, minimum-redundancy maximum-relevancy (mRMR), and ReliefF algorithms. The average experimental results of the aforementioned gene selection algorithms for 5 gene expression datasets demonstrate that, based on accuracy, the performance of the new K-S and CFS-based algorithm is better than those of the K-S test, CFS, mRMR, and ReliefF algorithms. Conclusions The experimental results show that the K-S test-CFS gene selection algorithm is a very effective and promising approach compared to the K-S test, CFS, mRMR, and ReliefF algorithms.

A New Terrain Classification Framework Using Proprioceptive Sensors for Mobile Robots

Mobile robots that operate in real-world environments interact with the surroundings to generate complex acoustics and vibration signals, which carry rich information about the terrain. This paper presents a new terrain classification framework that utilizes both acoustics and vibration signals resulting from the robot-terrain interaction. As an alternative to handcrafted domain-specific feature extraction, a two-stage feature selection method combining ReliefF and mRMR algorithms was developed to select optimal feature subsets that carry more discriminative information. As different data sources can provide complementary information, a multiclassifier combination method was proposed by considering a priori knowledge and fusing predictions from five data sources: one acoustic data source and four vibration data sources. In this study, four conceptually different classifiers were employed to perform the classification, each with a different number of optimal features. Signals were collected using a tracked robot moving at three different speeds on six different terrains. The new framework successfully improved classification performance of different classifiers using the newly developed optimal feature subsets. The greater improvement was observed for robot traversing at lower speeds.

Investigation of automatic feature weighting methods (Fisher, Chi-square and Relief-F) for landslide susceptibility mapping

In landslide susceptibility mapping, factor weights have been usually determined by expert judgements. A novel methodology for weighting landslide causative factors by integrating statistical feature weighting algorithms was proposed. The primary focus of this study is to investigate the effectiveness of automatic feature weighting algorithms, namely Fisher, Chi-square and Relief-F algorithms. Analytic hierarchy process (AHP) method was used as a benchmark method to compare the performances of the weighting algorithms. All weighted factors were tested using factor-weighted overlay method, and quality of these maps was assessed using overall accuracy, area under the ROC curve (AUC) and success rate curve. In addition, Wilcoxon’s signed-rank test was applied to evaluate statistical differences between both estimated overall accuracies and AUCs, respectively. Results showed that the weights determined by feature weighting methods outperformed the conventional AHP method by about 6% and this level of differences was found to be statistically significant.

Gene-gene interaction filtering with ensemble of filters

BackgroundComplex diseases are commonly caused by multiple genes and their interactions with each other. Genome-wide association (GWA) studies provide us the opportunity to capture those disease associated genes and gene-gene interactions through panels of SNP markers. However, a proper filtering procedure is critical to reduce the search space prior to the computationally intensive gene-gene interaction identification step. In this study, we show that two commonly used SNP-SNP interaction filtering algorithms, ReliefF and tuned ReliefF (TuRF), are sensitive to the order of the samples in the dataset, giving rise to unstable and suboptimal results. However, we observe that the ‘unstable’ results from multiple runs of these algorithms can provide valuable information about the dataset. We therefore hypothesize that aggregating results from multiple runs of the algorithm may improve the filtering performance.ResultsWe propose a simple and effective ensemble approach in which the results from multiple runs of an unstable filter are aggregated based on the general theory of ensemble learning. The ensemble versions of the ReliefF and TuRF algorithms, referred to as ReliefF-E and TuRF-E, are robust to sample order dependency and enable a more informative investigation of data characteristics. Using simulated and real datasets, we demonstrate that both the ensemble of ReliefF and the ensemble of TuRF can generate a much more stable SNP ranking than the original algorithms. Furthermore, the ensemble of TuRF achieved the highest success rate in comparison to many state-of-the-art algorithms as well as traditional χ2-test and odds ratio methods in terms of retaining gene-gene interactions.