Feature Ranking Algorithms Research Articles

Driving Mode Decision Making for Intelligent Vehicles in Stressful Traffic Events

The development of autonomous vehicles provides effective solutions and opportunities for reducing the probability of traffic accidents. However, because of technical limitations and economic and social challenges, achieving fully autonomous driving is a long-term endeavor. One principal research question is how to choose the suitable driving mode of an intelligent vehicle during stressful traffic events. For this purpose, an on-road experiment with 22 drivers was conducted in Wuhan, China; multisensor data were collected from the driver, the vehicle, the road, and the environment. Driving modes were classified into three categories on the basis of the driver’s self-reported records, and two physiological indexes that use the k-means cluster method were adopted to calibrate the self-reported driving modes. A feature-ranking algorithm based on the information gained was adopted to identify significant factors, and a driving mode decision-making model was established with the multiclass support vector machine algorithm. The results indicated that the SD of the front wheel angle, driver experience, vehicle speed, headway time, and acceleration had significant effects on the driving mode decision making. The driving mode decision-making model demonstrated a high predictive power with a prediction accuracy of 0.888 and area under the curve values of 0.918, 0.91, and 0.929 for the receiver operating characteristic curves. The conclusions provide theoretical support for decision making by the controller of a semiautomated vehicle.

Computer Aided Diagnosis System based on Performance Evaluation Agent Model

In this paper, we present a performance evaluation agent based on fuzzy cluster analysis and validity measures. The proposed agent is consists of three modules, fuzzy cluster analyzer, performance evaluation measures, and feature ranking algorithm for feature selection step in CAD system. Feature selection is an important step commonly used to create more accurate system to help human experts. Through this agent, we get the feature ranking on the dataset of mass and calcification lesions extracted from the public real world mammogram database DDSM. Also we design a CAD system incorporating the agent and apply five different feature combinations to the system. Experimental results proposed approach has higher classification accuracy and shows the feasibility as a diagnosis supporting tool.

Insights into the molecular basis of piezophilic adaptation: Extraction of piezophilic signatures

Piezophiles are the organisms which can successfully survive at extreme pressure conditions. However, the molecular basis of piezophilic adaptation is still poorly understood. Analysis of the protein sequence adjustments that had taken place during evolution can help to reveal the sequence adaptation parameters responsible for protein functional and structural adaptation at such high pressure conditions. In this current work we have used SVM classifier for filtering strong instances and generated human interpretable rules from these strong instances by using the PART algorithm. These generated rules were analyzed for getting insights into the molecular signature patterns present in the piezophilic proteins. The experiments were performed on three different temperature ranges piezophilic groups, namely psychrophilic–piezophilic, mesophilic–piezophilic, and thermophilic–piezophilic for the detailed comparative study. The best classification results were obtained as we move up the temperature range from psychrophilic–piezophilic to thermophilic–piezophilic. Based on the physicochemical classification of amino acids and using feature ranking algorithms, hydrophilic and polar amino acid groups have higher discriminative ability for psychrophilic–piezophilic and mesophilic–piezophilic groups along with hydrophobic and nonpolar amino acids for the thermophilic–piezophilic groups. We also observed an overrepresentation of polar, hydrophilic and small amino acid groups in the discriminatory rules of all the three temperature range piezophiles along with aliphatic, nonpolar and hydrophobic groups in the mesophilic–piezophilic and thermophilic–piezophilic groups.

Feature subset selection in dynamic stability assessment power system using artificial neural networks

This paper presents method of feature subset selection in dynamic stability assessment (DSA) power system using artificial neural networks (ANN). In the application of ANN on DSA power system, feature subset selection aims to reduce the number of training features, cost and memory computer. However, the major challenge is to reduce the number of features but classification rate gets a high accuracy. This paper proposes applying Sequential Forward Selection (SFS), Sequential Backward Selection (SBS), Sequential Forward Floating Selection (SFFS) and Feature Ranking (FR) algorithm to feature subset selection. The effectiveness of the algorithms was tested on the GSO-37bus power system. With the same number of features, the calculation results show that SFS algorithm yielded higher classification rate than FR, SBS algorithm. SFS algorithm yielded the same classification rate as SFFS algorithm.

A two-stage Markov blanket based feature selection algorithm for text classification

Designing a good feature selection (FS) algorithm is of utmost importance especially for text classification (TC), wherein a large number of features representing terms or words pose serious challenges to the effectiveness and efficiency of classifiers. FS algorithms are divided into two broad categories, namely, feature ranking (FR) and feature subset selection (FSS) algorithms. Unlike FSS, FR algorithms select those features that are individually highly relevant for the class or category without taking the feature interactions into account. This makes FR algorithms simple and computationally more efficient than FSS and thus, mostly a preferred choice for TC. Bi-normal separation (BNS) (Forman, 2003) and information gain (IG) (Yang and Pedersen, 1997) are well-known FR metrics. However, FR algorithms output a set of highly relevant features or terms which can possibly be redundant and can thus, deteriorate a classifier׳s performance. This paper suggests taking the interactions of words into account in order to eliminate redundant terms. Stand-alone FSS algorithms can be computationally expensive for the high-dimensional text data. We therefore suggest a two-stage FS algorithm, which employs an FR metric such as BNS or IG in the first stage and an FSS algorithm such as the Markov blanket filter (MBF) (Koller and Sahami, 1996) in the second stage. Most of the two-stage algorithms proposed in the literature for TC combine feature ranking and feature transformation such as principal component analysis (PCA) algorithms. To estimate the statistical significance of our two-stage algorithm, we carry out experiments on 10 different splits of training and test sets of each of the three (Reuters-21578, TREC, OHSUMED) data sets with naive Bayes׳ and support vector machines. Our results based on a paired two-sided t-test show that the macro F1 performance of BNS+MBF is statistically significant than that of stand-alone BNS in 69% of the total experimental trials. The macro F1 values of IG get enhanced in 72% of the trials when MBF is used in the second stage. We also compare our two-stage algorithm against two recently proposed FS algorithms, namely, distinguishing feature selector (DFS) (Uysal and Gunal, 2012) and a two stage algorithm consisting of IG and PCA algorithms (Uguz, 2011). BNS+MBF is found to be significantly better than DFS and IG+PCA in 74 and 78% of the trials respectively. IG+MBF outperforms DFS and IG+PCA in 93 and 80% of the experimental trials respectively. Similar results are observed for BNS+MBF and IG+MBF when the performances are evaluated in terms of balanced error rate.

Local energy-based framework for feature ranking

Feature selection has been a research topic with practical significance in pattern recognition, machine learning and data mining. In this paper, a local energy-based framework is proposed to estimate the features' relevance for ranking them. The key idea behind this framework is to transform a complex nonlinear problem into a set of locally linear ones through local energy-based learning. Moreover, the convergence of this framework is analyzed. Some experiments are conducted on benchmark data sets including high dimension small sample size data, such as gene data. The experimental results have shown the correctness of our algorithm derived from this framework and its performance is higher or similar to other classical feature ranking algorithms in most cases.

Computer-Aided Diagnosis for Preoperative Invasion Depth of Gastric Cancer with Dual-Energy Spectral CT Imaging

This study evaluates the accuracy of dual-energy spectral computed tomography (DEsCT) imaging with the aid of computer-aided diagnosis (CAD) system in assessing serosal invasion in patients with gastric cancer. Thirty patients with gastric cancer were enrolled in this study. Two types of features (information) were collected with the use of DEsCT imaging: conventional features including patient's clinical information (eg, age, gender) and descriptive characteristics on the CT images (eg, location of the lesion, wall thickness at the gastric cardia) and additional spectral CT features extracted from monochromatic images (eg, 60keV) and material-decomposition images (eg, iodine- and water-density images). The classification results of the CAD system were compared to pathologic findings. Important features can be found out using support vector machine classification method in combination with feature-selection technique thereby helping the radiologists diagnose better. Statistical analysis showed that for the collected cases, the feature "long axis" was significantly different between group A (serosa negative) and group B (serosa positive) (P < .05). By adding quantitative spectral features from several regions of interest (ROIs), the total classification accuracy was improved from 83.33% to 90.00%. Two feature ranking algorithms were used in the CAD scheme to derive the top-ranked features. The results demonstrated that low single-energy (approximately 60keV) CT values, tumor size (long axis and short axis), iodine (water) density, and Effective-Z values of ROIs were important for classification. These findings concurred with the experience of the radiologist. The CAD system designed using machine-learning algorithms may be used to improve the identification accuracy in the assessment of serosal invasion in patients of gastric cancer with DEsCT imaging and provide some indicators which may be useful in predicting prognosis.

Mining Gene Expression Data of Multiple Sclerosis

ObjectivesMicroarray produces a large amount of gene expression data, containing various biological implications. The challenge is to detect a panel of discriminative genes associated with disease. This study proposed a robust classification model for gene selection using gene expression data, and performed an analysis to identify disease-related genes using multiple sclerosis as an example.Materials and methodsGene expression profiles based on the transcriptome of peripheral blood mononuclear cells from a total of 44 samples from 26 multiple sclerosis patients and 18 individuals with other neurological diseases (control) were analyzed. Feature selection algorithms including Support Vector Machine based on Recursive Feature Elimination, Receiver Operating Characteristic Curve, and Boruta algorithms were jointly performed to select candidate genes associating with multiple sclerosis. Multiple classification models categorized samples into two different groups based on the identified genes. Models’ performance was evaluated using cross-validation methods, and an optimal classifier for gene selection was determined.ResultsAn overlapping feature set was identified consisting of 8 genes that were differentially expressed between the two phenotype groups. The genes were significantly associated with the pathways of apoptosis and cytokine-cytokine receptor interaction. TNFSF10 was significantly associated with multiple sclerosis. A Support Vector Machine model was established based on the featured genes and gave a practical accuracy of ∼86%. This binary classification model also outperformed the other models in terms of Sensitivity, Specificity and F1 score.ConclusionsThe combined analytical framework integrating feature ranking algorithms and Support Vector Machine model could be used for selecting genes for other diseases.

The correctness problem

In machine learning, feature ranking (FR) algorithms are used to rank features by relevance to the class variable. FR algorithms are mostly investigated for the feature selection problem and less s...

A novel feature ranking algorithm for biometric recognition with PPG signals

This study is intended for describing the application of the Photoplethysmography (PPG) signal and the time domain features acquired from its first and second derivatives for biometric identification. For this purpose, a sum of 40 features has been extracted and a feature-ranking algorithm is proposed. This proposed algorithm calculates the contribution of each feature to biometric recognition and collocates the features, the contribution of which is from great to small. While identifying the contribution of the features, the Euclidean distance and absolute distance formulas are used. The efficiency of the proposed algorithms is demonstrated by the results of the k-NN (k-nearest neighbor) classifier applications of the features. During application, each 15-period-PPG signal belonging to two different durations from each of the thirty healthy subjects were used with a PPG data acquisition card. The first PPG signals recorded from the subjects were evaluated as the 1st configuration; the PPG signals recorded later at a different time as the 2nd configuration and the combination of both were evaluated as the 3rd configuration. When the results were evaluated for the k-NN classifier model created along with the proposed algorithm, an identification of 90.44% for the 1st configuration, 94.44% for the 2nd configuration, and 87.22% for the 3rd configuration has successfully been attained. The obtained results showed that both the proposed algorithm and the biometric identification model based on this developed PPG signal are very promising for contactless recognizing the people with the proposed method.

Informatics-aided bandgap engineering for solar materials

This paper predicts the bandgaps of over 200 new chalcopyrite compounds for previously untested chemistries. An ensemble data mining approach involving Ordinary Least Squares (OLS), Sparse Partial Least Squares (SPLS) and Elastic Net/Least Absolute Shrinkage and Selection Operator (Lasso) regression methods coupled to Rough Set (RS) and Principal Component Analysis (PCA) methods was used to develop robust quantitative structure – activity relationship (QSAR) type models for bandgap prediction. The output of the regression analyses is the predicted bandgap for new compounds based on a model using the descriptors most related to bandgap. Feature ranking algorithms were then employed to: (i) assess the connection between bandgap and the chemical descriptors used in the predictive models; and (ii) understand the cause of outliers in the predictions. This paper provides a descriptor guided selection strategy for identifying new potential chalcopyrite chemistries materials for solar cell applications.

The correctness problem: evaluating the ordering of binary features in rankings

In machine learning, feature ranking (FR) algorithms are used to rank features by relevance to the class variable. FR algorithms are mostly investigated for the feature selection problem and less studied for the problem of ranking. This paper focuses on the latter. A question asked about the problem of ranking given in the terminology of FR is: as different FR criteria estimate the relationship between a feature and the class variable differently on a given data, can we determine which criterion better captures the "true" feature-to-class relationship and thus generates the most "correct" order of individual features? This is termed as the "correctness" problem. It requires a reference ordering against which the ranks assigned to features by a FR algorithm are directly compared. The reference ranking is generally unknown for real-life data. In this paper, we show through theoretical and empirical analysis that for two-class classification tasks represented with binary data, the ordering of binary features based on their individual predictive powers can be used as a benchmark. Thus, allowing us to test how correct is the ordering of a FR algorithm. Based on these ideas, an evaluation method termed as FR evaluation strategy (FRES) is proposed. Rankings of three different FR criteria (relief, mutual information, and the diff-criterion) are investigated on five artificially generated and four real-life binary data sets. The results indicate that FRES works equally good for synthetic and real-life data and the diff-criterion generates the most correct orderings for binary data.

Feature Selection Based on Class-Dependent Densities for High-Dimensional Binary Data

Data and knowledge management systems employ feature selection algorithms for removing irrelevant, redundant, and noisy information from the data. There are two well-known approaches to feature selection, feature ranking (FR) and feature subset selection (FSS). In this paper, we propose a new FR algorithm, termed as class-dependent density-based feature elimination (CDFE), for binary data sets. Our theoretical analysis shows that CDFE computes the weights, used for feature ranking, more efficiently as compared to the mutual information measure. Effectively, rankings obtained from both the two criteria approximate each other. CDFE uses a filtrapper approach to select a final subset. For data sets having hundreds of thousands of features, feature selection with FR algorithms is simple and computationally efficient but redundant information may not be removed. On the other hand, FSS algorithms analyze the data for redundancies but may become computationally impractical on high-dimensional data sets. We address these problems by combining FR and FSS methods in the form of a two-stage feature selection algorithm. When introduced as a preprocessing step to the FSS algorithms, CDFE not only presents them with a feature subset, good in terms of classification, but also relieves them from heavy computations. Two FSS algorithms are employed in the second stage to test the two-stage feature selection idea. We carry out experiments with two different classifiers (naive Bayes' and kernel ridge regression) on three different real-life data sets (NOVA, HIVA, and GINA) of the”Agnostic Learning versus Prior Knowledge” challenge. As a stand-alone method, CDFE shows up to about 92 percent reduction in the feature set size. When combined with the FSS algorithms in two-stages, CDFE significantly improves their classification accuracy and exhibits up to 97 percent reduction in the feature set size. We also compared CDFE against the winning entries of the challenge and found that it outperforms the best results on NOVA and HIVA while obtaining a third position in case of GINA.

Energy-based feature ranking for assessing the dysphonia measurements in Parkinson detection

The Parkinson&apos;s disease (PD) detection based on dysphonia has been drawn significant attention. However, all dysphonia measurements differ in the uncontrolled acoustic environments. In order to gain as much reliability as possible, measurements should be assessed and the robust ones are chosen. In this study, motivated by statistical learning theory, the problem of PD detection is addressed to classify the participant as healthy or PD using support vector machine (SVM) with the dysphonia measurements as the input feature vector. Therefore an energy-based feature-ranking algorithm is adopted to assess the dysphonia measurements. Moreover, in order to improve the stability of the proposed algorithm, an ensemble version is also presented where multiple feature-ranking results are aggregated. The experimental results on PD data sets have shown the proposed algorithm outperforms other classic ones, and the ensemble version obtain the higher stability than single one.

Dynamic emulation modelling of a 1D hydrodynamic-ecological model: Tono Dam case study

AbstractOptimal management of water resources systems is often limited by the high computational burden associated to the adoption of process-based models. In this paper we propose a procedural approach for the construction of simple, computationally efficient models that emulate the main dynamics of the original process-based model, but with reduced computational requirements, and that can thus be employed for the resolution of optimal control problems. The core of the procedure is the novel Iterative Feature Ranking algorithm, through which the most relevant variables in explaining the objective function of the control problem are selected among the large set of candidate variables associated with the original process-based model. The final emulation model is then identified using Extremely Randomized Trees. The approach is demonstrated on a real-world case study (Tono Dam, JP).

Support vector-based feature selection using Fisher’s linear discriminant and Support Vector Machine

The problem of feature selection is to find a subset of features for optimal classification. A critical part of feature selection is to rank features according to their importance for classification. The Support Vector Machine (SVM) has been applied to a number of applications, such as bioinformatics, face recognition, text categorization, handwritten digit recognition, and so forth. Based on the success of the SVM, several feature selection algorithms that use it have recently been proposed. This paper proposes a new feature-ranking algorithm based on support vectors (SVs). Support vectors refer to those sample vectors that lie around the decision boundary between two different classes. Although SV-based feature ranking can be applied to any discriminant analysis, two linear discriminants are considered here: Fisher’s linear discriminant and the Support Vector Machine. Features are ranked based on the weight associated with each feature or as determined by recursive feature elimination. The experiments show that our feature-ranking algorithms are competitive in accuracy with the existing methods and much faster.

Towards efficient variables ordering for Bayesian networks classifier

Traditionally, the task of learning Bayesian Networks (BNs) from data has been treated as a NP-Hard search problem. To overcome such difficulty in terms of computational complexity, several approximations have been designed, such as imposing a previous ordering on the domain attributes that restrict the number of Bayesian structures to be learned or using other approaches trying to reduce the state space of this problem. In this paper, we propose a simple method based on feature ranking algorithms which has low computational complexity (O( n 2), where n is the number of variables) and produces good results. We empirically demonstrate that feature ranking algorithms (namely, Chi-Squared and Information Gain) can be used to define efficient variables ordering in the BNC learning context. The proposed method can bring improvements, when using the K2 algorithm, to learn a Bayesian Network Classifier from data.