Feature Ranking Criterion Research Articles

Maximum margin and global criterion based-recursive feature selection

In this research paper, we aim to investigate and address the limitations of recursive feature elimination (RFE) and its variants in high-dimensional feature selection tasks. We identify two main challenges associated with these methods. Firstly, the feature ranking criterion utilized in these approaches is inconsistent with the maximum-margin theory. Secondly, the computation of the criterion is performed locally, lacking the ability to measure the importance of features globally. To overcome these challenges, we propose a novel feature ranking criterion called Maximum Margin and Global (MMG) criterion. This criterion utilizes the classification margin to determine the importance of features and computes it globally, enabling a more accurate assessment of feature importance. Moreover, we introduce an optimal feature subset evaluation algorithm that leverages the MMG criterion to determine the best subset of features. To enhance the efficiency of the proposed algorithms, we provide two alpha seeding strategies that significantly reduce computational costs while maintaining high accuracy. These strategies offer a practical means to expedite the feature selection process. Through extensive experiments conducted on ten benchmark datasets, we demonstrate that our proposed algorithms outperform current state-of-the-art methods. Additionally, the alpha seeding strategies yield significant speedups, further enhancing the efficiency of the feature selection process.

A Unified Multi-Class Feature Selection Framework for Microarray Data.

In feature selection research, simultaneous multi-class feature selection technologies are popular because they simultaneously select informative features for all classes. Recursive feature elimination (RFE) methods are state-of-the-art binary feature selection algorithms. However, extending existing RFE algorithms to multi-class tasks may increase the computational cost and lead to performance degradation. With this motivation, we introduce a unified multi-class feature selection (UFS) framework for randomization-based neural networks to address these challenges. First, we propose a new multi-class feature ranking criterion using the output weights of neural networks. The heuristic underlying this criterion is that "the importance of a feature should be related to the magnitude of the output weights of a neural network". Subsequently, the UFS framework utilizes the original features to construct a training model based on a randomization-based neural network, ranks these features by the criterion of the norm of the output weights, and recursively removes a feature with the lowest ranking score. Extensive experiments on 15 real-world datasets suggest that our proposed framework outperforms state-of-the-art algorithms. The code of UFS is available at https://github.com/SVMrelated/UFS.git.

Orthogonal least squares based fast feature selection for linear classification

An Orthogonal Least Squares (OLS) based feature selection method is proposed for both binomial and multinomial classification. The novel Squared Orthogonal Correlation Coefficient (SOCC) is defined based on Error Reduction Ratio (ERR) in OLS and used as the feature ranking criterion. The equivalence between the canonical correlation coefficient, Fisher’s criterion, and the sum of the SOCCs is revealed, which unveils the statistical implication of ERR in OLS for the first time. It is also shown that the OLS based feature selection method has speed advantages when applied for greedy search. The proposed method is comprehensively compared with the mutual information based feature selection methods and the embedded methods using both synthetic and real world datasets. The results show that the proposed method is always in the top 5 among the 12 candidate methods. Besides, the proposed method can be directly applied to continuous features without discretisation, which is another significant advantage over mutual information based methods.

Ensemble empirical mode decomposition-entropy and feature selection for pantograph fault diagnosis

In this article, a fault diagnosis approach for a pantograph is developed with collected vibration data from a test rig. Ensemble empirical mode decomposition is used to decompose the signals to get intrinsic mode function, and four kinds of entropies (permu1tation entropy, approximate entropy, sample entropy, and fuzzy entropy) reflecting the working state are extracted as the inputs of the support vector machine based on particle swarm optimization algorithm support vector machine. The effect of data length, embedded dimension, and other parameters on calculation of the entropy value has also been studied. Multiple feature ranking criteria are used to select the useful features and improve the fault diagnosis accuracy of certain measurement points. Experimental results on pantograph vibration analysis have then confirmed that the proposed method provides an effective measure for pantograph diagnosis.

ALGORITHM FOR THE DETECTION OF CONGESTIVE HEART FAILURE INDEX

This study documents our efforts to provide computer support for the diagnosis of congestive heart failure (CHF). That computer support takes the form of an index value. A high index value indicates a low probability of CHF, and an index value below a threshold of 25.6 suggests a high probability of CHF. To create that index, we have designed a sophisticated algorithm chain which takes electrocardiogram signals as input. The signals are pre-processed before they are sent to a range of nonlinear feature extraction algorithms. The top 10 feature extraction methods were used to create the CHF index. By using objective feature extraction algorithms, we avoid the problem of inter- and intra-observer variability. We observed that the nonlinear feature extraction methods reflect the nature of the human heart very well. That observation is based on the fact that the nonlinear features achieved low [Formula: see text]-values and high feature ranking criterion scores.

A Novel Single‐Feature and Synergetic‐Features Selection Method by Using ISE‐Based KDE and Random Permutation

The Integrated square error (ISE), as a robust criterion for measuring the difference of densities between two datasets, have been commonly used in pattern recognition. In this paper, two different criteria for evaluating candidate feature subsets are investigated: first, a novel supervised feature selection criterion based on ISE and random permutation of a single feature is proposed, which presents a feature ranking criterion to measure the importance of each feature by computing the ISE over the feature space. Second, a synergetic feature selection criterion is developed. Experimental results on synthetic and real data set show the superior or at least comparable performance compared with existing feature selection algorithms.

Emotion in the singing voice—a deeperlook at acoustic features in the light ofautomatic classification

We investigate the automatic recognition of emotions in the singing voice and study the worth and role of a variety of relevant acoustic parameters. The data set contains phrases and vocalises sung by eight renowned professional opera singers in ten different emotions and a neutral state. The states are mapped to ternary arousal and valence labels. We propose a small set of relevant acoustic features basing on our previous findings on the same data and compare it with a large-scale state-of-the-art feature set for paralinguistics recognition, the baseline feature set of the Interspeech 2013 Computational Paralinguistics ChallengE (ComParE). A feature importance analysis with respect to classification accuracy and correlation of features with the targets is provided in the paper. Results show that the classification performance with both feature sets is similar for arousal, while the ComParE set is superior for valence. Intra singer feature ranking criteria further improve the classification accuracy in a leave-one-singer-out cross validation significantly.

Shape and texture based novel features for automated juxtapleural nodule detection in lung CTs.

Lung cancer is one of the types of cancer with highest mortality rate in the world. In case of early detection and diagnosis, the survival rate of patients significantly increases. In this study, a novel method and system that provides automatic detection of juxtapleural nodule pattern have been developed from cross-sectional images of lung CT (Computerized Tomography). Shape-based and both shape and texture based 7 features are contributed to the literature for lung nodules. System that we developed consists of six main stages called preprocessing, lung segmentation, detection of nodule candidate regions, feature extraction, feature selection (with five feature ranking criteria) and classification. LIDC dataset containing cross-sectional images of lung CT has been utilized, 1410 nodule candidate regions and 40 features have been extracted from 138 cross-sectional images for 24 patients. Experimental results for 10 classifiers are obtained and presented. Adding our derived features to known 33 features has increased nodule recognition performance from 0.9639 to 0.9679 AUC value on generalized linear model regression (GLMR) for 22 selected features and being reached one of the most successful results in the literature.

The correctness problem: evaluating the ordering of binary features in rankings

In machine learning, feature ranking (FR) algorithms are used to rank features by relevance to the class variable. FR algorithms are mostly investigated for the feature selection problem and less studied for the problem of ranking. This paper focuses on the latter. A question asked about the problem of ranking given in the terminology of FR is: as different FR criteria estimate the relationship between a feature and the class variable differently on a given data, can we determine which criterion better captures the "true" feature-to-class relationship and thus generates the most "correct" order of individual features? This is termed as the "correctness" problem. It requires a reference ordering against which the ranks assigned to features by a FR algorithm are directly compared. The reference ranking is generally unknown for real-life data. In this paper, we show through theoretical and empirical analysis that for two-class classification tasks represented with binary data, the ordering of binary features based on their individual predictive powers can be used as a benchmark. Thus, allowing us to test how correct is the ordering of a FR algorithm. Based on these ideas, an evaluation method termed as FR evaluation strategy (FRES) is proposed. Rankings of three different FR criteria (relief, mutual information, and the diff-criterion) are investigated on five artificially generated and four real-life binary data sets. The results indicate that FRES works equally good for synthetic and real-life data and the diff-criterion generates the most correct orderings for binary data.

Feature ranking for support vector machine classification and its application to machinery fault diagnosis

This article provides a feature ranking criterion for multi-class support vector machine classification. In the proposed criterion, feature effectiveness is estimated for individual features by their contributions to class separability in the kernel space. Class separability, measured by cosine similarity, is defined by an objective function that consists of within-class and between-class separabilities. Feature ranking is achieved for individual features by sorting their effectiveness scores. The proposed criterion is validated on University of California Irvine benchmark datasets and also applied to pitting diagnosis for a planetary gearbox. The experimental results demonstrate that the proposed criterion of feature ranking is computationally economic and effective.

Automatically fast determining of feature number for ranking-based feature selection

The proposed feature number determining method for the ranking-based feature selection problem builds a convex hull in high-dimensional space for each category in the training dataset and estimates the discriminative degree by calculating the overlapped proportion of these high-dimensional convex hulls. Normalising these discriminative degrees, an initial selected feature number can be determined, then a local optimal result is output by using the hill climbing algorithm. This approach reduces the time consumed by the existing many ranking-based feature selection methods. Classification results on three data sets using three major feature ranking and selection criteria and an SVM classifier show considerable improvement in time consumed of feature selection and comparable accuracy.

Multilevel analysis of spatiotemporal association features for differentiation of tumor enhancement patterns in breast DCE‐MRI

Analyzing spatiotemporal enhancement patterns is an important task for the differential diagnosis of breast tumors in dynamic contrast-enhanced MRI (DCE-MRI), and yet remains challenging because of complexities in analyzing the time-series of three-dimensional image data. The authors propose a novel approach to breast MRI computer-aided diagnosis (CAD) using a multilevel analysis of spatiotemporal association features for tumor enhancement patterns in DCE-MRI. A database of 171 cases consisting of 111 malignant and 60 benign tumors was used. Time-series contrast-enhanced MR images were obtained from two different types of MR scanners and protocols. The images were first registered for motion compensation, and then tumor regions were segmented using a fuzzy c-means clustering-based method. Spatiotemporal associations of tumor enhancement patterns were analyzed at three levels: Mapping of pixelwise kinetic features within a tumor, extraction of spatial association features from kinetic feature maps, and extraction of kinetic association features at the spatial feature level. A total of 84 initial features were extracted. Predictable values of these features were evaluated with an area under the ROC curve, and were compared between the spatiotemporal association features and a subset of simple form features which do not reflect spatiotemporal association. Several optimized feature sets were identified among the spatiotemporal association feature group or among the simple feature group based on a feature ranking criterion using a support vector machine based recursive feature elimination algorithm. A least-squares support vector machine (LS-SVM) classifier was used for tumor differentiation and the performances were evaluated using a leave-one-out testing. Predictable values of the extracted single features ranged in 0.52-0.75. By applying multilevel analysis strategy, the spatiotemporal association features became more informative in predicting tumor malignancy, which was shown by a statistical testing in ten spatiotemporal association features. By using a LS-SVM classifier with the optimized second and third level feature set, the CAD scheme showed Az of 0.88 in classification of malignant and benign tumors. When this performance was compared to the same LS-SVM classifier with simple form features which do not reflect spatiotemporal association, there was a statistically significant difference (0.88 vs 0.79, p <0.05), suggesting that the multilevel analysis strategy yields a significant performance improvement. The results suggest that the multilevel analysis strategy characterizes the complex tumor enhancement patterns effectively with the spatiotemporal association features, which in turn leads to an improved tumor differentiation. The proposed CAD scheme has a potential for improving diagnostic performance in breast DCE-MRI.

Feature Selection for MLP Neural Network: The Use of Random Permutation of Probabilistic Outputs

This paper presents a new wrapper-based feature selection method for multilayer perceptron (MLP) neural networks. It uses a feature ranking criterion to measure the importance of a feature by computing the aggregate difference, over the feature space, of the probabilistic outputs of the MLP with and without the feature. Thus, a score of importance with respect to every feature can be provided using this criterion. Based on the numerical experiments on several artificial and real-world data sets, the proposed method performs, in general, better than several selected feature selection methods for MLP, particularly when the data set is sparse or has many redundant features. In addition, as a wrapper-based approach, the computational cost for the proposed method is modest.