Robust Feature Selection Method Research Articles

Robust feature selection via simultaneous sapped norm and sparse regularizer minimization

Abstract High dimension is one of the key characters of big data. Feature selection, as a framework to identify a small subset of illustrative and discriminative features, has been proved as a basic solution in dealing with high-dimensional data. In previous literatures, l2, p-norm regularization was studied by many researches as an effective approach to select features across data sets with sparsity. However, l2, p-norm loss function is just robust to noise but not considering the influence of outliers. In this paper, we propose a new robust and efficient feature selection method with emphasizing Simultaneous Capped l2-norm loss and l2, p-norm regularizer Minimization (SCM). The capped l2-norm based loss function can effectively eliminate the influence of noise and outliers in regression and the l2, p-norm regularization is used to select features across data sets with joint sparsity. An efficient approach is then introduced with proved convergence. Extensive experimental studies on synthetic and real-world datasets demonstrate the effectiveness of our method in comparison with other popular feature selection methods.

Reducing covariate factors of gait recognition using feature selection and dictionary-based sparse coding

Human gait recognition is a behavioral biometrics method that aims to determine the identity of individuals through the manner and style of their distinctive walk. It is still a very challenging problem because natural human gait is affected by many covariate factors such as changes in the clothing, variations in viewing angle, and changes in carrying condition. This paper evaluates the most important features of gait under the carrying and clothing conditions. We find that the intra-class variations of the features that remain static during the gait cycle affect the recognition accuracy adversely. Thus, we introduce an effective and robust feature selection method based on the gait energy image. The new gait representation is less sensitive to these covariate factors. We also propose an augmentation technique to overcome some of the problems associated with the intra-class gait fluctuations, as well as if the amount of the training data is relatively small. Finally, we use dictionary learning with sparse coding and linear discriminant analysis to seek the best discriminative data representation before feeding it to the Nearest Centroid classifier. When our method is applied on the large CASIA-B gait data set, we are able to outperform existing gait methods by achieving the highest average result.

Structure Regularized Unsupervised Discriminant Feature Analysis

Feature selection is an important technique in machine learning research. An effective and robust feature selection method is desired to simultaneously identify the informative features and eliminate the noisy ones of data. In this paper, we consider the unsupervised feature selection problem which is particularly difficult as there is not any class labels that would guide the search for relevant features. To solve this, we propose a novel algorithmic framework which performs unsupervised feature selection. Firstly, the proposed framework implements structure learning, where the data structures (including intrinsic distribution structure and the data segment) are found via a combination of the alternative optimization and clustering. Then, both the intrinsic data structure and data segmentation are formulated as regularization terms for discriminant feature selection. The results of the feature selection also affect the structure learning step in the following iterations. By leveraging the interactions between structure learning and feature selection, we are able to capture more accurate structure of data and select more informative features. Clustering and classification experiments on real world image data sets demonstrate the effectiveness of our method.

Human facial expression recognition using curvelet feature extraction and normalized mutual information feature selection

To recognize expressions accurately, facial expression systems require robust feature extraction and feature selection methods. In this paper, a normalized mutual information based feature selection technique is proposed for FER systems. The technique is derived from an existing method, that is, the max-relevance and min-redundancy (mRMR) method. We, however, propose to normalize the mutual information used in this method so that the domination of the relevance or of the redundancy can be eliminated. For feature extraction, curvelet transform is used. After the feature extraction and selection the feature space is reduced by employing linear discriminant analysis (LDA). Finally, hidden Markov model (HMM) is used to recognize the expressions. The proposed FER system (CNF-FER) is validated using four publicly available standard datasets. For each dataset, 10-fold cross validation scheme is utilized. CNF-FER outperformed the existing well-known statistical and state-of-the-art methods by achieving a weighted average recognition rate of 99 % across all the datasets.

An Intelligent Methodology for Railways Monitoring Using Ultrasonic Guided Waves

It is far from trivial to inspect railways for defections. In particular, for the foot area of the rail non destructive testing methods are known to be difficult to apply. In this paper, an ultrasonic guided wave method is considered along with classification methods for automated rail foot defect detection. In effect, given a set of gathered ultrasonic signals, multiple features are extracted from time-, frequency- and time–frequency domains. Next, a robust feature selection method is performed, to collect a small set of complementary features. The classification task is accomplished by means of a kernel-based support vector machine. To demonstrate the performance capabilities of our approach, an extensive experimental setup is designed under representative environmental and operational conditions. The sensitivity and the resolution of the proposed defect detection system are reported. A study on the influence of rail fastening on the proposed method is also reported where robust defect detection rates, greater than 93 %, are achieved assuming that a compact feature subset is considered. However, it is evident in experiments that even in the case of large defects, changes in the environmental conditions (temperature and humidity) increase the interpretation of the acquired signals, thus making the detection task more difficult.

Multiclass classification of sarcomas using pathway based feature selection method

Feature selection is an important research topic in bioinformatics, to date a large number of methods have been developed. Recently several pathway based feature selection protocols, such as the condition-responsive genes method, have been proposed for better classification performance. However, these conventional pathway based methods may lead to the selection of relevant but redundant genes in a given pathway while missing the other useful genes. Also these methods were limited to binary classification, while in many clinical problems a multiclass protocol is preferred such as the classification of sarcomas. Here, we propose a new pathway based feature selection method named Redundancy Removable Pathway based feature selection method (RRP) for the binary and multiclass classification problems. Three classifiers were implemented to compare the performance and gene functions of gene-based, conventional pathway based, and our RRP method. The validation results suggest that the RRP method is a feasible and robust feature selection method for multi-class prediction problems.

Robust feature selection method in high-dimensional data mining

According to the feature of high-dimensional data,the number of variables is usually larger than the sample size and the data are often heterogeneous,a robust and effective feature selection method was proposed by using the dimensional reduction technique of variable selection and the modal regression based estimation method.The estimation algorithm was given by using Local Quadratic Algorithm(LQA) and Expectation-Maximum(EM) algorithm,and the selection method of the parameter adjustment was also discussed.Data analysis of the simulation shows that the proposed method is overall better than the least square and median regression based regularized method.Compared with the existing methods,the proposed method has higher prediction ability and stronger robustness especially for the non-normal error distribution.

Scoring relevancy of features based on combinatorial analysis of Lasso with application to lymphoma diagnosis

One challenge in applying bioinformatic tools to clinical or biological data is high number of features that might be provided to the learning algorithm without any prior knowledge on which ones should be used. In such applications, the number of features can drastically exceed the number of training instances which is often limited by the number of available samples for the study. The Lasso is one of many regularization methods that have been developed to prevent overfitting and improve prediction performance in high-dimensional settings. In this paper, we propose a novel algorithm for feature selection based on the Lasso and our hypothesis is that defining a scoring scheme that measures the "quality" of each feature can provide a more robust feature selection method. Our approach is to generate several samples from the training data by bootstrapping, determine the best relevance-ordering of the features for each sample, and finally combine these relevance-orderings to select highly relevant features. In addition to the theoretical analysis of our feature scoring scheme, we provided empirical evaluations on six real datasets from different fields to confirm the superiority of our method in exploratory data analysis and prediction performance. For example, we applied FeaLect, our feature scoring algorithm, to a lymphoma dataset, and according to a human expert, our method led to selecting more meaningful features than those commonly used in the clinics. This case study built a basis for discovering interesting new criteria for lymphoma diagnosis. Furthermore, to facilitate the use of our algorithm in other applications, the source code that implements our algorithm was released as FeaLect, a documented R package in CRAN.

Fissures Segmentation Using Surface Features: Content-based Retrieval for Mammographic Mass Using Ensemble Classifier

Accurate classification is critical in mammography computer-aided diagnosis using content-based image retrieval approaches (CBIR CAD). The objectives of this study were to: 1) develop an accurate ensemble classifier based on domain knowledge and a robust feature selection method for CBIR CAD; 2) propose three new features; and 3) assess the performance of the proposed method and new features by using a relatively large imaging data set. The data set used in this study consisted of 2114 regions of interest (ROI) extracted from a publicly available image database. The proposed ensemble classifier method we called E-DGA-KNN included four steps. In the first step, 804 ROIs depict masses were divided into five classes according to their boundary types. Then, each class of ROI with an equal number of negative ROIs were put together to create a sub-database. Second, a dual-stage genetic algorithm, which was called DGA, was applied on those five sub-databases for feature selection and weights determination respectively. In the third step, five base K-nearest neighbor (KNN) classifiers were created by using the results of the second step on 2114 ROIs, and five detection scores for a given queried ROI were obtained. Finally, these classifiers are combined to yield a final classification. The performances of the proposed methods were evaluated by using receiver operating characteristic (ROC) analysis. A comparison with eight different methods on the data set was provided which include the stepwise linear discriminative analysis algorithm (SLDA) and particle swarm optimization (PSO) algorithm with KNN classifier. When four hybrid feature selection methods were applied with single KNN classifier (ie, DGA-KNN, SLDA-WGA-KNN, SLDA-PSO-KNN, GA-PSO-KNN) and the proposed E-DGA-KNN method to the data set, the computed areas under the ROC curve (Az) were 0.8782 ± 0.0080, 0.8675 ± 0.0081, 0.8623 ± 0.0083, 0.8725 ± 0.0079, and 0.8927 ± 0.0073, respectively. If all features and single KNN classifier were used, the Az value was 0.8478 ± 0.0088. Az values were 0.8592 ± 0.0083 and 0.8632 ± 0.0081 when SLDA or GA algorithm used alone. In this study, an ensemble classifier based on domain knowledge and a dual-stage feature selection method was proposed. Evaluation results indicated that the proposed method achieved largest value of ROC compared to other algorithms. The proposed method shows better performance and has the potential to improve the performance of CBIR CAD in interpreting and analyzing mammograms.

A Jackknife and Voting Classifier Approach to Feature Selection and Classification

With technological advances now allowing measurement of thousands of genes, proteins and metabolites, researchers are using this information to develop diagnostic and prognostic tests and discern the biological pathways underlying diseases. Often, an investigator’s objective is to develop a classification rule to predict group membership of unknown samples based on a small set of features and that could ultimately be used in a clinical setting. While common classification methods such as random forest and support vector machines are effective at separating groups, they do not directly translate into a clinically-applicable classification rule based on a small number of features.We present a simple feature selection and classification method for biomarker detection that is intuitively understandable and can be directly extended for application to a clinical setting. We first use a jackknife procedure to identify important features and then, for classification, we use voting classifiers which are simple and easy to implement. We compared our method to random forest and support vector machines using three benchmark cancer ‘omics datasets with different characteristics. We found our jackknife procedure and voting classifier to perform comparably to these two methods in terms of accuracy. Further, the jackknife procedure yielded stable feature sets. Voting classifiers in combination with a robust feature selection method such as our jackknife procedure offer an effective, simple and intuitive approach to feature selection and classification with a clear extension to clinical applications.

A high-order feature synthesis and selection algorithm applied to insurance risk modelling

In many jurisdictions, automobile insurers have access to risk-sharing pools to which they can transfer some risks. We consider different feature selection and modelling approaches to maximise profitability of these transfers through better risk selection. For that purpose, we introduce a flexible scoring model and devise a robust feature synthesis and selection method. We show what should be the most suitable sorting criterion depending on pool regulations. We use a technique, similar to cross validation, but that is coherent with the sequential structure of insurance data. We explain how software maturity level impacts profitability.

A Robust Gene selection Method for Microarray-based Cancer Classification

Gene selection is of vital importance in molecular classification of cancer using high-dimensional gene expression data. Because of the distinct characteristics inherent to specific cancerous gene expression profiles, developing flexible and robust feature selection methods is extremely crucial. We investigated the properties of one feature selection approach proposed in our previous work, which was the generalization of the feature selection method based on the depended degree of attribute in rough sets. We compared the feature selection method with the established methods: the depended degree, chi-square, information gain, Relief-F and symmetric uncertainty, and analyzed its properties through a series of classification experiments. The results revealed that our method was superior to the canonical depended degree of attribute based method in robustness and applicability. Moreover, the method was comparable to the other four commonly used methods. More importantly, the method can exhibit the inherent classification difficulty with respect to different gene expression datasets, indicating the inherent biology of specific cancers.