Wrapper-based Feature Selection Method Research Articles

A semi-supervised feature selection method using a non-parametric technique with pairwise instance constraints

Many feature selection methods based on supervised learning have investigated side information that considers labelled instances. In this paper, we present a new feature selection method for semi-supervised learning, helped by side information, and the selected features are useful for discovering natural clusters. To select salient features for clustering, most existing wrapper-based feature selection methods study features from the partitioned clusters that result from a prespecified clustering algorithm. The selected features often suffer from both noisy features participating in the partitioning process and bias in the clustering algorithm selection. To overcome these problems, our method does not need clustered information to train features. In particular, our method uses side information to select features directly from the nearest and the farthest neighbours regardless of any clustering algorithm helping to partition data into clusters; specifically, the side information is considered to form pairwise instance constraints to guide the feature selection process and assign a weight for each instance for training. Experimental results indicate that our method, tested on image datasets, outperforms many well known wrapper-based feature selection methods in selecting features for clustering.

Comparison of classification algorithms with wrapper-based feature selection for predicting osteoporosis outcome based on genetic factors in a taiwanese women population.

An essential task in a genomic analysis of a human disease is limiting the number of strongly associated genes when studying susceptibility to the disease. The goal of this study was to compare computational tools with and without feature selection for predicting osteoporosis outcome in Taiwanese women based on genetic factors such as single nucleotide polymorphisms (SNPs). To elucidate relationships between osteoporosis and SNPs in this population, three classification algorithms were applied: multilayer feedforward neural network (MFNN), naive Bayes, and logistic regression. A wrapper-based feature selection method was also used to identify a subset of major SNPs. Experimental results showed that the MFNN model with the wrapper-based approach was the best predictive model for inferring disease susceptibility based on the complex relationship between osteoporosis and SNPs in Taiwanese women. The findings suggest that patients and doctors can use the proposed tool to enhance decision making based on clinical factors such as SNP genotyping data.

Feature Selection Using Probabilistic Prediction of Support Vector Regression

This paper presents a new wrapper-based feature selection method for support vector regression (SVR) using its probabilistic predictions. The method computes the importance of a feature by aggregating the difference, over the feature space, of the conditional density functions of the SVR prediction with and without the feature. As the exact computation of this importance measure is expensive, two approximations are proposed. The effectiveness of the measure using these approximations, in comparison to several other existing feature selection methods for SVR, is evaluated on both artificial and real-world problems. The result of the experiments show that the proposed method generally performs better than, or at least as well as, the existing methods, with notable advantage when the dataset is sparse.

Feature Selection for MLP Neural Network: The Use of Random Permutation of Probabilistic Outputs

This paper presents a new wrapper-based feature selection method for multilayer perceptron (MLP) neural networks. It uses a feature ranking criterion to measure the importance of a feature by computing the aggregate difference, over the feature space, of the probabilistic outputs of the MLP with and without the feature. Thus, a score of importance with respect to every feature can be provided using this criterion. Based on the numerical experiments on several artificial and real-world data sets, the proposed method performs, in general, better than several selected feature selection methods for MLP, particularly when the data set is sparse or has many redundant features. In addition, as a wrapper-based approach, the computational cost for the proposed method is modest.

A comparison of classification methods for predicting Chronic Fatigue Syndrome based on genetic data.

BackgroundIn the studies of genomics, it is essential to select a small number of genes that are more significant than the others for the association studies of disease susceptibility. In this work, our goal was to compare computational tools with and without feature selection for predicting chronic fatigue syndrome (CFS) using genetic factors such as single nucleotide polymorphisms (SNPs).MethodsWe employed the dataset that was original to the previous study by the CDC Chronic Fatigue Syndrome Research Group. To uncover relationships between CFS and SNPs, we applied three classification algorithms including naive Bayes, the support vector machine algorithm, and the C4.5 decision tree algorithm. Furthermore, we utilized feature selection methods to identify a subset of influential SNPs. One was the hybrid feature selection approach combining the chi-squared and information-gain methods. The other was the wrapper-based feature selection method.ResultsThe naive Bayes model with the wrapper-based approach performed maximally among predictive models to infer the disease susceptibility dealing with the complex relationship between CFS and SNPs.ConclusionWe demonstrated that our approach is a promising method to assess the associations between CFS and SNPs.

Effective Discretization and Hybrid feature selection using Naïve Bayesian classifier for Medical datamining

As a probability-based statistical classification method, the Naive Bayesian classifier has gained wide popularity despite its assumption that attributes are conditionally mutually independent given the class label. Improving the predictive accuracy and achieving dimensionality reduction for statistical classifiers has been an active research area in datamining. Our experimental results suggest that on an average, with Minimum Description Length (MDL) discretization the Naive Bayes Classifier seems to be the best performer compared to popular variants of Naive Bayes as well as some popular non-Naive Bayesian statistical classifiers. We propose a Hybrid feature selection algorithm (CHI-WSS) that helps in achieving dimensionality reduction by removing irrelevant data, increasing learning accuracy and improving result comprehensibility. Experimental results suggest that on an average the Hybrid Feature Selector gave best results compared to individual techniques with popular filter as well as wrapper based feature selection methods. The proposed algorithm which is a multi-step process utilizes discretization, filters out irrelevant and least relevant features and finally uses a greedy algorithm such as best first search or wrapper subset selector. For experimental validation we have utilized two established measures to compare the performance of statistical classifiers namely; classification accuracy (or error rate) and the area under ROC. Our work demonstrates that the proposed algorithm using generative Naive Bayesian classifier on the average is more efficient than using discriminative models namely Logistic Regression and Support Vector Machine. This work based on empirical evaluation on publicly available datasets validates our hypothesis of development of parsimonious models from our generalized approach.