Feature Subset Selection Problem Research Articles

A parsimonious threshold-independent protein feature selection method through the area under receiver operating characteristic curve

Protein expression profiling for differences indicative of early cancer holds promise for improving diagnostics. Due to their high dimensionality, statistical analysis of proteomic data from mass spectrometers is challenging in many aspects such as dimension reduction, feature subset selection as well as construction of classification rules. Search of an optimal feature subset, commonly known as the feature subset selection (FSS) problem, is an important step towards disease classification/diagnostics with biomarkers. We develop a parsimonious threshold-independent feature selection (PTIFS) method based on the concept of area under the curve (AUC) of the receiver operating characteristic (ROC). To reduce computational complexity to a manageable level, we use a sigmoid approximation to the empirical AUC as the criterion function. Starting from an anchor feature, the PTIFS method selects a feature subset through an iterative updating algorithm. Highly correlated features that have similar discriminating power are precluded from being selected simultaneously. The classification rule is then determined from the resulting feature subset. The performance of the proposed approach is investigated by extensive simulation studies, and by applying the method to two mass spectrometry data sets of prostate cancer and of liver cancer. We compare the new approach with the threshold gradient descent regularization (TGDR) method. The results show that our method can achieve comparable performance to that of the TGDR method in terms of disease classification, but with fewer features selected. Supplementary Material and the PTIFS implementations are available at http://staff.ustc.edu.cn/~ynyang/PTIFS. Supplementary data are available at Bioinformatics online.

Solving feature subset selection problem by a Parallel Scatter Search

The aim of this paper is to develop a Parallel Scatter Search metaheuristic for solving the Feature Subset Selection Problem in classification. Given a set of instances characterized by several features, the classification problem consists of assigning a class to each instance. Feature Subset Selection Problem selects a relevant subset of features from the initial set in order to classify future instances. We propose two methods for combining solutions in the Scatter Search metaheuristic. These methods provide two sequential algorithms that are compared with a recent Genetic Algorithm and with a parallelization of the Scatter Search. This parallelization is obtained by running simultaneously the two combination methods. Parallel Scatter Search presents better performance than the sequential algorithms.

Fast leave-one-out evaluation for dynamic gene selection

Gene selection procedure is a necessary step to increase the accuracy of machine learning algorithms that help in disease diagnosis based on gene expression data. This is commonly known as a feature subset selection problem in machine learning domain. A fast leave-one-out (LOO) evaluation formula for least-squares support vector machines (LS-SVMs) is introduced here that can guide our backward feature selection process. Based on that, we propose a fast LOO guided feature selection (LGFS) algorithm. The gene selection step size is dynamically adjusted according to the LOO accuracy estimation. For our experiments, the application of LGFS to the gene selection process improves the classifier accuracy and reduces the number of features required as well. The least number of genes that can maximize the disease classification accuracy is automatically determined by our algorithm.

Filter- versus wrapper-based feature selection for credit scoring

We address the problem of credit scoring as a classification and feature subset selection problem. Based on the current framework of sophisticated feature selection methods, we identify features that contain the most relevant information to distinguish good loan payers from bad loan payers. The feature selection methods are validated on several real-world datasets with different types of classifiers. We show the advantages following from using the subspace approach to classification. We discuss many practical issues related to the applicability of feature selection methods. We show and discuss some difficulties that used to be insufficiently emphasized in standard feature selection literature.

The k-F eature S et problem is W[2]-complete

We prove the W[2]-completeness of the feature subset selection problem when the cardinality of the subset is the parameter. Aside from the many applications the problem has in data mining literature, the problem is highly relevant in Computational Biology since it arises in differential gene expression analysis using microarray technologies. It is also related to genetic-based prognosis and regulatory interaction discovery using DNA chip technologies.

Feature subset selection by Bayesian networks: a comparison with genetic and sequential algorithms

In this paper we perform a comparison among FSS–EBNA, a randomized, population-based and evolutionary algorithm, and two genetic and other two sequential search approaches in the well-known feature subset selection (FSS) problem. In FSS–EBNA, the FSS problem, stated as a search problem, uses the estimation of Bayesian network algorithm (EBNA) search engine, an algorithm within the estimation of distribution algorithm (EDA) approach. The EDA paradigm is born from the roots of the genetic algorithm (GA) community in order to explicitly discover the relationships among the features of the problem and not disrupt them by genetic recombination operators. The EDA paradigm avoids the use of recombination operators and it guarantees the evolution of the population of solutions and the discovery of these relationships by the factorization of the probability distribution of best individuals in each generation of the search. In EBNA, this factorization is carried out by a Bayesian network induced by a cheap local search mechanism. FSS–EBNA can be seen as a hybrid Soft Computing system, a synergistic combination of probabilistic and evolutionary computing to solve the FSS task. Promising results on a set of real Data Mining domains are achieved by FSS–EBNA in the comparison respect to well-known genetic and sequential search algorithms.

Generalized rough sets based feature selection

The problem of feature subset selection can be defined as the selection of a relevant subset of features which allows a learning algorithm to induce small high-accuracy models. This problem is of primary important because irrelevant and redundant features may degrade the learner speed, especially in the context of high dimensionality, and reduce both the accuracy and comprehensibility of the induced model. Two main approaches have been developed, the first one is algorithm-independent (filter approach) which considers only the data, when the second approach which is algorithm-dependent takes into account both the data and a given learning algorithm (wrapper approach). Recent work was developed to study the interest of the rough set theory and more particularly its notions of reducts and core to deal with the problem of feature subset selection. Different methods were proposed to select features using both the core and the reduct concepts, whereas other researches show that useful feature subsets do not necessarily contain all features in cores. In this paper, we underline the fact that rough set theory is concerned with deterministic analysis of attribute dependencies which are at the basis of the two notions of reduct and core. We extend the notion of dependency which allows to find both deterministic and non-deterministic dependencies. A new notion of strong reducts is then introduced and leads to the definition of strong feature subsets (SFS). The interest of SFS is illustrated by the improvement of the accuracy of C4.5 on real-world datasets. Our study shows that generally the highest-accuracy-subset is not the best one as regards to the filter criteria. The highest accuracy subset is found by the new approach with minimum cost. The contribution of this work is four folds : (1) analysis of feature subset selection in the rough sets context, (2) introduction of new definitions based on a generalized rough set theory, i.e., \alpha-RST, (3) reformulation of the selection problem, (4) description of a hybrid method combining combining both the filter and the wrapper approaches.

Towards operational knowledge-based remote-sensing image analysis

This paper focuses on the problem of cost-based feature subset selection in the framework of the verification and updating of land-cover maps with imagery from different space-borne sensors. The concept of objects is applied and prior knowledge about an object (polygon under analysis) and its direct neighbors is combined with knowledge about the sensor and utilized in a feature subset selection step.

Wrappers for feature subset selection

In the feature subset selection problem, a learning algorithm is faced with the problem of selecting a relevant subset of features upon which to focus its attention, while ignoring the rest. To achieve the best possible performance with a particular learning algorithm on a particular training set, a feature subset selection method should consider how the algorithm and the training set interact. We explore the relation between optimal feature subset selection and relevance. Our wrapper method searches for an optimal feature subset tailored to a particular algorithm and a domain. We study the strengths and weaknesses of the wrapper approach and show a series of improved designs. We compare the wrapper approach to induction without feature subset selection and to Relief, a filter approach to feature subset selection. Significant improvement in accuracy is achieved for some datasets for the two families of induction algorithms used: decision trees and Naive-Bayes.

The minimum feature subset selection problem

In applications of learning from examples to real-world tasks, feature subset selection is important to speed up training and to improve generalization performance. Ideally, an inductive algorithm should use subset of features as small as possible. In this paper however, the authors show that the problem of selecting the minimum subset of features is NP-hard. The paper then presents a greedy algorithm for feature subset selection. The result of running the greedy algorithm on hand-written numeral recognition problem is also given.