Feature Subset Selection Algorithm Research Articles

Leafcutter Ant Colony Optimization Algorithm for Feature Subset Selection on Classifying Digital Mammograms

Ant Colony Optimization (ACO) has been applied in wide range of applications. In ACO, for every iteration the entire problem space is considered for the solution construction using the probability of the pheromone deposits. After convergence, the global solution is made with the path which has highest pheromone deposit. In this paper, a novel solution construction technique has been proposed to reduce the time complexity and to improve the performance of the ACO. The idea is derived from the behavior of a special ant species called ‘Leafcutter Ants', they spend much of their time for cutting leaves to make fertilizer to gardens in which they grow the fungi that they eat. This behavior is incorporated with the general ACO algorithm to propose a novel feature selection method called ‘Leafcutter Ant Colony Optimization' (LACO) algorithm. The LACO has been applied to select the relevant features for digital mammograms and their corresponding classification performance is studied and compared.

IFSS An Improved Filter-Wrapper Algorithm for Feature Subset Selection

The ever increasing growth of databases in the real time application is a major issue for the handling of large data. The data mining of the same is also a tedious task. The feature subset selection is a process for finding the irrelevant and redundant data and handling them. The proposed algorithm IFSSImproved Feature Subset Selection works in 2 major steps: 1. Find the irrelevant features and 2. Evaluate its fitness with Ant Colony Optimization (ACO). The Computation time taken to derive the results is taken to compare with different FSS algorithms.

Suboptimal branch and bound algorithms for feature subset selection: A comparative study

The branch and bound algorithm is an optimal feature selection method that is well-known for its computational efficiency. However, when the dimensionality of the original feature space is large, the computational time of the branch and bound algorithm becomes very excessive. If the optimality of the solution is allowed to be compromised, one can further improve the search speed of the branch and bound algorithm; the look-ahead search strategy can be employed to eliminate many solutions deemed to be suboptimal early in the search. In this paper, a comparative study of the look-ahead scheme in terms of the computational cost and the solution quality on four major branch and bound algorithms is carried out on real data sets. We also explore the use of suboptimal branch and bound algorithms on a high-dimensional data set and compare its performance with other well-known suboptimal feature selection algorithms.

Speeding up incremental wrapper feature subset selection with Naive Bayes classifier

This paper deals with the problem of wrapper feature subset selection (FSS) in classification-oriented datasets with a (very) large number of attributes. In high-dimensional datasets with thousands of variables, wrapper FSS becomes a laborious computational process because of the amount of CPU time it requires. In this paper we study how under certain circumstances the wrapper FSS process can be speeded up by embedding the classifier into the wrapper algorithm, instead of dealing with it as a black-box. Our proposal is based on the combination of the NB classifier (which is known to be largely beneficial for FSS) with incremental wrapper FSS algorithms. The merit of this approach is analyzed both theoretically and experimentally, and the results show an impressive speed-up for the embedded FSS process.

Evolutionary computation for feature selection in classification problems

Feature subset selection (FSS) has received a great deal of attention in statistics, machine learning, and data mining. Real world data analyzed by data mining algorithms can involve a large number of redundant or irrelevant features or simply too many features for a learning algorithm to handle them efficiently. Feature selection is becoming essential as databases grow in size and complexity. The selection process is expected to bring benefits in terms of better performing models, computational efficiency, and simpler more understandable models. Evolutionary computation (EC) encompasses a number of naturally inspired techniques such as genetic algorithms, genetic programming, ant colony optimization, or particle swarm optimization algorithms. Such techniques are well suited to feature selection because the representation of a feature subset is straightforward and the evaluation can also be easily accomplished through the use of wrapper or filter algorithms. Furthermore, the capability of such heuristic algorithms to efficiently search large search spaces is of great advantage to the feature selection problem. Here, we review the use of different EC paradigms for feature selection in classification problems. We discuss details of each implementation including representation, evaluation, and validation. The review enables us to uncover the best EC algorithms for FSS and to point at future research directions. WIREs Data Mining Knowl Discov 2013, 3:381–407. doi: 10.1002/widm.1106This article is categorized under: Technologies > Classification Technologies > Computational Intelligence Technologies > Data Preprocessing

A hybrid genetic algorithm for feature subset selection in rough set theory

Rough set theory has been proven to be an effective tool to feature subset selection. Current research usually employ hill-climbing as search strategy to select feature subset. However, they are inadequate to find the optimal feature subset since no heuristic can guarantee optimality. Due to this, many researchers study stochastic methods. Since previous works of combination of genetic algorithm and rough set theory do not show competitive performance compared with some other stochastic methods, we propose a hybrid genetic algorithm for feature subset selection in this paper, called HGARSTAR. Different from previous works, HGARSTAR embeds a novel local search operation based on rough set theory to fine-tune the search. This aims to enhance GA's intensification ability. Moreover, all candidates (i.e. feature subsets) generated in evolutionary process are enforced to include core features to accelerate convergence. To verify the proposed algorithm, experiments are performed on some standard UCI datasets. Experimental results demonstrate the efficiency of our algorithm.

A novel feature subset selection algorithm based on association rule mining

In this paper, a novel feature selection algorithm FEAST is proposed based on association rule mining. The proposed algorithm first mines association rules from a data set; then, it identifies the relevant and interactive feature values with the constraint association rules whose consequent is the target concept, detects and eliminates the redundant feature values with the constraint association rules whose consequent and antecedent are both of single feature value. Finally, it obtains the feature subset by mapping the feature values to the corresponding features. As the support and confidence thresholds are two important parameters in association rule mining and play a vital role in FEAST, a partial least square regression PLSR based threshold prediction method is presented as well. The effectiveness of FEAST is tested on both synthetic and real world data sets, and the classification results of five different types of classifiers with seven representative feature selection algorithms are compared. The results on the synthetic data sets show that FEAST can effectively identify irrelevant and redundant features while reserving interactive ones. The results on the real world data sets show that FEAST outperforms other feature selection algorithms in terms of classification accuracies. In addition, the PLSR based threshold prediction method is performed on the real world data sets, and the results show it works well in recommending proper support and confidence thresholds for FEAST.

Research on Pattern Recognition Applied for Volume Estimation of Blood Chamber with Matrix of Optical Sensors

The following article describes research on possibility of using pattern recognition algorithms in the optical measurement system for estimation of the blood chamber volume in the Polish Ventricular Assist Device (POLVAD). The optical system is being developed at the Department of Optoelectronics, Silesian University of Technology, Poland. Data analysis methods include a feature subset selection algorithm involving principal components analysis and objective function as quality criterion. The analysis takes into account 17 patterns re ecting particular volumes. The k-nearest neighbours method is used as pattern classi er. The pattern recognition system was initially designed for an array of gas sensors and this paper describes its further development.

A Briefest Feature Subset Selection Algorithm Based on Preference Attribute

Concept lattice is the core data structure of formal concept analysis and represents the order relationship between the concepts iconically. Feature selection has been the focus of research in machine learning.And feature selection has been shown very effective in removing irrelevant and redundant features,also increasing efficiency in learning process and obtaining more intelligible learned results.This paper proposes a new briefest feature subset selection algorithm based on preference attribute on the basis of study of concept lattice theory. User can put forward a preference attribute according to their subjective experiences, all the briefest feature subsets containing the given attribute can be discovered by the algorithm. It firstly find some special concept pairs and calculate their waned-value hypergraph, then obtain the minimal transversal of the hypergraph as a result. A practical example proves the method is cogent and effective.

A Feature Subset Selection Algorithm Automatic Recommendation Method

Many feature subset selection (FSS) algorithms have been proposed, but not all of them are appropriate for a given feature selection problem. At the same time, so far there is rarely a good way to choose appropriate FSS algorithms for the problem at hand. Thus, FSS algorithm automatic recommendation is very important and practically useful. In this paper, a meta learning based FSS algorithm automatic recommendation method is presented. The proposed method first identifies the data sets that are most similar to the one at hand by the k-nearest neighbor classification algorithm, and the distances among these data sets are calculated based on the commonly-used data set characteristics. Then, it ranks all the candidate FSS algorithms according to their performance on these similar data sets, and chooses the algorithms with best performance as the appropriate ones. The performance of the candidate FSS algorithms is evaluated by a multi-criteria metric that takes into account not only the classification accuracy over the selected features, but also the runtime of feature selection and the number of selected features. The proposed recommendation method is extensively tested on 115 real world data sets with 22 well-known and frequently-used different FSS algorithms for five representative classifiers. The results show the effectiveness of our proposed FSS algorithm recommendation method.

面向田间籽棉成熟度判别的二种特征选择算法比较

To discriminate the ripeness of cotton quickly and accurately,15shape structure features were extracted from cotton images and the execute efficiencies and classification accuracy of their feature subset selection algorithms such as Wrapper-based Exhaustive searching and Wrapper-based stopping(WE-W)and Filter-based Heuristic searching and Wrapper-based stopping(FH-W)were compared by using 10-fold cross-validation.By taking the error rate of a Bayes classifier on validation set(WE-W)and the class-separability measuring value on a training set(FH-W)as assessing functions,the optimal l(l=1,2,3,…,15)feature subset was searched by using exhaustive(WE-W)and heuristic(FH-W)strategies on the training set,which stops at the minimum error rate of Bayes-classifier on the validation set(WE-W and FH-W).Experimental results show that the average classification rates of WE-W and FH-W algorithms on the prediction set are 85.39%(WE-W)and 85.28%(FHW)at l=3,respectively.It concludes that the FH-W algorithm can be a reference in practice for its higher execute efficiency and good classification accuracy.

Selecting Optimal Feature Set in High-Dimensional Data by Swarm Search

Selecting the right set of features from data of high dimensionality for inducing an accurate classification model is a tough computational challenge. It is almost a NP-hard problem as the combinations of features escalate exponentially as the number of features increases. Unfortunately in data mining, as well as other engineering applications and bioinformatics, some data are described by a long array of features. Many feature subset selection algorithms have been proposed in the past, but not all of them are effective. Since it takes seemingly forever to use brute force in exhaustively trying every possible combination of features, stochastic optimization may be a solution. In this paper, we propose a new feature selection scheme called Swarm Search to find an optimal feature set by using metaheuristics. The advantage of Swarm Search is its flexibility in integrating any classifier into its fitness function and plugging in any metaheuristic algorithm to facilitate heuristic search. Simulation experiments are carried out by testing the Swarm Search over some high-dimensional datasets, with different classification algorithms and various metaheuristic algorithms. The comparative experiment results show that Swarm Search is able to attain relatively low error rates in classification without shrinking the size of the feature subset to its minimum.

A Fast Clustering-Based Feature Subset Selection Algorithm for High-Dimensional Data

Feature selection involves identifying a subset of the most useful features that produces compatible results as the original entire set of features. A feature selection algorithm may be evaluated from both the efficiency and effectiveness points of view. While the efficiency concerns the time required to find a subset of features, the effectiveness is related to the quality of the subset of features. Based on these criteria, a fast clustering-based feature selection algorithm (FAST) is proposed and experimentally evaluated in this paper. The FAST algorithm works in two steps. In the first step, features are divided into clusters by using graph-theoretic clustering methods. In the second step, the most representative feature that is strongly related to target classes is selected from each cluster to form a subset of features. Features in different clusters are relatively independent, the clustering-based strategy of FAST has a high probability of producing a subset of useful and independent features. To ensure the efficiency of FAST, we adopt the efficient minimum-spanning tree (MST) clustering method. The efficiency and effectiveness of the FAST algorithm are evaluated through an empirical study. Extensive experiments are carried out to compare FAST and several representative feature selection algorithms, namely, FCBF, ReliefF, CFS, Consist, and FOCUS-SF, with respect to four types of well-known classifiers, namely, the probability-based Naive Bayes, the tree-based C4.5, the instance-based IB1, and the rule-based RIPPER before and after feature selection. The results, on 35 publicly available real-world high-dimensional image, microarray, and text data, demonstrate that the FAST not only produces smaller subsets of features but also improves the performances of the four types of classifiers.

Pattern Recognition Applied to Analysis of Gas Sensors' Array Data

For an array of eight chemoresistive gas sensors a computer pattern recognition system was built. Multivariate data analysis was performed for measurements of three gases' dilutions: hydrogen (H2), methane (CH4), and carbon monoxide (CO). The pattern recognition system included a feature subset selection algorithm involving PCA and objective function. Dimensionality reduction was applied to two kinds of patterns: three aforementioned gases and six di erent concentrations of hydrogen. For patterns of the three gases, classi cation tests were performed using k-NN algorithm and N -fold based validation method.

Selecting feature subset for high dimensional data via the propositional FOIL rules

Feature interaction is an important issue in feature subset selection. However, most of the existing algorithms only focus on dealing with irrelevant and redundant features. In this paper, a propositional FOIL rule based algorithm FRFS, which not only retains relevant features and excludes irrelevant and redundant ones but also considers feature interaction, is proposed for selecting feature subset for high dimensional data. FRFS first merges the features appeared in the antecedents of all FOIL rules, achieving a candidate feature subset which excludes redundant features and reserves interactive ones. Then, it identifies and removes irrelevant features by evaluating features in the candidate feature subset with a new metric CoverRatio, and obtains the final feature subset. The efficiency and effectiveness of FRFS are extensively tested upon both synthetic and real world data sets, and it is compared with other six representative feature subset selection algorithms, including CFS, FCBF, Consistency, Relief-F, INTERACT, and the rule-based FSBAR, in terms of the number of selected features, runtime and the classification accuracies of the four well-known classifiers including Naive Bayes, C4.5, PART and IB1 before and after feature selection. The results on the five synthetic data sets show that FRFS can effectively identify irrelevant and redundant features while reserving interactive ones. The results on the 35 real world high dimensional data sets demonstrate that compared with other six feature selection algorithms, FRFS cannot only efficiently reduce the feature space, but also can significantly improve the performance of the four well-known classifiers.

Analysis of the effectiveness of control measures to mitigate road dust emissions in a regional network

The effectiveness of control measures to reduce road dust emissions is analyzed using a year’s data of road dust emissions collected with a mobile sampling platform and a survey of road maintenance practices in the Lake Tahoe Basin of Nevada and California US. Attributes such as sweeping practices, anti-icing, shoulder improvement, pavement condition, trackout, and abrasive material from road segments were analyzed with a feature subset selection algorithm. Street sweeping was found to be an effective means of controlling dust emissions from roads. Road dust from dirty tertiary roads served as a continuous source of suspendable material for adjacent high-speed roads in the winter time. To be most effective, emission control strategies require that not only primary roads, but all roads be swept after snow storms to recover applied abrasive material.

Identifying malignant transformations in recurrent low grade gliomas using high resolution magic angle spinning spectroscopy

The objective of this study was to determine whether metabolic parameters derived from ex vivo analysis of tissue samples are predictive of biologic characteristics of recurrent low grade gliomas (LGGs). This was achieved by exploring the use of multivariate pattern recognition methods to generate statistical models of the metabolic characteristics of recurrent LGGs that correlate with aggressive biology and poor clinical outcome. Statistical models were constructed to distinguish between patients with recurrent gliomas that had undergone malignant transformation to a higher grade and those that remained grade 2. The pattern recognition methods explored in this paper include three filter-based feature selection methods (chi-square, gain ratio, and two-way conditional probability), a genetic search wrapper-based feature subset selection algorithm, and five classification algorithms (linear discriminant analysis, logistic regression, functional trees, support vector machines, and decision stump logit boost). The accuracy of each pattern recognition framework was evaluated using leave-one-out cross-validation and bootstrapping. The population studied included fifty-three patients with recurrent grade 2 gliomas. Among these patients, seven had tumors that transformed to grade 4, twenty-four had tumors that transformed to grade 3, and twenty-two had tumors that remained grade 2. Image-guided tissue samples were obtained from these patients using surgical navigation software. Part of each tissue sample was examined by a pathologist for histological features and for consistency with the tumor grade diagnosis. The other part of the tissue sample was analyzed with ex vivo nuclear magnetic resonance (NMR) spectroscopy. Distinguishing between recurrent low grade gliomas that transformed to a higher grade and those that remained grade 2 was achieved with 96% accuracy, using areas of the ex vivo NMR spectrum corresponding to myoinositol, 2-hydroxyglutarate, hypo-taurine, choline, glycerophosphocholine, phosphocholine, glutathione, and lipid. Logistic regression and decision stump boosting models were able to distinguish between recurrent gliomas that transformed to a higher grade and those that did not with 100% training accuracy (95% confidence interval [93-100%]), 96% leave-one-out cross-validation accuracy (95% confidence interval [87-100%]), and 96% bootstrapping accuracy (95% confidence interval [95-97%]). Linear discriminant analysis, functional trees, and support vector machines were able to achieve leave-one-out cross-validation accuracy above 90% and bootstrapping accuracy above 85%. The three feature ranking methods were comparable in performance. This study demonstrates the feasibility of using quantitative pattern recognition methods for the analysis of metabolic data from brain tissue obtained during the surgical resection of gliomas. All pattern recognition techniques provided good diagnostic accuracies, though logistic regression and decision stump boosting slightly outperform the other classifiers. These methods identified biomarkers that can be used to detect malignant transformations in individual low grade gliomas, and can lead to a timely change in treatment for each patient.

Feature Selection Based on Class-Dependent Densities for High-Dimensional Binary Data

Data and knowledge management systems employ feature selection algorithms for removing irrelevant, redundant, and noisy information from the data. There are two well-known approaches to feature selection, feature ranking (FR) and feature subset selection (FSS). In this paper, we propose a new FR algorithm, termed as class-dependent density-based feature elimination (CDFE), for binary data sets. Our theoretical analysis shows that CDFE computes the weights, used for feature ranking, more efficiently as compared to the mutual information measure. Effectively, rankings obtained from both the two criteria approximate each other. CDFE uses a filtrapper approach to select a final subset. For data sets having hundreds of thousands of features, feature selection with FR algorithms is simple and computationally efficient but redundant information may not be removed. On the other hand, FSS algorithms analyze the data for redundancies but may become computationally impractical on high-dimensional data sets. We address these problems by combining FR and FSS methods in the form of a two-stage feature selection algorithm. When introduced as a preprocessing step to the FSS algorithms, CDFE not only presents them with a feature subset, good in terms of classification, but also relieves them from heavy computations. Two FSS algorithms are employed in the second stage to test the two-stage feature selection idea. We carry out experiments with two different classifiers (naive Bayes' and kernel ridge regression) on three different real-life data sets (NOVA, HIVA, and GINA) of the”Agnostic Learning versus Prior Knowledge” challenge. As a stand-alone method, CDFE shows up to about 92 percent reduction in the feature set size. When combined with the FSS algorithms in two-stages, CDFE significantly improves their classification accuracy and exhibits up to 97 percent reduction in the feature set size. We also compared CDFE against the winning entries of the challenge and found that it outperforms the best results on NOVA and HIVA while obtaining a third position in case of GINA.

A hybrid feature subset selection algorithm for analysis of high correlation proteomic data

Pathological changes within an organ can be reflected as proteomic patterns in biological fluids such as plasma, serum, and urine. The surface-enhanced laser desorption and ionization time-of-flight mass spectrometry (SELDI-TOF MS) has been used to generate proteomic profiles from biological fluids. Mass spectrometry yields redundant noisy data that the most data points are irrelevant features for differentiating between cancer and normal cases. In this paper, we have proposed a hybrid feature subset selection algorithm based on maximum-discrimination and minimum-correlation coupled with peak scoring criteria. Our algorithm has been applied to two independent SELDI-TOF MS datasets of ovarian cancer obtained from the NCI-FDA clinical proteomics databank. The proposed algorithm has used to extract a set of proteins as potential biomarkers in each dataset. We applied the linear discriminate analysis to identify the important biomarkers. The selected biomarkers have been able to successfully diagnose the ovarian cancer patients from the noncancer control group with an accuracy of 100%, a sensitivity of 100%, and a specificity of 100% in the two datasets. The hybrid algorithm has the advantage that increases reproducibility of selected biomarkers and able to find a small set of proteins with high discrimination power.

An Incremental Feature Subset Selection Algorithm Based on Boolean Matrix in Decision System

At present, some static feature subset selection algorithms are computationally time-consuming when decision table often change, facing such situations, in fact, the feature subset selection can be computed by utilizing the original decision table. Thus an incremental computation feature subset selection algorithm is provided in this paper, which originated from Boolean matrix technique, to select useful features for the given data objective efficiently. At first, the simplified decision table is introduced. Then using the simplified decision table to design Boolean matrix, the simplified Boolean matrix is constructed. Meanwhile, it is proved that the feature subset selection of the simplified Boolean matrix as the same to that of Boolean matrix. At last, the results on an example show that the efficiency and feasibility of the algorithm.