Feature Subset Selection Algorithm Research Articles

Diagnosis of COPD Based on a Knowledge Graph and Integrated Model

Chronic obstructive pulmonary disease (COPD) is a chronic lung disease that causes a progressive decline in respiratory function. Diagnosing COPD in the early curable stages is very important and may even save the life of a patient. In this paper, we present an integrated model for diagnosing COPD based on a knowledge graph. First, we construct a knowledge graph of COPD to analyze the relationship between feature subsets and further discover the knowledge of implied diseases from the data. Second, we propose an algorithm for sorting features and an adaptive feature subset selection algorithm CMFS- $\eta $ , which selects an optimal subset of features from the original high-dimensional set. Finally, the DSA-SVM integrated model is suggested to build the classifier for the diagnosis and prediction of COPD. We performed extensive experiments on the dataset from the hospital outpatient electronic medical record database. The classification accuracy of our method was 95.1%. It is superior to some state-of-the-art classification methods for this problem.

A Hybridization Approach for Optimal Feature Subset Selection in High Dimensional Data

Feature subset selection assumes an essential part in the fields of data mining and machine learning. A good feature subset selection algorithm can adequately expel unimportant and repetitive elements and consider feature interaction. This not just paves the way to an understanding comprehension of the information, additionally enhances the execution of a learner by improving the generalization capacity and the interpretability of the learning model. Initially, the input micro array dataset is selected from the medical database. Then preprocessing step is done in the input micro array dataset. The resultant output is fed to the second step; here the features are optimally selected using clustering and optimization process. In our proposed technique, the optimal hybrid fuzzy c-means clustering algorithm with artificial bee colony algorithm is applied on the high dimensional micro array dataset to select the important features. Here the proposed method is optimally select the features with the help of social spider optimization algorithm. After that, the classification is done through improved support vector machine classifier. At last, the experimentation is performed by means of different micro array dataset. Experimental results indicate that the proposed classification framework has outperformed by having better accuracy of 93.19% for GLA-BRA-180 dataset when compared existing SVM and neuro fuzzy classifier only achieved 90.69% and 89%.

Comparison of Machine Learning Approaches with a General Linear Model To Predict Personal Exposure to Benzene.

Machine learning techniques (MLTs) offer great power in analyzing complex data sets and have not previously been applied to non-occupational pollutant exposure. MLT models that can predict personal exposure to benzene have been developed and compared with a standard model using a linear regression approach (GLM). The models were tested against independent data sets obtained from three personal exposure measurement campaigns. A correlation-based feature subset (CFS) selection algorithm identified a reduced attribute set, with common attributes grouped under the use of paints in homes, upholstery materials, space heating, and environmental tobacco smoke as the attributes suitable to predict the personal exposure to benzene. Personal exposure was categorized as low, medium, and high, and for big data sets, both the GLM and MLTs show high variability in performance to correctly classify greater than 90 percentile concentrations, but the MLT models have a higher score when accounting for divergence of incorrectly classified cases. Overall, the MLTs perform at least as well as the GLM and avoid the need to input microenvironment concentrations.

Evaluating the influence of parameter values on the performance of random subset feature selection algorithm on scientific data

Random Subset Feature Selection (RSFS) is Feature Subset Selection (FSS) algorithm based on the random forest technique. This algorithm is useful for selecting relevant features from large datasets resulting from scientific experiments. The random selection process eliminates bias and offers superior performance compared to other feature selection algorithms. The performance of the RSFS algorithm, which is primarily used in data mining, depends on proper parameter selection. The RSFS algorithm parameters dummy features, stopping criteria (Delta), maximum number of iterations, and K nearest neighbor distance are used for selecting the feature subset. The resulting subset, which is a reduced dataset is subjected to further processing such as classification and, detection. This study, is based on the design of experiments approach and model the effects of parameter variation on the RSFS algorithm performance. In this study, the influence of algorithm parameters on classification accuracy is evaluated.

Emotional Classification of Acoustic Information With Optimal Feature Subset Selection Methods

This paper mainly focuses on classification of various Acoustic emotional corpora with frequency domain features using feature subset selection methods. The emotional speech samples are classified into neutral, happy, fear , anger, disgust and sad states by using properties of statistics of spectral features estimated from Berlin and Spanish emotional utterances. The Sequential Forward Selection(SFS) and Sequential Floating Forward Selection(SFFS)feature subset selection algorithms are for extracting more informative features. The number of speech emotional samples available for training is smaller than that of the number of features extracted from the speech sample in both Berlin and Spanish corpora which is called curse of dimensionality. Because of this feature vector of high dimensionality the efficiency of the classifier decreases and at the same time the computational time also increases. For additional improvement in the efficiency of the classifier a subset of features which are optimal is needed and is obtained by using feature subset selection methods. This will enhances the performance of the system with high efficiency and lower computation time. The classifier used in this work is the standard K Nearest Neighbour (KNN) Classifier. Experimental evaluation proved that the performance of the classifier is enhanced with SFFS because it vanishes the nesting effect suffered by SFS. The results also showed that an optimal feature subset is a better choice for classification rather than full feature set.

Random subset feature selection for ecological niche models of wildfire activity in Western North America

Variable selection in ecological niche modelling can influence model projections to a degree comparable to variations in future climate scenarios. Consequently, it is important to select feature (variable) subsets for optimizing model performance and characterizing variability. We utilize a novel random subset feature selection algorithm (RSFSA) for niche modelling to select an ensemble of optimally sized feature subsets of limited correlation (|r| < 0.7) from 90 climatic, topographic and anthropogenic indices, generating wildfire activity models for western North America with higher performance. Monitoring Trends in Burn Severity and LANDFIRE wildfire data were used to develop thousands of MaxEnt, GLM and Glmnet models. The RSFSA-selected models performed better than random models, having higher accuracy (Area Under the Curve statistic; AUC), lower complexity (corrected Akaike Information Criterion; AICc), and, in some cases, lower overfitting (AUCdiff). The RSFSA-selected MaxEnt quadratic/hinge (β-regularization 2) feature models generally had higher AUC and lower AICc, outperforming other niche model parameterizations and methods. Feature subset ensembles of RSFSA-selected 15-variable MaxEnt quadratic/hinge models were used to characterize variability in projected areas of large wildfires for three burn severities under current, 2050, and 2070 climate scenarios. Expert screening of variables before RSFSA did not improve model performance. Widespread contemporary wildfire deficits and projected regional changes in wildfires highlight the need to manage fuel loads and restore natural fire regimes. The RSFSA is valuable for optimizing niche model performance and generating feature subset ensembles to characterize model variability across niche models of various feature subset sizes, modelling methods, and climate scenarios.

A novel hybrid algorithm for feature selection

Feature selection is an important filtering method for data analysis, pattern classification, data mining, and so on. Feature selection reduces the number of features by removing irrelevant and redundant data. In this paper, we propose a hybrid filter---wrapper feature subset selection algorithm called the maximum Spearman minimum covariance cuckoo search (MSMCCS). First, based on Spearman and covariance, a filter algorithm is proposed called maximum Spearman minimum covariance (MSMC). Second, three parameters are proposed in MSMC to adjust the weights of the correlation and redundancy, improve the relevance of feature subsets, and reduce the redundancy. Third, in the improved cuckoo search algorithm, a weighted combination strategy is used to select candidate feature subsets, a crossover mutation concept is used to adjust the candidate feature subsets, and finally, the filtered features are selected into optimal feature subsets. Therefore, the MSMCCS combines the efficiency of filters with the greater accuracy of wrappers. Experimental results on eight common data sets from the University of California at Irvine Machine Learning Repository showed that the MSMCCS algorithm had better classification accuracy than the seven wrapper methods, the one filter method, and the two hybrid methods. Furthermore, the proposed algorithm achieved preferable performance on the Wilcoxon signed-rank test and the sensitivity---specificity test.

Classification Performance Improvement Using Random Subset Feature Selection Algorithm for Data Mining

Abstract This study focuses on feature subset selection from high dimensionality databases and presents modification to the existing Random Subset Feature Selection (RSFS) algorithm for the random selection of feature subsets and for improving stability. A standard k-nearest-neighbor (kNN) classifier is used for classification. The RSFS algorithm is used for reducing the dimensionality of a data set by selecting useful novel features. It is based on the random forest algorithm. The current implementation suffers from poor dimensionality reduction and low stability when the database is very large. In this study, an attempt is made to improve the existing algorithm's performance for dimensionality reduction and increase its stability. The proposed algorithm was applied to scientific data to test its performance. With 10 fold cross-validation and modifying the algorithm classification accuracy is improved. The applications of the improved algorithm are presented and discussed in detail. From the results it is concluded that the improved algorithm is superior in reducing the dimensionality and improving the classification accuracy when used with a simple kNN classifier. The data sets are selected from public repository. The datasets are scientific in nature and mostly used in cancer detection. From the results it is concluded that the algorithm is highly recommended for dimensionality reduction while extracting relevant data from scientific datasets.

Fisher score and Matthews correlation coefficient-based feature subset selection for heart disease diagnosis using support vector machines

Heart is one of the essential operating organs of the human body and its failure is a major contributing factor toward the human deaths. Coronary heart disease may be asymptotic but can be anticipated through the medical tests and daily life routine of the subject. Diagnosis of the coronary heart disease needs a specialized medical resource with the plenty of experience. All over the world and particularly in the developing countries, there is a lack of such experts which make the diagnosis more difficult. In this paper, we present a clinical heart disease diagnostic system by proposing feature subset selection methodology with an object of achieving improved performance. The proposed methodology presents three algorithms for selecting candidate feature subsets: (1) mean Fisher score-based feature selection algorithm, (2) forward feature selection algorithm and (3) reverse feature selection algorithm. Feature subset selection algorithm is presented to select the most decisive subset from the candidate feature subsets. The features are added to the feature subsets on the basis of their individual Fisher scores, while the selection of a feature subset depends on its Matthews correlation coefficient score and dimension. The selected feature subset with the reduced dimension is fed to the RBF kernel-based SVM which results in binary classification: (1) heart disease patient and (2) normal control subject. The proposed methodology is validated through accuracy, specificity and sensitivity using four UCI datasets, i.e., Cleveland, Switzerland, Hungarian and SPECTF. The statistical results achieved using the proposed technique are shown in comparison with the existing techniques reflecting its better performance. It has an accuracy of 81.19, 84.52, 92.68 and 82.7% for Cleveland, Hungarian, Switzerland and SPECTF, respectively.

A general framework for boosting feature subset selection algorithms

Feature selection is one of the most important tasks in many machine learning and data mining problems. Due to the increasing size of the problems, removing useless, erroneous or noisy features is frequently an initial step that is performed before other data mining algorithms are applied. The aim is to reproduce, or even improve, the performance of the data mining algorithm when all the features are used. Furthermore, the selection of the most relevant features may offer the expert valuable information about the problem to be solved.Over the past few decades, many different feature selection algorithms have been proposed, each with its own strengths and weaknesses. However, as in the case of classification, it is unlikely that a single feature selection algorithm would be able to achieve good results across many different datasets and application fields. Furthermore, when we are dealing with thousands of features, the most powerful feature selection methods are frequently too time consuming to be applied. In classification, one of the most successful ways of consistently improving the performance of a single weak learner is to construct ensembles using boosting methods. In this paper, we propose a general framework for feature selection boosting in the same way boosting is applied to classification.The proposed approach opens a new field of research in which to apply the many techniques developed for boosting classifiers. Using 120 datasets, the experiments reported show a clear improvement in several state-of-the-art feature selection algorithms using the proposed methodology.

An Optimal Multi-Level Backward Feature Subset Selection for Object Recognition

ABSTRACTFeature subset selection is an active research field in which dimensionality reduction technique is used to select a subset of relevant features. It is a data pre-processing technique used to reduce the number of features in high-dimensional data-sets, crucial in identifying the behaviour and performance of the system. Feature selection finds applications in the areas of image processing, forecasting, document classification, object recognition, anomaly detection, and bioinformatics. The benefits of using feature subset selection include improvements in the data mining algorithm's accuracy, efficiency, and scalability. Feature selection methods are classified into filter and wrapper method, based on the classifier's evaluation strategy. The existing feature subset selection methods are protracted and parameter-dependent, since the user input or threshold value is used to identify the total number of features in the final set. Current feature selection methods result in over estimating feature significance, culminating in the selection of redundant and irrelevant features. To address this issue both theoretically and experimentally, this paper introduced a novel approach to select optimal features for object recognition based on multi-level backward feature subset selection (MLBFSS) algorithm. The proposed method performs better against state-of-the-art methods, verified using benchmark real-world databases. It also outperforms other feature selection methods in terms of classification accuracy and error measures.

A new penalty-based wrapper fitness function for feature subset selection with evolutionary algorithms

ABSTRACTFeature subset selection is an important preprocessing task for any real life data mining or pattern recognition problem. Evolutionary computational (EC) algorithms are popular as a search algorithm for feature subset selection. With the classification accuracy as the fitness function, the EC algorithms end up with feature subsets having considerably high recognition accuracy but the number of residual features also remain quite high. For high dimensional data, reduction of number of features is also very important to minimize computational cost of overall classification process. In this work, a wrapper fitness function composed of classification accuracy with another penalty term which penalizes for large number of features has been proposed. The proposed wrapper fitness function is used for feature subset evaluation and subsequent selection of optimal feature subset with several EC algorithms. The simulation experiments are done with several benchmark data sets having small to large number of features. The simulation results show that the proposed wrapper fitness function is efficient in reducing the number of features in the final selected feature subset without significant reduction of classification accuracy. The proposed fitness function has been shown to perform well for high-dimensional data sets with dimension up to 10,000.

A filter attribute selection method based on local reliable information

In this article, a filter feature weighting technique for attribute selection in classification problems is proposed (LIA). It has two main characteristics. First, unlike feature weighting methods, it is able to consider attribute interactions in the weighting process, rather than only evaluating single features. Attribute subsets are evaluated by projecting instances into a grid defined by attributes in the subset. Then, the joint relevance of the subset is computed by measuring the information present in the cells of the grid. The final weight for each attribute is computed by taking into account its performance in each of the grids it participates. Second, many real problems contain low signal-to-noise ratios, due to instance of high noise levels, class overlap, class imbalance, or small training samples. LIA computes reliable local information for each of the cells by estimating the number of target class instances not due to chance, given a confidence value. In order to study its properties, LIA has been evaluated with a collection of 18 real datasets and compared to two feature weighting methods (Chi-Squared and ReliefF) and a subset feature selection algorithm (CFS). Results show that the method is significantly better in many cases, and never significantly worse. LIA has also been tested with different grid dimensions (1, 2, and 3). The method works best when evaluating attribute subsets larger than 1, hence showing the usefulness of considering attribute interactions.

A novel Hybrid Genetic Local Search Algorithm for feature selection and weighting with an application in strategic decision making in innovation management

In some applications, one needs not only to determine the relevant features but also provide a preferential ordering among the set of relevant features by weights. This paper presents a novel Hybrid Genetic Local Search Algorithm (HGA) in combination with the k-nearest neighbor classifier for simultaneous feature subset selection and feature weighting, particularly for medium-sized data sets. The performance of the proposed algorithm is compared with the performance of alternative feature subset selection algorithms and classifiers through experimental analyses in the various benchmark data sets publicly available on the UCI database. The developed HGA is then applied to a data set gathered from 184 manufacturing firms in the context of innovation management. The data set consists of scores of manufacturing firms in terms of various factors that are known to influence the innovation performance of manufacturing firms and referred to as innovation determinants, and their innovation performances. HGA is used to determine the relative significance of the innovation determinants. Our results demonstrated that the developed HGA is capable of eliminating the irrelevant features and successfully assess feature weights. Moreover, our work is an example how data mining can play a role in the context of strategic management decision making.

A Novel Hybrid Genetic Local Search Algorithm for Feature Selection and Weighting with an Application in Strategic Decision Making in Innovation Management

In some applications, one needs not only to determine the relevant features but also provide a preferential ordering among the set of relevant features by weights. This paper presents a novel Hybrid Genetic Local Search Algorithm (HGA) in combination with the k-nearest neighbor classifier for simultaneous feature subset selection and feature weighting, particularly for medium-sized data sets. The performance of the proposed algorithm is compared with the performance of alternative feature subset selection algorithms and classifiers through experimental analyses in the various benchmark data sets publicly avail- able on the UCI database. The developed HGA is then applied to a data set gathered from 184 manufacturing firms in the context of innovation manage- ment. The data set consists of scores of manufacturing firms in terms of various factors that are known to influence the innovation performance of manufacturing firms and referred to as innovation determinants, and their innovation perfor- mances. HGA is used to determine the relative significance of the innovation determinants. Our results demonstrated that the developed HGA is capable of eliminating the irrelevant features and successfully assess feature weights. More- over, our work is an example how data mining can play a role in the context of strategic management decision making.

Robust multiobjective evolutionary feature subset selection algorithm for binary classification using machine learning techniques

This study investigates the success of a multiobjective genetic algorithm (GA) combined with state-of-the-art machine learning (ML) techniques for the feature subset selection (FSS) in binary classification problem (BCP). Recent studies have focused on improving the accuracy of BCP by including all of the features, neglecting to determine the best performing subset of features. However, for some problems, the number of features may reach thousands, which will cause too much computation power to be consumed during the feature evaluation and classification phases, also possibly reducing the accuracy of the results. Therefore, selecting the minimum number of features while preserving and/or increasing the accuracy of the results at a high level becomes an important issue for achieving fast and accurate binary classification. Our multiobjective evolutionary algorithm includes two phases, FSS using a GA and applying ML techniques for the BCP. Since exhaustively investigating all of the feature subsets is intractable, a GA is preferred for the first phase of the algorithm for intelligently detecting the most appropriate feature subset. The GA uses multiobjective crossover and mutation operators to improve a population of individuals (each representing a selected feature subset) and obtain (near-) optimal solutions through generations. In the second phase of the algorithms, the fitness of the selected subset is decided by using state-of-the-art ML techniques; Logistic Regression, Support Vector Machines, Extreme Learning Machine, K-means, and Affinity Propagation. The performance of the multiobjective evolutionary algorithm (and the ML techniques) is evaluated with comprehensive experiments and compared with state-of-the-art algorithms, Greedy Search, Particle Swarm Optimization, Tabu Search, and Scatter Search. The proposed algorithm was observed to be robust and it performed better than the existing methods on most of the datasets.

A feature cluster taxonomy based feature selection technique

Feature subset selection is basically an optimization problem for choosing the most important features from various alternatives in order to facilitate classification or mining problems. Though lots of algorithms have been developed so far, none is considered to be the best for all situations and researchers are still trying to come up with better solutions. In this work, a flexible and user-guided feature subset selection algorithm, named as FCTFS (Feature Cluster Taxonomy based Feature Selection) has been proposed for selecting suitable feature subset from a large feature set. The proposed algorithm falls under the genre of clustering based feature selection techniques in which features are initially clustered according to their intrinsic characteristics following the filter approach. In the second step the most suitable feature is selected from each cluster to form the final subset following a wrapper approach. The two stage hybrid process lowers the computational cost of subset selection, especially for large feature data sets. One of the main novelty of the proposed approach lies in the process of determining optimal number of feature clusters. Unlike currently available methods, which mostly employ a trial and error approach, the proposed method characterises and quantifies the feature clusters according to the quality of the features inside the clusters and defines a taxonomy of the feature clusters. The selection of individual features from a feature cluster can be done judiciously considering both the relevancy and redundancy according to user’s intention and requirement. The algorithm has been verified by simulation experiments with different bench mark data set containing features ranging from 10 to more than 800 and compared with other currently used feature selection algorithms. The simulation results prove the superiority of our proposal in terms of model performance, flexibility of use in practical problems and extendibility to large feature sets. Though the current proposal is verified in the domain of unsupervised classification, it can be easily used in case of supervised classification.

An empirical study on attribute selection of student performance prediction model

Despite improvement in the standard of education globally, students' failure rates have risen. Data mining has been implemented in several domains, including education, for extracting valuable information from raw data. The aim of this study was to develop a model for predicting student performance and thereby identifying the students who might under perform in examinations. Student data used for the study consisted of demographic and academic information of students. Systematic analysis of different attributes of the student data was done using feature subset selection algorithms. The model was tested using classification algorithms. Based on these results a small attribute set, namely student data feature set (SDFS) was proposed. The experimental results demonstrate that the learning model using SDFS gives the best results and also minimises the errors. This model can be utilised to identify the academically weaker students so that appropriate preventive action can be taken to avoid failures. Adoption of data analytics in education can help create a smart education system beneficial for society.

Mixed-variable ant colony optimisation algorithm for feature subset selection and tuning support vector machine parameter

This paper presents a hybrid classification algorithm, ACOMV-SVM which is based on ant colony and support vector machine. A new direction for ant colony optimisation is to optimise mixed discrete and continuous variables. The optimised variables are then feed into selecting feature subset and tuning its parameters are two main problems of SVM. Most approaches related to tuning support vector machine parameters will discretise the continuous value of the parameters which will give a negative effect on the performance. The objective of this paper is to formulate an algorithm for tuning SVM parameters and feature subset selection. This can be achieved by simultaneously performing the selection of feature subset and tuning SVM parameters tasks. ACO

Mixed-variable ant colony optimisation algorithm for feature subset selection and tuning support vector machine parameter

This paper presents a hybrid classification algorithm, ACOMV-SVM which is based on ant colony and support vector machine. A new direction for ant colony optimisation is to optimise mixed (discrete and continuous) variables. The optimised variables are then feed into selecting feature subset and tuning its parameters are two main problems of SVM. Most approaches related to tuning support vector machine parameters will discretise the continuous value of the parameters which will give a negative effect on the performance. The objective of this paper is to formulate an algorithm for tuning SVM parameters and feature subset selection. This can be achieved by simultaneously performing the selection of feature subset and tuning SVM parameters tasks. ACOMV-SVM algorithm is able to simultaneously tune SVM parameters and feature subset selection. Experimental results obtained from the proposed algorithm are better compared with other approaches in terms of classification accuracy and feature subset selection. The work in this paper also contributes to a new direction for ACO that can deal with mixed variable ACO.