Subset Selection Algorithm Research Articles

SVM and KNN Based SGO Feature Selection Algorithm for Breast Cancer Diagnosis

In diagnosis and prediction systems, algorithms working on datasets with a high number of dimensions tend to take more time than those with fewer dimensions. Feature subset selection algorithms enhance the efficiency of Machine Learning algorithms in prediction problems by selecting a subset of the total features and thus pruning redundancy and noise. In this article, such a feature subset selection method is proposed and implemented to diagnose breast cancer using Support Vector Machine (SVM) and K-Nearest Neighbor (KNN) algorithms. This feature selection algorithm is based on Social Group Optimization (SGO) an evolutionary algorithm. Higher accuracy in diagnosing breast cancer is achieved using our proposed model when compared to other feature selection-based Machine Learning algorithms.

A permutational-based Differential Evolution algorithm for feature subset selection

This paper describes a permutational-based Differential Evolution algorithm implemented in a wrapper scheme to find a feature subset to be applied in the construction of a near-optimal classifier. In this approach, the relevance of a feature chosen to build a better classifier is represented through its relative position in an integer-valued vector, and by using a permutational-based mutation operator, it is possible to create new feasible candidate solutions only. Furthermore, to provide a controlled diversity rate in the population, a straightforward repair-based recombination operator is utilized to evolve a population of candidate solutions. Unlike the other approaches in the existing literature using integer-valued vectors and requiring a predefined subset size, in this approach, this size is determined by an additional element included in the encoding scheme, allowing to find an adequate feature subset size to each specific dataset. Experimental results show that this approach is an effective way to create more accurate classifiers as they are compared with those obtained by other similar approaches.

Principal Component Clustered Factors for Determining Study Performance in Computer Programming Class

Studying computer programming requires not only an understanding of theories and concepts, but also coding pragmatism. Success in studying or conducting such a course is definitely a challenge. This paper proposes a model that transforms students’ attributes (including the cognitive and non-cognitive abilities, and traditional lagging measures of academic background) into a set of principal components (PCs). As opposed to traditional approaches, the proposed model optimally extracts the orthogonal PCs to form a basis for determining the studying performance of students in terms of declarative knowledge and procedural proficiency (or skill). The obtained relationship model yields two contributive values (1) an optimal set of determinants, in the form of students’ clusters, to determine study performance and (2) the fully preserved interpretability of the original attributes of students in each PC. The experiment was conducted using 115 complete datasets of IT major students who enrolled the Introduction to Computer Programming course. The Best Subset Selection and LASSO algorithms were deployed to find the optimal set of features. The effectiveness of the model was validated by multiple linear regression to predict the performance in terms of knowledge and skills with an accuracy of 76.52%, and 70.44%, respectively. Insights into the interpretability of student clusters are provided.

An Implementation of Novel Feature Subset Selection Algorithm for IDS in Mobile Networks

An Implementation of Novel Feature Subset Selection Algorithm for IDS in Mobile Networks

Feature subset selection combining maximal information entropy and maximal information coefficient

Feature subset selection is an efficient step to reduce the dimension of data, which remains an active research field in decades. In order to develop highly accurate and fast searching feature subset selection algorithms, a filter feature subset selection method combining maximal information entropy (MIE) and the maximal information coefficient (MIC) is proposed in this paper. First, a new metric mMIE-mMIC is defined to minimize the MIE among features while maximizing the MIC between the features and the class label. The mMIE-mMIC algorithm is designed to evaluate whether a candidate subset is valid for classification. Second, two searching strategies are adopted to identify a suitable solution in the candidate subset space, including the binary particle swarm optimization algorithm (BPSO) and sequential forward selection (SFS). Finally, classification is performed on UCI datasets to validate the performance of our work compared to 9 existing methods. Experimental results show that in most cases, the proposed method behaves equally or better than the other 9 methods in terms of classification accuracy and F1-score.

Evolutionary feature subsets selection based on interaction information for high dimensional imbalanced data classification

With the advent of technology in various scientific fields, high dimensional data are becoming abundant. A general approach to tackle the resulting challenges is to reduce data dimensionality through feature selection. Traditional feature selection approaches concentrate on selecting relevant features and ignoring irrelevant or redundant ones. However, most of these approaches neglect feature interactions. On the other hand, some datasets have imbalanced classes, which may result in biases towards the majority class. The main goal of this paper is to propose a novel feature selection method based on the interaction information (II) to provide higher level interaction analysis and improve the search procedure in the feature space. In this regard, an evolutionary feature subset selection algorithm based on interaction information is proposed, which consists of three stages. At the first stage, candidate features and candidate feature pairs are identified using traditional feature weighting approaches such as symmetric uncertainty (SU) and bivariate interaction information. In the second phase, candidate feature subsets are formed and evaluated using multivariate interaction information. Finally, the best candidate feature subsets are selected using dominant/dominated relationships. The proposed algorithm is compared with some other feature selection algorithms including mRMR, WJMI, IWFS, IGFS, DCSF, IWFS, K_OFSD, WFLNS, Information Gain and ReliefF in terms of the number of selected features, classification accuracy, F-measure and algorithm stability using three different classifiers, namely KNN, NB, and CART. The results justify the improvement of classification accuracy and the robustness of the proposed method in comparison with the other approaches.

Gene Subset Selection for Leukemia Classification Using Microarray Data

Background: Cancer subtype identification is an active research field which helps in the diagnosis of various cancers with proper treatments. Leukemia is one such cancer with various subtypes. High throughput technologies such as Deoxyribo Nucleic Acid (DNA) microarray are highly active in the field of cancer detection and classification alternatively. Objective: Yet, a precise analysis is important in microarray data applications as microarray experiments provide huge amount of data. Gene selection techniques promote microarray usage in the field of medicine. The objective of gene selection is to select a small subset of genes, which are the most informative in classification. associations while known disease-lncRNA associations are required only. Method: In this study, multi-objective evolutionary algorithm is used for gene subset selection in Leukemia classification. An initial redundant and irrelevant gene removal is followed by multiobjective evolutionary based gene subset selection. Gene subset selection highly influences the perfect classification. Thus, selecting the appropriate algorithm for subset selection is important. Results: The performance of the proposed method is compared against the standard genetic algorithm and evolutionary algorithm. Three Leukemia microarray datasets were used to evaluate the performance of the proposed method. Perfect classification was achieved for all the datasets only with few significant genes using the proposed approach. Conclusion: Thus, it is obvious that the proposed study perfectly classifies Leukemia with only few significant genes.</P>

A hybrid gene selection algorithm based on interaction information for microarray-based cancer classification.

We address gene selection and machine learning methods for cancer classification using microarray gene expression data. Due to the high dimensionality of microarray data, traditional gene selection algorithms are filter-based, focusing on intrinsic properties of the data such as distance, dependency, and correlation. These methods are fast but select far too many genes to use for the classification task. In this work, we present a new hybrid filter-wrapper gene subset selection algorithm that is an improved modification of our prior algorithm. Our proposed method employs interaction information to rank candidate genes to add into a gene subset. It then conditionally adds one gene at a time into the current subset and verifies whether the resultant subset improves the classification performance significantly. Only significant genes are selected, and the candidate gene list is updated every time a gene is added to the subset. Thus, our gene selection algorithm is very dynamic. Experimental results on ten public cancer microarray data sets show that our method consistently outperforms prior gene selection algorithms in terms of classification accuracy, while requiring a small number of selected genes.

Swarm intelligence based optimal feature selection for enhanced predictive sentiment accuracy on twitter

A lot of uncertainty is generally associated with the micro-blog content, primarily due to the presence of noisy, heterogeneous, structured or unstructured data which may be high-dimensional, ambiguous, vague or imprecise. This makes feature engineering for predicting the sentiment arduous and challenging. Population-based meta-heuristics, especially the ones inspired by nature have been proposed in various pertinent studies for feature selection because of their probability to accept a less optimal solution and averting being stuck in local optimal solutions. This research demonstrates the use of two such swarm intelligence algorithms, namely, binary grey wolf and binary moth flame for feature optimization to enhance the sentiment classification performance accuracy. The study is conducted on tweets from two benchmark Twitter corpus (SemEval 2016 and SemEval 2017) and is initially analyzed using the conventional term frequency-inverse document frequency statistical weighting filter for feature extraction and subsequently using the swarm-based algorithms. The features are trained over five baseline classifiers namely, the Naive Bayesian, support vector machines, k-nearest neighbor, multilayer perceptron and decision tree. The results validate that the population-based meta-heuristic algorithms for feature subset selection outperform the baseline supervised learning algorithms. For the binary grey wolf algorithm, an average improvement of 9.4% in accuracy is observed with an approximate 20.5% average reduction in features. Also, for the binary moth flame algorithm, an average accuracy improvement of 10.6% is observed with an approximate 40% average reduction in features. The highest accuracy of 76.5% is observed for support vector machine with binary grey wolf optimizer on SemEval 2016 benchmark dataset.

Sensor Placement Design Strategy and Quality Estimation in Modern Car Body Production Using Stochastic Finite Element Methods

Abstract For automotive body panel’s producers, any failures of tooling or formed components pose significant risk in the forms of costly repairs and production delays. Accordingly, methods of reducing the risk of tooling damage and part failure through predictive measures are of great value. Such predictive methods are particularly beneficial in optimizing forming operations when process parameters such as friction and material behavior are uncertain or variable. AUDI has invented the Intelligent Tool which uses sensor measurements to observe the quality of the part and adjusts actuators to eliminate differences between desired quality and actual quality. A key question in planning such intelligent tool systems is the appropriate number and positions of sensors to maximize information content. A stochastic finite element method-based approach has been developed in the scope of this work to critically assess the observability of split and wrinkling type failures, based on laser sensor draw-in measurements. Furthermore, a design strategy for the number of sensors and the respective positions around the blank is proposed which enables an accurate yet cost-effective acquisition of process data. The methodology is applied to the tailgate of an AUDI Model A4. Stochastic finite element simulations are computed in AutoForm. The quality of the part under varying process parameters is evaluated and critical zones are identified. A principal component analysis is utilized to reveal that correlations exist between the quality criteria; therefore, only principal quality criteria need to be observed in order to estimate global part quality. Regression models are trained to connect quality criteria to flange draw-in measurements and a subset selection algorithm is used to find the optimal sensor layout which delivers the highest information content.

A Novel Hybrid Algorithm for Feature Selection Based on Whale Optimization Algorithm

Feature selection enhances classification accuracy by removing irrelevant and redundant feature. Feature selection plays an important role in data mining and pattern recognition. In this paper, we propose a hybrid feature subset selection algorithm called the maximum Pearson maximum distance improved whale optimization algorithm (MPMDIWOA). First, based on Pearson's correlation coefficient and correlation distance, a filter algorithm is proposed named maximum Pearson maximum distance (MPMD). Two parameters are proposed in MPMD to adjust the weights of the relevance and redundancy. Second, the modified whale optimization algorithm can act as a wrapper algorithm. After introducing the maximum value without change (MVWC) and the threshold, the filter algorithm and the wrapper algorithm are combined to form an algorithm called MPMDIWOA. In MPMDIWOA, the filter algorithm and wrapper algorithm are called different times according to changes in MVWC and threshold. Finally, the optimal classification accuracy was found. The proposed method is tested on 10 benchmark datasets from UCI machine learning databases. The experimental results show that the classification accuracy of the proposed algorithm is significantly higher than that of the other three wrapper algorithms and one hybrid algorithm.

Diagnosis of COPD Based on a Knowledge Graph and Integrated Model

Chronic obstructive pulmonary disease (COPD) is a chronic lung disease that causes a progressive decline in respiratory function. Diagnosing COPD in the early curable stages is very important and may even save the life of a patient. In this paper, we present an integrated model for diagnosing COPD based on a knowledge graph. First, we construct a knowledge graph of COPD to analyze the relationship between feature subsets and further discover the knowledge of implied diseases from the data. Second, we propose an algorithm for sorting features and an adaptive feature subset selection algorithm CMFS- $\eta $ , which selects an optimal subset of features from the original high-dimensional set. Finally, the DSA-SVM integrated model is suggested to build the classifier for the diagnosis and prediction of COPD. We performed extensive experiments on the dataset from the hospital outpatient electronic medical record database. The classification accuracy of our method was 95.1%. It is superior to some state-of-the-art classification methods for this problem.

A Hybridization Approach for Optimal Feature Subset Selection in High Dimensional Data

Feature subset selection assumes an essential part in the fields of data mining and machine learning. A good feature subset selection algorithm can adequately expel unimportant and repetitive elements and consider feature interaction. This not just paves the way to an understanding comprehension of the information, additionally enhances the execution of a learner by improving the generalization capacity and the interpretability of the learning model. Initially, the input micro array dataset is selected from the medical database. Then preprocessing step is done in the input micro array dataset. The resultant output is fed to the second step; here the features are optimally selected using clustering and optimization process. In our proposed technique, the optimal hybrid fuzzy c-means clustering algorithm with artificial bee colony algorithm is applied on the high dimensional micro array dataset to select the important features. Here the proposed method is optimally select the features with the help of social spider optimization algorithm. After that, the classification is done through improved support vector machine classifier. At last, the experimentation is performed by means of different micro array dataset. Experimental results indicate that the proposed classification framework has outperformed by having better accuracy of 93.19% for GLA-BRA-180 dataset when compared existing SVM and neuro fuzzy classifier only achieved 90.69% and 89%.

Speech corpora subset selection based on time-continuous utterances features

An extremely large corpus with rich acoustic properties is very useful for training new speech recognition and semantic analysis models. However, it also brings some troubles, because the complexity of the acoustic model training usually depends on the size of the corpora. In this paper, we propose a corpora subset selection method considering data contributions from time-continuous utterances and multi-label constraints that are not limited to single-scale metrics. Our goal is to extract a sufficiently rich subset from large corpora under certain meaningful constraints. In addition, taking into account the uniform coverage of the target subset and its internal property, we design a constrained subset selection algorithm. Specifically, a fast subset selection algorithm is designed by introducing n-grams models. Experiments are implemented based on very large real speech corpora database and validate the effectiveness of our method.

Comparison of Machine Learning Approaches with a General Linear Model To Predict Personal Exposure to Benzene.

Machine learning techniques (MLTs) offer great power in analyzing complex data sets and have not previously been applied to non-occupational pollutant exposure. MLT models that can predict personal exposure to benzene have been developed and compared with a standard model using a linear regression approach (GLM). The models were tested against independent data sets obtained from three personal exposure measurement campaigns. A correlation-based feature subset (CFS) selection algorithm identified a reduced attribute set, with common attributes grouped under the use of paints in homes, upholstery materials, space heating, and environmental tobacco smoke as the attributes suitable to predict the personal exposure to benzene. Personal exposure was categorized as low, medium, and high, and for big data sets, both the GLM and MLTs show high variability in performance to correctly classify greater than 90 percentile concentrations, but the MLT models have a higher score when accounting for divergence of incorrectly classified cases. Overall, the MLTs perform at least as well as the GLM and avoid the need to input microenvironment concentrations.

Evaluating the influence of parameter values on the performance of random subset feature selection algorithm on scientific data

Random Subset Feature Selection (RSFS) is Feature Subset Selection (FSS) algorithm based on the random forest technique. This algorithm is useful for selecting relevant features from large datasets resulting from scientific experiments. The random selection process eliminates bias and offers superior performance compared to other feature selection algorithms. The performance of the RSFS algorithm, which is primarily used in data mining, depends on proper parameter selection. The RSFS algorithm parameters dummy features, stopping criteria (Delta), maximum number of iterations, and K nearest neighbor distance are used for selecting the feature subset. The resulting subset, which is a reduced dataset is subjected to further processing such as classification and, detection. This study, is based on the design of experiments approach and model the effects of parameter variation on the RSFS algorithm performance. In this study, the influence of algorithm parameters on classification accuracy is evaluated.

F-WSS $$^{++}$$ + + : incremental wrapper subset selection algorithm for fuzzy extreme learning machine

Fuzzy extreme learning machine (F-ELM) is a hybrid combination made to get the benefits of fuzzy system and extreme learning machine (ELM). F-ELM randomly initializes the weights between input layer to the hidden layer and analytically tunes the weights between hidden layer to the output layer. Due to this random initialization and the availability of redundant and irrelevant features, F-ELM may degrade the overall (generalization) performance. To solve the mentioned problem in this paper, an advanced classification algorithm $$\hbox {F-WSS}^{++}$$ (incremental wrapper subset selection algorithm for F-ELM) is designed for multiclass and binary class classification problems. The merits of the proposed algorithm are analyzed theoretically and experimentally. The test results are cross checked for $$\hbox {F-WSS}^{++}$$ and $$\hbox {E-WSS}^{++}$$ (incremental wrapper subset selection algorithm for ELM) for the clinical dataset. The effectiveness of the $$\hbox {F-WSS}^{++}$$ algorithm is verified by statistical methods. It is observed that $$\hbox {F-WSS}^{++}$$ has the capability to handle weighted classification problem, feature subset selection problem and optimization problem. It also improves 9–10% classification accuracy by using only 50% features.

Analysis of a multi-axial quantum-informed ferroelectric continuum model: Part 2—sensitivity analysis

We illustrate the use of global sensitivity analysis, and a parameter subset selection algorithm based on local sensitivity analysis, to quantify the relative influence of parameters in polarization and electrostrictive energy relations for a quantum-informed, single-domain, ferroelectric material model. A motivating objective is to determine which parameters are identifiable or influential in the sense that they are uniquely determined by density functional theory–generated data. Noninfluential parameters will be fixed at nominal values for subsequent Bayesian inference, uncertainty propagation, and material design since variations in these parameters are minimally reflected in responses. Whereas global sensitivity analysis is typically based on the assumption of mutually independent, uniformly distributed parameters, we demonstrate that inherent parameter correlations must be accommodated to achieve correct interpretations of parameter influence. For the considered energy functionals, we demonstrate that all of the parameters are influential and will be informed by density functional theory–simulated data.

Emotional Classification of Acoustic Information With Optimal Feature Subset Selection Methods

This paper mainly focuses on classification of various Acoustic emotional corpora with frequency domain features using feature subset selection methods. The emotional speech samples are classified into neutral, happy, fear , anger, disgust and sad states by using properties of statistics of spectral features estimated from Berlin and Spanish emotional utterances. The Sequential Forward Selection(SFS) and Sequential Floating Forward Selection(SFFS)feature subset selection algorithms are for extracting more informative features. The number of speech emotional samples available for training is smaller than that of the number of features extracted from the speech sample in both Berlin and Spanish corpora which is called curse of dimensionality. Because of this feature vector of high dimensionality the efficiency of the classifier decreases and at the same time the computational time also increases. For additional improvement in the efficiency of the classifier a subset of features which are optimal is needed and is obtained by using feature subset selection methods. This will enhances the performance of the system with high efficiency and lower computation time. The classifier used in this work is the standard K Nearest Neighbour (KNN) Classifier. Experimental evaluation proved that the performance of the classifier is enhanced with SFFS because it vanishes the nesting effect suffered by SFS. The results also showed that an optimal feature subset is a better choice for classification rather than full feature set.

A novel hybrid algorithm for feature selection

Feature selection is an important filtering method for data analysis, pattern classification, data mining, and so on. Feature selection reduces the number of features by removing irrelevant and redundant data. In this paper, we propose a hybrid filter---wrapper feature subset selection algorithm called the maximum Spearman minimum covariance cuckoo search (MSMCCS). First, based on Spearman and covariance, a filter algorithm is proposed called maximum Spearman minimum covariance (MSMC). Second, three parameters are proposed in MSMC to adjust the weights of the correlation and redundancy, improve the relevance of feature subsets, and reduce the redundancy. Third, in the improved cuckoo search algorithm, a weighted combination strategy is used to select candidate feature subsets, a crossover mutation concept is used to adjust the candidate feature subsets, and finally, the filtered features are selected into optimal feature subsets. Therefore, the MSMCCS combines the efficiency of filters with the greater accuracy of wrappers. Experimental results on eight common data sets from the University of California at Irvine Machine Learning Repository showed that the MSMCCS algorithm had better classification accuracy than the seven wrapper methods, the one filter method, and the two hybrid methods. Furthermore, the proposed algorithm achieved preferable performance on the Wilcoxon signed-rank test and the sensitivity---specificity test.