Hybrid Feature Subset Selection Research Articles

Hybrid Feature Selection based on BTLBO and RNCA to Diagnose the Breast Cancer

Feature selection is a feasible solution to improve the speed and performance of machine learning models. Optimization algorithms are doing a significant job in searching for optimal variables from feature space. Recent feature selection methods are purely depending on various meta heuristic algorithms for searching a good combination of features without considering the importance of individual features, which makes classification models to suffer from local optima or overfitting problems. In this paper, a novel hybrid feature subset selection technique is introduced based on Regularized Neighborhood Component Analysis (RNCA) and Binary Teaching Learning Based Optimization (BTLBO) algorithms to overcome the above problems. RNCA algorithm assigns weights to the attributes based on their contribution in building the learning models for classification. BTLBO algorithm computes the fitness of individuals with respect to the weights of features and selects the best ones. The results of similar feature selection methods are matched with the proposed hybrid model and proved better performance in terms of classification accuracy, recall and AUC measures over breast cancer datasets.

Effective hybrid feature subset selection for multilevel datasets using decision tree classifiers

Feature selection is one of the most significant procedures in machine learning algorithms. It is particularly to improve the performance and prediction accuracy for complex data classification. This paper discusses a hybrid feature selection technique with the decision tree-based classification algorithm. The feature selected using information gain (IG) is combined with the features selected from ReliefF which generates the resultant feature subset. Then the resultant feature subset is in turn combined with a correlation-based feature selection (CFS) method to generate the aggregated feature subset. To perform classification accuracy on the aggregated feature subset, different decision trees-based classification algorithm such as C4.5, decision stumps, naive Bayes tree, and random forest with ten-fold cross-validation have been deployed. To check the prediction accuracy of the proposed work eight different multilevel University of California, Irvine (UCI) machine learning datasets have been used with minimum to maximum numbers of features. The main objective of the hybrid feature selection is to improve the classification accuracy, prediction and to reduce the execution time using standard datasets.

Effective hybrid feature subset selection for multilevel datasets using decision tree classifiers

Effective hybrid feature subset selection for multilevel datasets using decision tree classifiers

An intelligent hybrid feature subset selection and production pattern recognition method for modeling ethylene plant

A data-driven model framework integrating Feature Subset Selection (FSS), production pattern clustering analysis and prediction was proposed for predicting ethylene yield of ethylene plant by using the massive sensing data recorded by the Distributed Control System (DCS) of petrochemical enterprises. Firstly, an Ensemble-Filter FSS model based on three different metrics is designed to initially filter all the steam cracking furnace features, and then a Wrapper FSS model based on GA-SVR is used to obtain the optimal subset of features affecting ethylene yield. The steam cracking furnace was identified based on the Density Peak Clustering (DPC) algorithm based on the production patterns embedded in the data. Ethylene yield prediction models were separately developed for each production pattern to summarize the final prediction results. The proposed model was validated against historical data from an industrial steam cracking furnace in northwest China. Results show that the number of features have a 93.4% reduction in the FSS stage. and a 40.6% reduction in predicted MSE. Compared with the benchmark ANN model, the proposed DPNN model has a 56.6% reduction in MSE based on the optimal cluster result. What’s more, the proposed framework has a strong generalization ability and with a modular structure which is easy to modify., which is expected to be used to guide the ethylene plant operating in reasonable intervals.

An efficient feature subset selection approach for machine learning

This literature introduces CAPPER, an efficient hybrid feature subset selection approach which is the combination of feature subsets from the Correlation-based feature selection (CFS) and Wrapper approaches. CFS is basically a filter approach that appraises the merits of subset attributes by classifying the feature ability according to the amount of redundancy between them and the feature subset selection for Wrappers that examines the attributes by applying the induction of various machine learning algorithms. For the evaluation of metrics, the CAPPER approach is tested on the different domains of datasets. The reduced, highly merit and accurate feature subsets obtained from CAPPER approach are then trained with the machine learning algorithm and evaluated by cross-validation for the set of attributes. Moreover, a statistical approach is applied for the significance of the result. It was observed that the CAPPER approach surpasses the CFS and Wrapper approaches on different domains of datasets.

Hybrid approaches to feature subset selection for data classification in high-dimensional feature space

This paper proposes two hybrid feature subset selection approaches based on the combination (union or intersection) of both supervised and unsupervised filter approaches before using a wrapper, aiming to obtain low-dimensional features with high accuracy and interpretability and low time consumption. Experiments with the proposed hybrid approaches have been conducted on seven high-dimensional feature datasets. The classifiers adopted are support vector machine (SVM), linear discriminant analysis (LDA), and K-nearest neighbour (KNN). Experimental results have demonstrated the advantages and usefulness of the proposed methods in feature subset selection in high-dimensional space in terms of the number of selected features and time spent to achieve the best classification accuracy.

A New Hybrid Feature Subset Selection Framework Based on Binary Genetic Algorithm and Information Theory

The explosion of the high-dimensional dataset in the scientific repository has been encouraging interdisciplinary research on data mining, pattern recognition and bioinformatics. The fundamental problem of the individual Feature Selection (FS) method is extracting informative features for classification model and to seek for the malignant disease at low computational cost. In addition, existing FS approaches overlook the fact that for a given cardinality, there can be several subsets with similar information. This paper introduces a novel hybrid FS algorithm, called Filter-Wrapper Feature Selection (FWFS) for a classification problem and also addresses the limitations of existing methods. In the proposed model, the front-end filter ranking method as Conditional Mutual Information Maximization (CMIM) selects the high ranked feature subset while the succeeding method as Binary Genetic Algorithm (BGA) accelerates the search in identifying the significant feature subsets. One of the merits of the proposed method is that, unlike an exhaustive method, it speeds up the FS procedure without lancing of classification accuracy on reduced dataset when a learning model is applied to the selected subsets of features. The efficacy of the proposed (FWFS) method is examined by Naive Bayes (NB) classifier which works as a fitness function. The effectiveness of the selected feature subset is evaluated using numerous classifiers on five biological datasets and five UCI datasets of a varied dimensionality and number of instances. The experimental results emphasize that the proposed method provides additional support to the significant reduction of the features and outperforms the existing methods. For microarray data-sets, we found the lowest classification accuracy is 61.24% on SRBCT dataset and highest accuracy is 99.32% on Diffuse large B-cell lymphoma (DLBCL). In UCI datasets, the lowest classification accuracy is 40.04% on the Lymphography using k-nearest neighbor (k-NN) and highest classification accuracy is 99.05% on the ionosphere using support vector machine (SVM).

A Novel Hybrid Algorithm for Feature Selection Based on Whale Optimization Algorithm

Feature selection enhances classification accuracy by removing irrelevant and redundant feature. Feature selection plays an important role in data mining and pattern recognition. In this paper, we propose a hybrid feature subset selection algorithm called the maximum Pearson maximum distance improved whale optimization algorithm (MPMDIWOA). First, based on Pearson's correlation coefficient and correlation distance, a filter algorithm is proposed named maximum Pearson maximum distance (MPMD). Two parameters are proposed in MPMD to adjust the weights of the relevance and redundancy. Second, the modified whale optimization algorithm can act as a wrapper algorithm. After introducing the maximum value without change (MVWC) and the threshold, the filter algorithm and the wrapper algorithm are combined to form an algorithm called MPMDIWOA. In MPMDIWOA, the filter algorithm and wrapper algorithm are called different times according to changes in MVWC and threshold. Finally, the optimal classification accuracy was found. The proposed method is tested on 10 benchmark datasets from UCI machine learning databases. The experimental results show that the classification accuracy of the proposed algorithm is significantly higher than that of the other three wrapper algorithms and one hybrid algorithm.

Hybrid Feature Subset Selection Approach for Fuzzy-Extreme Learning Machine

Extreme learning machine (ELM) is a fast learning algorithm which has various advancements in terms of speed, reliability and generalized performance. Perhaps, it lacks the ability to solve the weighted classification problem. One of the alternative solutions to this problem is fuzzy-ELM (F-ELM) which combines the advantages of a fuzzy system and ELM. The major aim of this research is to handle the feature subset selection (FSS) and weighted classification problem. In this paper, a hybrid FSS approach for F-ELM classifier (H-FELM) is designed for binary and multiclass classification on the clinical dataset. Experimental results illustrate that H-FELM has the capability to handle these problems with improved classification accuracy by selecting prominent features which are relevant and non-redundant. The selection of such features can reduce the computational overhead and hence minimize the learning time. In order to validate the efficiency and effectiveness of the proposed algorithm, the comparative performance is evaluated by using four different strategies—1. existing F-ELM, 2. existing F-ELM with FSS, 3. ELM and F-ELM and 4. ELM with FSS and F-ELM with FSS. It is observed that the designed algorithm achieves an improvement of 9–10% in accuracy over existing results with almost reduction of 50% features for the clinical dataset.

Network Traffic Feature Selection Based on the Evaluation of Every Attribute

Traffic classification is making a significant difference in network resource scheduling, safety analysis and future tendency prediction. As the essential step for machine learning based traffic classification, feature subset selection is often used to realize dimension reduction and redundant information decrease. A three-stage hybrid feature subset selection method is proposed to improve the classification performance of hybrid methods at low evaluation consumption. The proposed algorithm disposes features by the rank in the level of block and also evaluates all the remaining features that are verified as useless to take advantage of the interactions among all the features. Our theoretical analysis and experimental observations reveal that the proposed method selects feature subset with impressive classification performance on every index while depleting fewer evaluations

Using a Feature Subset Selection method and Support Vector Machine to address curse of dimensionality and redundancy in Hyperion hyperspectral data classification

The curse of dimensionality resulted from insufficient training samples and redundancy is considered as an important problem in the supervised classification of hyperspectral data. This problem can be handled by Feature Subset Selection (FSS) methods and Support Vector Machine (SVM). The FSS methods can manage the redundancy by removing redundant spectral bands. Moreover, kernel based methods, especially SVM have a high ability to classify limited-sample data sets. This paper mainly aims to assess the capability of a FSS method and the SVM in curse of dimensional circumstances and to compare results with the Artificial Neural Network (ANN), when they are used to classify alteration zones of the Hyperion hyperspectral image acquired from the greatest Iranian porphyry copper complex. The results demonstrated that by decreasing training samples, the accuracy of SVM was just decreased 1.8% while the accuracy of ANN was highly reduced i.e. 14.01%. In addition, a hybrid FSS was applied to reduce the dimension of Hyperion. Accordingly, among the 165 useable spectral bands of Hyperion, 18 bands were only selected as the most important and informative bands. Although this dimensionality reduction could not intensively improve the performance of SVM, ANN revealed a significant improvement in the computational time and a slightly enhancement in the average accuracy. Therefore, SVM as a low-sensitive method respect to the size of training data set and feature space can be applied to classify the curse of dimensional problems. Also, the FSS methods can improve the performance of non-kernel based classifiers by eliminating redundant features.

Sliding Block-Based Hybrid Feature Subset Selection in Network Traffic

Traffic classification is currently a significant challenge for network monitoring and management. Feature selection is an effective method to realize dimension reduction and decrease redundant information. To realize accurate traffic classification at lower price of evaluations, a hybrid feature subset selection method is proposed on the base of sliding block, the size of which is flexible according to the classification performance. Furthermore, an incremental strategy of convergence is designed on the base of hybrid feature subset selection methods. The strategy gathers all the features that have been selected. To discover the value of relationship among all the selected features, an extra round of selection is added on the base of the original algorithm. The performances are examined by three groups of experiments. Our theoretical analysis and experimental observations reveal that the proposed method consumes fewer evaluations with similar or even better classification performance at different initialized size of block. Moreover, the incremental strategy of convergence makes a further improvement on the classification accuracy.

A Hybrid Feature Subset Selection using Metrics and Forward Selection

The aim of this study is to design a Feature Subset Selection Technique that speeds up the Feature Selection (FS) process in high dimensional datasets with reduced computational cost and great efficiency. FS has become the focus of much research on decision support system areas for which data with tremendous number of variables are analyzed. Filters and wrappers are proposed techniques for the feature subset selection process. Filters make use of association based approach but wrappers adopt classification algorithms to identify important features. Filter method lacks the ability of minimization of simplification error while wrapper method burden weighty computational resource. To pull through these difficulties, a hybrid approach is proposed combining both filters and wrappers. Filter approach uses a permutation of ranker search methods and a wrapper which improves the learning accurateness and obtains a lessening in the memory requirements and finishing time. The UCI machine learning repository was chosen to experiment the approach. The classification accuracy resulted from our approach proves to be higher.

A hybrid feature subset selection algorithm for analysis of high correlation proteomic data

Pathological changes within an organ can be reflected as proteomic patterns in biological fluids such as plasma, serum, and urine. The surface-enhanced laser desorption and ionization time-of-flight mass spectrometry (SELDI-TOF MS) has been used to generate proteomic profiles from biological fluids. Mass spectrometry yields redundant noisy data that the most data points are irrelevant features for differentiating between cancer and normal cases. In this paper, we have proposed a hybrid feature subset selection algorithm based on maximum-discrimination and minimum-correlation coupled with peak scoring criteria. Our algorithm has been applied to two independent SELDI-TOF MS datasets of ovarian cancer obtained from the NCI-FDA clinical proteomics databank. The proposed algorithm has used to extract a set of proteins as potential biomarkers in each dataset. We applied the linear discriminate analysis to identify the important biomarkers. The selected biomarkers have been able to successfully diagnose the ovarian cancer patients from the noncancer control group with an accuracy of 100%, a sensitivity of 100%, and a specificity of 100% in the two datasets. The hybrid algorithm has the advantage that increases reproducibility of selected biomarkers and able to find a small set of proteins with high discrimination power.