Wrapper Feature Selection Method Research Articles

Mapping wheat rust based on high spatial resolution satellite imagery

Timely and accurate disease incidence monitoring and estimation using satellite imagery is critical for the effective control of wheat rust disease. Though studies have been conducted using different spectral and sensing technologies to detect and monitor the wheat rust disease, few studies have been conducted in the application of high resolution multi-spectral satellite imagery in the monitoring of wheat rust to facilitate an operational monitoring over large areas. Since there are not many options for the multispectral data than the hyperspectral data, it is more important to select appropriate vegetation indices for the classification model. However, there have been few literatures about the comparison of these feature selection methods on the application of wheat disease. We proposed a wheat rust disease mapping protocol including removing the non-vegetated and lodged area, calculating and filtering the vegetation indices and mapping the wheat rust. In the process of filtering the proper features, we applied wrapper feature selection instead of traditional filter feature selection combined with the classification methods (support vector machine and random forests). The experiment data was a scene of ZY-3 satellite image of the wheat field in Changge County in Henan Province with a certain portion of rust disease. The classification results can achieve overall accuracy of higher than 90%, ranging from 90.80% to 95.10%. The wrapper feature selection method with the overall accuracy of 93.60% is better than filters feature selection method with the overall accuracy of 92.65%. The random forests method with the overall accuracy of 94.80% is better than support vector machine method with the overall accuracy of 91.45%. The high accuracies thus justified the feasibility of using high-resolution multi-spectral satellite images for mapping wheat rust disease, which is promising for this technology to be applied in the practical wheat production management.

Residential Electricity Consumption Level Impact Factor Analysis Based on Wrapper Feature Selection and Multinomial Logistic Regression

This paper aims to identity the significant impact factors (IFs) of the residential electricity consumption level (RECL) and to better understand the influence mechanism of IFs on RECL. The analysis of influence mechanism is commonly through regression model where feature selection must first be performed to pick out non-redundant IFs that is highly correlated with RECL. In contrast to the existing studies, this study recognizes the problem that majority feature selection methods (e.g., step regression) are limited to the identification of linear relationships and proposes a novel wrapper feature selection (WFS) method to address this issue. The WFS is based on genetic algorithm (GA) and multinomial logistic regression (MLR). GA is a searching algorithm used to generate different feature subsets (FSs) that consist of several IFs. MLR is a modeling algorithm used to score these FSs. Further, maximal information coefficient (MIC) is utilized to verify the validity of WFS for selecting IFs. Finally, MLR based explanatory model is established to excavate the relationship between selected IFs and RECL. The results of Ireland dataset based case study show that WFS can identify the significant and non-redundant IFs that are linearly or nonlinearly related to RECL. The details about how selected IFs affect RECL are also provided via the explanatory model. Such research can provide useful guidance for a wide range of stakeholders including local governments, electric power companies, and individual households.

IRSpot-PDI: Identification of recombination spots by incorporating dinucleotide property diversity information into Chou's pseudo components

Recombination spot identification plays an important role in revealing genome evolution and developing DNA function study. Although some computational methods have been proposed, extracting discriminatory information embedded in DNA properties has not received enough attention. The DNA properties include dinucleotide flexibility, structure and thermodynamic parameter, which are significant for genome evolution research. To explore the potential effect of DNA properties, a novel feature extraction method, called iRSpot-PDI, is proposed. A wrapper feature selection method with the best first search is used to identify the best feature set. To verify the effectiveness of the proposed method, support vector machine is employed on the obtained features. Prediction results are reported on two benchmark datasets. Compared with the recently reported methods, iRSpot-PDI achieves the highest values of individual specificity, Matthew's correlation coefficient and overall accuracy. The experimental results confirm that iRSpot-PDI is effective for accurate identification of recombination spots. The datasets can be downloaded from the following URL: http://stxy.neuq.edu.cn/info/1095/1157.htm.

MSMD: maximum separability and minimum dependency feature selection for cropland classification from optical and radar data

ABSTRACTCropland classification using optical and full polarimetric synthetic aperture radar (PolSAR) images is a topic of considerable interest in the remote-sensing community. These two data sources can provide a diverse set of temporal, spectral, textural and polarimetric features which can be invaluable for cropland classification. However, some optical features or some radar features may have a relatively high correlation with other features. Hence, it seems to be necessary to choose the optimum features in order to reduce the dimensions of the data and to improve cropland classification accuracy. This article proposes a strategic feature selection method from a feature set of bitemporal RapidEye and Uninhabited Aerial Vehicle synthetic aperture radar (UAVSAR) images. The proposed method is designed to select the most relevant features and to remove redundant features based on the two concepts of separability and dependency. The proposed method is therefore referred to as maximum separability and minimum dependency (MSMD). For evaluating efficiency, MSMD and some well-known filter and wrapper feature selection methods are compared using a random forest classifier. Experimental tests confirmed that the classification results obtained from the MSMD feature selection method were more accurate than those achieved by filter methods. Moreover, they had an accuracy comparable to that of the results from the wrapper method. Furthermore, with regard to running time, MSMD operated as fast as the filter methods. It had a straightforward structure compared to the wrapper method, and as a result was faster than this method.

Mid-Price Prediction Based on Machine Learning Methods with Technical and Quantitative Indicators

Stock price prediction is a challenging task, but machine learning methods have recently been used successfully for this purpose. In this paper, we extract over 270 hand-crafted features (factors) inspired by technical and quantitative analysis and tested their validity on short-term mid-price movement prediction. We focus on a wrapper feature selection method using entropy, least-mean squares, and linear discriminant analysis. We also build a quantitative feature based on adaptive logistic regression for online learning, which is constantly selected first among the majority of the proposed feature selection methods. This study examines the best combination of features using high frequency limit order book data from Nasdaq Nordic. Our results suggest that sorting methods and classifiers can be used in such a way that one can reach the best performance with a combination of only very few informative features. This paper opens avenues for developing more advanced features combined with more sophisticated feature selection methods. It also provides helpful insight to market makers and traders in general by providing useful results that can be used to gain an information edge in trading.

Intelligent Cooperative Adaptive Weight Ranking Policy via dynamic aging based on NB and J48 classifiers

The increased usage of World Wide Web leads to increase in network traffic and create a bottleneck over the internet performance. For most people, the accessing speed or the response time is the most critical factor when using the internet. Reducing response time was done by using web proxy cache technique that storing a copy of pages between client and server sides. If requested pages are cached in the proxy, there is no need to access the server. But, the cache size is limited, so cache replacement algorithms are used to remove pages from the cache when it is full. On the other hand, the conventional algorithms for replacement such as Least Recently Use (LRU), First in First Out (FIFO), Least Frequently Use (LFU), Randomised Policy, etc. may discard essential pages just before use. Furthermore, using conventional algorithms cannot be well optimized since it requires some decision to evict intelligently before a page is replaced. Hence, this paper proposes an integration of Adaptive Weight Ranking Policy (AWRP) with intelligent classifiers (NB-AWRP-DA and J48-AWRP-DA) via dynamic aging factor. To enhance classifiers power of prediction before integrating them with AWRP, particle swarm optimization (PSO) automated wrapper feature selection methods are used to choose the best subset of features that are relevant and influence classifiers prediction accuracy. Experimental Result shows that NB-AWRP-DA enhances the performance of web proxy cache across multi proxy datasets by 4.008%,4.087% and 14.022% over LRU, LFU, and FIFO while, J48-AWRP-DA increases HR by 0.483%, 0.563% and 10.497% over LRU, LFU, and FIFO respectively. Meanwhile, BHR of NB-AWRP-DA rises by 0.9911%,1.008% and 11.5842% over LRU, LFU, and FIFO respectively while 0.0204%, 0.0379% and 10.6136 for LRU, LFU, FIFO respectively using J48-AWRP-DA.

Intelligent Cooperative Adaptive Weight Ranking Policy via dynamic aging based on NB and J48 classifiers

The increased usage of World Wide Web leads to increase in network traffic and create a bottleneck over the internet performance. For most people, the accessing speed or the response time is the most critical factor when using the internet. Reducing response time was done by using web proxy cache technique that storing a copy of pages between client and server sides. If requested pages are cached in the proxy, there is no need to access the server. But, the cache size is limited, so cache replacement algorithms are used to remove pages from the cache when it is full. On the other hand, the conventional algorithms for replacement such as Least Recently Use (LRU), First in First Out (FIFO), Least Frequently Use (LFU), Randomised Policy, etc. may discard essential pages just before use. Furthermore, using conventional algorithms cannot be well optimized since it requires some decision to evict intelligently before a page is replaced. Hence, this paper proposes an integration of Adaptive Weight Ranking Policy (AWRP) with intelligent classifiers (NB-AWRP-DA and J48-AWRP-DA) via dynamic aging factor. To enhance classifiers power of prediction before integrating them with AWRP, particle swarm optimization (PSO) automated wrapper feature selection methods are used to choose the best subset of features that are relevant and influence classifiers prediction accuracy. Experimental Result shows that NB-AWRP-DA enhances the performance of web proxy cache across multi proxy datasets by 4.008%,4.087% and 14.022% over LRU, LFU, and FIFO while, J48-AWRP-DA increases HR by 0.483%, 0.563% and 10.497% over LRU, LFU, and FIFO respectively. Meanwhile, BHR of NB-AWRP-DA rises by 0.9911%,1.008% and 11.5842% over LRU, LFU, and FIFO respectively while 0.0204%, 0.0379% and 10.6136 for LRU, LFU, FIFO respectively using J48-AWRP-DA.

A novel ensemble-based wrapper method for feature selection using extreme learning machine and genetic algorithm

This paper presents a novel wrapper feature selection algorithm for classification problems, namely hybrid genetic algorithm (GA)- and extreme learning machine (ELM)-based feature selection algorithm (HGEFS). It utilizes GA to wrap ELM to search for the optimum subsets in the huge feature space, and then, a set of subsets are selected to make ensemble to improve the final prediction accuracy. To prevent GA from being trapped in the local optimum, we propose a novel and efficient mechanism specifically designed for feature selection problems to maintain GA’s diversity. To measure each subset’s quality fairly and efficiently, we adopt a modified ELM called error-minimized extreme learning machine (EM-ELM) which automatically determines an appropriate network architecture for each feature subsets. Moreover, EM-ELM has good generalization ability and extreme learning speed which allows us to perform wrapper feature selection processes in an affordable time. In other words, we simultaneously optimize feature subset and classifiers’ parameters. After finishing the search process of GA, to further promote the prediction accuracy and get a stable result, we select a set of EM-ELMs from the obtained population to make the final ensemble according to a specific ranking and selecting strategy. To verify the performance of HGEFS, empirical comparisons are carried out on different feature selection methods and HGEFS with benchmark datasets. The results reveal that HGEFS is a useful method for feature selection problems and always outperforms other algorithms in comparison.

Enhancement web proxy cache performance using Wrapper Feature Selection methods with NB and J48

Web proxy cache technique reduces response time by storing a copy of pages between client and server sides. If requested pages are cached in the proxy, there is no need to access the server. Due to the limited size and excessive cost of cache compared to the other storages, cache replacement algorithm is used to determine evict page when the cache is full. On the other hand, the conventional algorithms for replacement such as Least Recently Use (LRU), First in First Out (FIFO), Least Frequently Use (LFU), Randomized Policy etc. may discard important pages just before use. Furthermore, using conventional algorithm cannot be well optimized since it requires some decision to intelligently evict a page before replacement. Hence, most researchers propose an integration among intelligent classifiers and replacement algorithm to improves replacement algorithms performance. This research proposes using automated wrapper feature selection methods to choose the best subset of features that are relevant and influence classifiers prediction accuracy. The result present that using wrapper feature selection methods namely: Best First (BFS), Incremental Wrapper subset selection(IWSS)embedded NB and particle swarm optimization(PSO)reduce number of features and have a good impact on reducing computation time. Using PSO enhance NB classifier accuracy by 1.1%, 0.43% and 0.22% over using NB with all features, using BFS and using IWSS embedded NB respectively. PSO rises J48 accuracy by 0.03%, 1.91 and 0.04% over using J48 classifier with all features, using IWSS-embedded NB and using BFS respectively. While using IWSS embedded NB fastest NB and J48 classifiers much more than BFS and PSO. However, it reduces computation time of NB by 0.1383 and reduce computation time of J48 by 2.998.

A novel feature selection framework with Hybrid Feature-Scaled Extreme Learning Machine (HFS-ELM) for indoor occupancy estimation

Indoor occupancy estimation can be an important parameter for automating Air Conditioning and Mechanical Ventilation (ACMV) operations in buildings. In this work, we propose a feature selection framework for constructing an occupancy estimator. The framework has two main components: a filter component, which uses a filter method of feature selection and a wrapper component, which implements a wrapper method of feature selection with a machine learning based occupancy estimator. The framework is thus a kind of filter-wrapper hybrid feature selection method. However, the framework is novel in that it uses a combination of static and dynamic features. We use the static features for the purpose of speed, since filter methods of feature selection (which work with static features) are quite fast. Thus, the overall computation time of the framework is kept low, while ensuring good accuracy of estimation due to the use of dynamic features in the wrapper stage. To perform occupancy estimation within the proposed framework, we present a novel technique called the Hybrid Feature-Scaled Extreme Learning Machine (HFS-ELM). The HFS-ELM is a dynamic model of the occupancy level that extracts dynamic features from its inputs. The architecture of the HFS-ELM method is explained in detail. Experimental results in an office space show the effectiveness of the proposed framework.

Feature Selection from 3D Brain Model for Some Dementia Subtypes Using Genetic Algorithm

Brain scans that are appropriate to the medical standards are obtained from magnetic resonance imaging devices. Through image processing techniques, 3D brain models can be constructed by mapping medical brain imaging files structurally. Physical characteristics of patient brains can be extracted from those 3D brain models. Characteristics of some specific brain regions are more efficacious in predicting the type of the disease. For that reason, researches are made for finding the worthwhile features out using cortical volumes, gray volumes, surface areas, and thickness averages for left and right brain parts separately or together. The main objective of this work is determining more influential sections throughout the entire brain in establishing the clinical diagnosis. To that end, among all the measurements exported from 3D models, the significant brain features that are effective in identifying some dementia subtypes are sought. The dataset has 3D brain models generated from magnetic resonance scans of 63 samples. Each sample is labeled with one of the following three disease types: Alzheimer’s disease (19), frontotemporal dementia (19), and vascular dementia (25). The genetic algorithm based wrapper feature selection method with various classifiers is proposed to select the features that state the aforementioned dementia subtypes best. The tests are performed by applying cross validation technique and confusion matrices are shown. At the end, the best features are listed, and the accuracy results up to 95.2% are achieved.

Wrapper method for feature selection to classify cardiac arrhythmia.

Efficient monitoring of cardiac patients can save tremendous amount of lives. Cardiac disease prediction and classification has gained utmost significance in this regard during the past few years. This paper presents a predictive model for classification of arrhythmias. The model works by selecting best features using wrapper algorithm around random forest, followed by implementing various machine learning classifiers on the selected features. Cardiac arrhythmia dataset from University of California, Irvine (UCI) machine learning repository has been used for the experimental purpose. After normalizing the data, repeated cross validation with 10 folds is applied on support vector machine (SVM), K nearest neighbor (KNN), Naïve Bayes, random forest, and Multi-Layer perceptron (MLP). The experimental results demonstrate that MLP beats other classifiers by achieving an average accuracy of 78.26%, while accuracies calculated for KNN and SVM are 76.6% and 74.4% respectively, outperforming the accuracies of previous models.

Hybrid Whale Optimization Algorithm with simulated annealing for feature selection

Hybrid metaheuristics are of the most interesting recent trends in optimization and memetic algorithms. In this paper, two hybridization models are used to design different feature selection techniques based on Whale Optimization Algorithm (WOA). In the first model, Simulated Annealing (SA) algorithm is embedded in WOA algorithm, while it is used to improve the best solution found after each iteration of WOA algorithm in the second model. The goal of using SA here is to enhance the exploitation by searching the most promising regions located by WOA algorithm. The performance of the proposed approaches is evaluated on 18 standard benchmark datasets from UCI repository and compared with three well-known wrapper feature selection methods in the literature. The experimental results confirm the efficiency of the proposed approaches in improving the classification accuracy compared to other wrapper-based algorithms, which insures the ability of WOA algorithm in searching the feature space and selecting the most informative attributes for classification tasks.

A BPSO-SVM algorithm based on memory renewal and enhanced mutation mechanisms for feature selection

Feature selection (FS) is an essential component of data mining and machine learning. Most researchers devoted to get more effective method with high accuracy and fewer features, it has become one of the most challenging problems in FS. Certainly, some algorithms have been proven to be effectively, such as binary particle swarm optimization (BPSO), genetic algorithm (GA) and support vector machine (SVM). BPSO is a metaheuristic algorithm having been widely applied to various fields and applications successfully, including FS. As a wrapper method of FS, BPSO-SVM tends to be trapped into premature easily. In this paper, we present a novel mutation enhanced BPSO-SVM algorithm by adjusting the memory of local and global optimum (LGO) and increasing the particles’ mutation probability for feature selection to overcome convergence premature problem and achieve high quality features. Typical simulated experimental results carried out on Sonar, LSVT and DLBCL datasets indicated that the proposed algorithm improved the accuracy and decreased the number of feature subsets, comparing with existing modified BPSO algorithms and GA.

A new feature selection method based on a validity index of feature subset

The wrapper feature selection method can achieve high classification accuracy. However, the cross-validation scheme of the wrapper method in evaluation phase is very expensive regarding computing resource consumption. In this paper, we propose a new statistical measure named as LW-index which could replace the expensive cross-validation scheme to evaluate the feature subset. Then, a new feature selection method, which is the combination of the proposed LW-index with Sequence Forward Search algorithm (SFS-LW), is presented in this paper. Further, we show through plenty of experiments conducted on nine UCI datasets that the proposed method can obtain similar classification accuracy as the wrapper method with centroid-based classifier or support vector machine, and its computation cost is approximate to the compared filter methods.

Comparison and integration of feature reduction methods for land cover classification with RapidEye imagery

Feature reduction (FR) methods can effectively reduce the feature set and improve the accuracy for land cover classification (LCC) using high resolution remote sensing data with high dimensional or strongly correlated feature sets. However, FR methods have rarely been applied for LCC in arid regions with complex geographic environments, especially for the integration of feature selection (FS) and feature extraction (FE) methods. This study investigated the comparison and integration of FR methods for LCC in part of Dunhuang Basin, northwestern China, which is a typical inland arid region and groundwater-dependent ecosystems. Five spectral bands and 9 vegetation indices features that derived from RapidEye satellite imagery were used with support vector machines algorithm. Two wrapper FS methods, based on random forest algorithm (varSelRF and Boruta packages in R software), were used. Three FE methods (principal component analysis, PCA; independent component analysis, ICA; and minimum noise fraction transformation, MNF), were employed to extract a reduced number of reconstructed new features. Integration of varSelRF and PCA methods (varSelRF-PCA) was attempted. All 14 features were relevant, indicated by Boruta method; only 6 features, including the red-edge band selected by the varSelRF module, had higher importance. All the five FR methods could improve classification accuracy, but only varSelRF achieved significant improvement. The varSelRF outperformed the FE methods, followed by MNF, PCA, and ICA. The proposed varSelRF-PCA model significantly improved classification accuracy and outperformed all the FS or FE methods.

The Fusion Model of Multidomain Context Information for the Internet of Things

The Internet of Things aims to provide the user with deep adaptive intelligence services according to the user’s personalized characteristics. Most of the characteristics are presented in the form of high-level context. But it often lacks methods to obtain high-level context information directly in the Internet of Things. In this paper, so as to achieve the corresponding high-level context information using the specific low-level multidomain context directly obtained by different sensors in the Internet of Things, we present a machine learning method to construct a context fusion model based on the feature selection algorithm and the multiclassification algorithm. First, we propose a wrapper feature selection method based on the genetic algorithm to obtain a simpler and more important subset of the context features from the low-level multidomain context, by defining a suitable fitness function and a convergence condition. Then, we use the decision tree algorithm which is a multiclassification algorithm, based on the rules obtained by training the subset of context features, to determine which high-level context the record set of the low-level context information belongs to. Experiments confirm that the model can be used to achieve higher classification accuracy without more significant time consumption.

Detecting Bipolar Depression From Geographic Location Data.

ObjectiveThis paper aims to identify periods of depression using geolocation movements recorded from mobile phones in a prospective community study of individuals with bipolar disorder (BD).MethodsAnonymized geographic location recordings from 22 BD participants and 14 healthy controls (HC) were collected over 3 months. Participants reported their depressive symptomatology using a weekly questionnaire (QIDS-SR16). Recorded location data were preprocessed by detecting and removing imprecise data points and features were extracted to assess the level and regularity of geographic movements of the participant. A subset of features were selected using a wrapper feature selection method and presented to 1) a linear regression model and a quadratic generalized linear model with a logistic link function for questionnaire score estimation; and 2) a quadratic discriminant analysis classifier for depression detection in BD participants based on their questionnaire responses.ResultsHC participants did not report depressive symptoms and their features showed similar distributions to nondepressed BD participants. Questionnaire score estimation using geolocation-derived features from BD participants demonstrated an optimal mean absolute error rate of 3.73, while depression detection demonstrated an optimal (median ± IQR) F1 score of 0.857 ± 0.022 using five features (classification accuracy: 0.849 ± 0.016; sensitivity: 0.839 ± 0.014; specificity: 0.872 ± 0.047).ConclusionThese results demonstrate a strong link between geographic movements and depression in bipolar disorder.SignificanceTo our knowledge, this is the first community study of passively recorded objective markers of depression in bipolar disorder of this scale. The techniques could help individuals monitor their depression and enable healthcare providers to detect those in need of care or treatment.

A new hybrid filter–wrapper feature selection method for clustering based on ranking

Feature selection is a common task in areas such as Pattern Recognition, Data Mining, and Machine Learning since it can help to improve prediction quality, reduce computation time and build more understandable models. Although feature selection for supervised classification has been widely studied, feature selection in the absence of class labels, namely feature selection for clustering or unsupervised feature selection, has been less addressed. Most existing unsupervised feature selection approaches suffer from the called “Bias of Criterion Values to Dimension,” which arises when feature subsets with different cardinality are evaluated by an internal evaluation clustering criterion. In this paper, we introduce a new hybrid filter–wrapper method for clustering, which combines the spectral feature selection framework using the Laplacian Score ranking and a modified Calinski–Harabasz index. The proposed method in the filter stage sorts the features according to their relevance, while in the wrapper stage, through our modified Calinski–Harabasz index that takes into account the cardinality of the feature subsets under evaluation, evaluates the features considering them as a subset rather than individually by using two well-known selection strategies. Experiments on different datasets show that the proposed method alleviates the “Bias of Criterion Values to Dimension” and, identifies and selects more relevant features than those selected by other reported hybrid filter–wrapper feature selection methods for clustering. Additionally, we also contrast our results against other filter and wrapper methods of the state-of-the-art.

Classification of patients with MCI and AD from healthy controls using directed graph measures of resting-state fMRI

Brain network alterations in patients with Alzheimer’s disease (AD) has been the subject of much investigation, but the biological mechanisms underlying these alterations remain poorly understood. Here, we aim to identify the changes in brain networks in patients with AD and mild cognitive impairment (MCI), and provide an accurate algorithm for classification of these patients from healthy control subjects (HC) by using a graph theoretical approach and advanced machine learning methods. Multivariate Granger causality analysis was performed on resting-state functional magnetic resonance imaging (rs-fMRI) data of 34 AD, 89 MCI, and 45 HC to calculate various directed graph measures. The graph measures were used as the original feature set for the machine learning algorithm. Filter and wrapper feature selection methods were applied to the original feature set to select an optimal subset of features. An accuracy of 93.3% was achieved for classification of AD, MCI, and HC using the optimal features and the naïve Bayes classifier. We also performed a hub node analysis and found that the number of hubs in HC, MCI, and AD were 12, 10, and 9, respectively, suggesting that patients with AD experience disturbance of critical communication areas in their brain network as AD progresses. The findings of this study provide insight into the neurophysiological mechanisms underlying MCI and AD. The proposed classification method highlights the potential of directed graph measures of rs-fMRI data for identification of the early stage of AD.