Wrapper Feature Selection Technique Research Articles

Prediction of Pavement Overall Condition Index Based on Wrapper Feature-Selection Techniques Using Municipal Pavement Data

Municipalities make significant efforts with limited resources to collect pavement condition data. Overall Condition Index (OCI), which uses the pavement surface evaluation and rating manual to identify roadways needing repair, is a convenient and common way of pavement condition assessment. Many data used in assessing the OCI are collected from fieldwork. Some data features give little insight into road conditions, and one feature may provide similar information to another; thus, effective data-collection resources can be optimized by selecting which data feature to keep and which to discard. In addition, the OCI reflects how local agencies highlight the important variables driving their pavement-management investments. It is also a reflection of the triggers that they use to propose various treatment strategies. This research aimed to evaluate pavement distresses in West Des Moines, Iowa, using machine-learning methods, and determine which combination of distresses and their distress proportions can accurately predict the OCI class of a particular pavement type. The wrapper feature-selection methods were used, fitting their results to classification-tree models. Automatic feature selection, using Featurewiz, was also considered to select the desired number of variables. Feature parameters were screened for OCI prediction using the mean decrease impurity, and the results could be used to model classification that may be used for year-long predictions. Results showed that an effective OCI estimate methodology could be determined with significant accuracy with fewer features.

A hybrid framework for short term load forecasting with a navel feature engineering and adaptive grasshopper optimization in smart grid

Short-term load forecasting (STLF) enables distribution system operators to perform efficient energy management by flexibly engaging energy consumers under the intelligent demand-response program in the smart grid (SG). This paper develops a fast and accurate hybrid load forecasting model with this motivation. The proposed model integrates a locally weighted support vector regression (LWSVR) based forecaster with two modules. These modules are feature engineering (FE) and adaptive grasshopper optimization (AGO) based optimizers. In the FE module, firstly, the proposed hybrid feature selector (HFS) is developed using wrapper and filter techniques to determine an optimal subset of features. Furthermore, the instance-based Relief-F (REF) and information theoretic-based mutual information (MI) filters are used to decrease the curse of feature dimensionality by finding and eradicating extraneous features. In addition, to overcome the overfitting problem, the HFS module is further optimized using the recursive feature elimination (RFE) wrapper feature selection technique. The essential features are then extracted using a radial basis Kernel-based principal component analysis (RBF-KPCA) algorithm to eliminate the dimensionality reduction problem. The AGO algorithm tunes the LWSVR model’s appropriate parameters to effectively evade entrapping into local optimum and yield accurate prediction results. However, the efficacy and productiveness of the forecasting model are differentiated equally by its convergence rate and stability. Actual hourly load data of two states of Australia (New South Wales (NSW) and Victoria (VIC)) and California Independent System Operator United States (CAISO-US) are employed as a case study to estimate the effectiveness and applicability of the designed model. Empirical results show that the devised model surpasses benchmark models (single and hybrid) in terms of stability, accuracy, and convergence rate.

Performance Analysis of Feature Subset Selection Techniques for Intrusion Detection

An intrusion detection system is one of the main defense lines used to provide security to data, information, and computer networks. The problems of this security system are the increased processing time, high false alarm rate, and low detection rate that occur due to the large amount of data containing various irrelevant and redundant features. Therefore, feature selection can solve this problem by reducing the number of features. Choosing appropriate feature selection methods that can reduce the number of features without a negative effect on the classification accuracy is a major challenge. This challenge motivated us to investigate the application of different wrapper feature selection techniques in intrusion detection. The performance of the selected techniques, such as the genetic algorithm (GA), sequential forward selection (SFS), and sequential backward selection (SBS), were analyzed, addressed, and compared to the existing techniques. The efficiency of the three feature selection techniques with two classification methods, including support vector machine (SVM) and multi perceptron (MLP), was compared. The CICIDS2017, CSE-CIC-IDS218, and NSL-KDD datasets were considered for the experiments. The efficiency of the proposed models was proved in the experimental results, which indicated that it had highest accuracy in the selected datasets.

Implementation of Features Selection Based on Dragonfly Optimization Algorithm

Nowadays increasing dimensionality of data produces several issues in machine learning. Therefore, it is needed to decrease the number of features by choosing just the most important ones and eliminating duplicate features, also reducing the number of features that are important to the model. For this purpose, many methodologies known as Feature Selection are applied. In this study, a feature selection approach is proposed based on Swarm Intelligence methods, which search for the best points in the search area to achieve optimization. In this paper, a wrapper feature selection technique based on the Dragonfly algorithm is proposed. The dragonfly optimization technique is used to find the optimal subset of features that could accurately classify breast cancer as benign or malignant. Many times, the fitness function is defined as classification accuracy. In this study, hard vote classes are employed as a model developed to evaluate feature subsets that have been chosen. It is used as an evaluation function (fitness function) to evaluate each dragonfly in the population. The proposed ensemble hard voting classifier utilizes a combination of five machine-learning algorithms to produce a binary classification for feature selection: Support Vector Machine (SVM), K-Nearest Neighbors (K-NN), Naive Bayes (NB), Decision Tree (DT), and Random Forest (RF). According to the results of the experiments, the voting ensemble classifier has the greatest accuracy value among the single classifiers. The proposed method showed that when training the subset features, the accuracy generated by the voting classifier is high at 98.24%, whereas the training of all features achieved an accuracy of 96.49%. The proposed approach makes use of the UCI repository's Wisconsin Diagnostic Breast Cancer (WDBC) Dataset. Which consists of 569 instances and 30 features.

Hybrid Binary Butterfly Optimization Algorithm and Simulated Annealing for Feature Selection Problem

Feature selection is performed to eliminate irrelevant features to reduce computational overheads. Metaheuristic algorithms have become popular for the task of feature selection due to their effectiveness and flexibility. Hybridization of two or more such metaheuristics has become popular in solving optimization problems. In this paper, we propose a hybrid wrapper feature selection technique based on binary butterfly optimization algorithm (bBOA) and Simulated Annealing (SA). The SA is combined with the bBOA in a pipeline fashion such that the best solution obtained by the bBOA is passed on to the SA for further improvement. The SA solution improves the best solution obtained so far by searching in its neighborhood. Thus the SA tries to enhance the exploitation property of the bBOA. The proposed method is tested on twenty datasets from the UCI repository and the results are compared with five popular algorithms for feature selection. The results confirm the effectiveness of the hybrid approach in improving the classification accuracy and selecting the optimal feature subset.

An Intelligent handcrafted feature selection using Archimedes optimization algorithm for facial analysis.

Human facial analysis (HFA) has recently become an attractive topic for computer vision research due to technological progress and mobile applications. HFA explores several issues as gender recognition (GR), facial expression, age, and race recognition for automatically understanding social life. This study explores HFA from the angle of recognizing a person’s gender from their face. Several hard challenges are provoked, such as illumination, occlusion, facial emotions, quality, and angle of capture by cameras, making gender recognition more difficult for machines. The Archimedes optimization algorithm (AOA) was recently designed as a metaheuristic-based population optimization method, inspired by the Archimedes theory’s physical notion. Compared to other swarm algorithms in the realm of optimization, this method promotes a good balance between exploration and exploitation. The convergence area is increased By incorporating extra data into the solution, such as volume and density. Because of the preceding benefits of AOA and the fact that it has not been used to choose the best area of the face, we propose utilizing a wrapper feature selection technique, which is a real motivation in the field of computer vision and machine learning. The paper’s primary purpose is to automatically determine the optimal face area using AOA to recognize the gender of a human person categorized by two classes (Men and women). In this paper, the facial image is divided into several subregions (blocks), where each area provides a vector of characteristics using one method from handcrafted techniques as the local binary pattern (LBP), histogram-oriented gradient (HOG), or gray-level co-occurrence matrix (GLCM). Two experiments assess the proposed method (AOA): The first employs two benchmarking datasets: the Georgia Tech Face dataset (GT) and the Brazilian FEI dataset. The second experiment represents a more challenging large dataset that uses Gallagher’s uncontrolled dataset. The experimental results show the good performance of AOA compared to other recent and competitive optimizers for all datasets. In terms of accuracy, the AOA-based LBP outperforms the state-of-the-art deep convolutional neural network (CNN) with 96.08% for the Gallagher’s dataset.

Feature Selection for Cluster Analysis in Spectroscopy

Cluster analysis in spectroscopy presents some unique challenges due to the specific data characteristics in spectroscopy, namely, high dimensionality and small sample size. In order to improve cluster analysis outcomes, feature selection can be used to remove redundant or irrelevant features and reduce the dimensionality. However, for cluster analysis, this must be done in an unsupervised manner without the benefit of data labels. This paper presents a novel feature selection approach for cluster analysis, utilizing clusterability metrics to remove features that least contribute to a dataset's tendency to cluster. Two versions are presented and evaluated: The Hopkins clusterability filter which utilizes the Hopkins test for spatial randomness and the Dip clusterability filter which utilizes the Dip test for unimodality. These new techniques, along with a range of existing filter and wrapper feature selection techniques were evaluated on eleven real-world spectroscopy datasets using internal and external clustering indices. Our newly proposed Hopkins clusterability filter performed the best of the six filter techniques evaluated. However, it was observed that results varied greatly for different techniques depending on the specifics of the dataset and the number of features selected, with significant instability observed for most techniques at low numbers of features. It was identified that the genetic algorithm wrapper technique avoided this instability, performed consistently across all datasets and resulted in better results on average than utilizing the all the features in the spectra.

Detecting Cognitive Impairment Status Using Keystroke Patterns and Physical Activity Data among the Older Adults: A Machine Learning Approach.

Cognitive impairment has a significantly negative impact on global healthcare and the community. Holding a person's cognition and mental retention among older adults is improbable with aging. Early detection of cognitive impairment will decline the most significant impact of extended disease to permanent mental damage. This paper aims to develop a machine learning model to detect and differentiate cognitive impairment categories like severe, moderate, mild, and normal by analyzing neurophysical and physical data. Keystroke and smartwatch have been used to extract individuals' neurophysical and physical data, respectively. An advanced ensemble learning algorithm named Gradient Boosting Machine (GBM) is proposed to classify the cognitive severity level (absence, mild, moderate, and severe) based on the Standardised Mini-Mental State Examination (SMMSE) questionnaire scores. The statistical method “Pearson's correlation” and the wrapper feature selection technique have been used to analyze and select the best features. Then, we have conducted our proposed algorithm GBM on those features. And the result has shown an accuracy of more than 94%. This paper has added a new dimension to the state-of-the-art to predict cognitive impairment by implementing neurophysical data and physical data together.

Swarm Intelligence-Based Feature Selection for Amphetamine-Type Stimulants (ATS) Drug 3D Molecular Structure Classification

ABSTRACT Swarm intelligence-based feature selection techniques are implemented by this work to increase classifier performance in classifying Amphetamine-type Stimulants (ATS) drugs. A recently proposed 3D Exact Legendre Moment Invariants (3D-ELMI) molecular descriptors as 3D molecular structure representational for ATS drugs. These descriptors are utilized as the dataset in this study. However, a large number of descriptors may cause performance degradation in the classifier. To complement this issue, this research applies three swarm algorithms with k-Nearest Neighbor (k-NN) classifier in the wrapper feature selection technique to ensure only relevant descriptors are selected for the ATS drug classification task. For this purpose, the binary version of swarm algorithms facilitated with the S-shaped or sigmoid transfer function known as binary whale optimization algorithm (BWOA), binary particle swarm optimization algorithm (BPSO), and new binary manta-ray foraging optimization algorithm (BMRFO) are developed for feature selection. Their performance is evaluated and compared based on seven performance criteria. Furthermore, the optimal feature subset was then evaluated with seven different classifiers. Findings from this study have revealed the dominance of BWOA by obtaining the highest classification accuracy with the small feature size.

Outlier Detection Based Feature Selection Exploiting Bio-Inspired Optimization Algorithms

The curse of dimensionality problem occurs when the data are high-dimensional. It affects the learning process and reduces the accuracy. Feature selection is one of the dimensionality reduction approaches that mainly contribute to solving the curse of the dimensionality problem by selecting the relevant features. Irrelevant features are the dependent and redundant features that cause noise in the data and then reduce its quality. The main well-known feature-selection methods are wrapper and filter techniques. However, wrapper feature selection techniques are computationally expensive, whereas filter feature selection methods suffer from multicollinearity. In this research study, four new feature selection methods based on outlier detection using the Projection Pursuit method are proposed. Outlier detection involves identifying abnormal data (irrelevant features of the transpose matrix obtained from the original dataset matrix). The concept of outlier detection using projection pursuit has proved its efficiency in many applications but has not yet been used as a feature selection approach. To the author’s knowledge, this study is the first of its kind. Experimental results on nineteen real datasets using three classifiers (k-NN, SVM, and Random Forest) indicated that the suggested methods enhanced the classification accuracy rate by an average of 6.64% when compared to the classification accuracy without applying feature selection. It also outperformed the state-of-the-art methods on most of the used datasets with an improvement rate ranging between 0.76% and 30.64%. Statistical analysis showed that the results of the proposed methods are statistically significant.

Binary Jaya algorithm based on binary similarity measure for feature selection

Feature selection (FS) has become an indispensable data preprocessing task because of the huge amount of high dimensional data being generated by current technologies. These high dimensional data contains irrelevant, redundant, and noisy features that deteriorate classification accuracy. FS reduces dimensionality by removing the unwanted features thus improves classification accuracy. FS can be considered as a binary optimization problem. In order to solve this problem, this work proposes a new wrapper feature selection technique based on the Jaya algorithm. Three binary variants of the Jaya algorithm are proposed, the first and second ones are based on transfer functions namely BJaya-S and BJaya-V. The third variant (BJaya-JS) explores the search space on the basis of the Jaccard Similarity index. In addition, a probability-based local search technique, namely Neighbourhood Search is proposed to balance the exploration and exploitation. The variants of Jaya algorithm are evaluated and the best variant is selected. The best variant is further compared with six state-of-the-art feature selection techniques. All the performances are tested on 18 high dimensional standard UCI datasets. Experimental result comparison shows that the proposed feature selection technique performs better than other competitors.

Feature selection using Binary Crow Search Algorithm with time varying flight length

Crow Search Algorithm (CSA) is a simple yet effective meta-heuristic algorithm that has been applied to solve many engineering problems. In CSA, fl parameter controls the search capability of crows and AP parameter balances the trade-off between exploration and exploitation. The parameter fl is initialized to a constant value in CSA. However, CSA faces the problem of being trapped in local minima. This work proposes the solution to this problem by introducing the new concept of time varying flight length in CSA. The value of fl should be large in initial stages of algorithm in order to support random exploration and it should gradually decrease in later iterations to encourage the exploitation of good solutions found so far. The proposed approach, Binary Crow Search Algorithm with Time Varying Flight Length (BCSA-TVFL) is applied to feature selection problems in wrapper mode. Eight variants of BCSA-TVFL based on eight different transfer functions are tested. The best performing variant is then selected and compared with other state-of-the-art wrapper feature selection techniques and standard filter feature selection techniques. Performance of proposed approach is tested on 20 standard UCI datasets. Experimental result comparison shows that the proposed feature selection technique performs better than other competitors.

Quantum-like mutation-induced dragonfly-inspired optimization approach

This study proposed an improved dragonfly algorithm (DA). This algorithm is a recently proposed metaheuristic optimizer inspired by swarming behaviors of dragonflies, which has reasonably achieved satisfactory results in dealing with engineering, education, and other fields. However, the original method will show some shortcomings in convergence speed or falling into local optimum. Given these shortcomings, this paper proposes an improved optimizer to balance the relationship between exploitation and exploration and mitigate any deficiency. First, by implementing the idea of the quantum rotation gate, the swarm of agents can be shifted to a position more conducive to the optimal value. Then, Gaussian mutation is adopted to improve the swarm’s ability to mutate and realize its diversity, which enables the primary method to have a strong local search capability. The proposed method was compared against six other common meta-heuristics and five state-of-the-art algorithms on a comprehensive set of nineteen functions selected from twenty-three classic benchmark problems and thirty IEEE (Institute of Electrical and Electronics Engineers) CEC (Congress on Evolutionary Computation) 2014 benchmark tasks. To verify the effectiveness of the approach, the non-parametric statistical Wilcoxon signed-rank and Friedman tests were performed to validate the significance of the proposed method against other counterparts. The results of experimental simulations demonstrate that two introduced strategies can significantly improve the exploitative and exploratory tendencies of the original algorithm. Furthermore, the convergence speed of the conventional approach has been improved to a large extent. Additionally, quantum-behaved and Gaussian mutational dragonfly algorithm (QGDA) is utilized as a searching core in a wrapper feature selection technique, and it is compared with other advanced feature selection methods. The results show that QGDA achieves substantial superiority in feature selection through optimum fitness and minimum error rate. Also, the results of QGDA on the three classical engineering design problems have demonstrated that the proposed method can effectively solve these constraints problems. It is encouraging that the proposed method can be used as a useful, auxiliary tool for solving complex optimization problems.

A Stable SVM-RFE Feature Selection Method for Gene Expression Data

Feature Selection techniques are generally employed to remove the inessential attributes before machine learning technique could be applied. It thus plays an extremely important role by eliminating the unnecessary features that do not contribute and sometimes degrade the performance and prediction accuracy of the machine learning technique. With the growth of dimensionality of data, Feature Selection becomes even more important because it helps to reduce the dimensions of data and hence decreases the requisite memory and computational complexity of the machine learning techniques. Support vector machine-recursive feature elimination (SVM-RFE) has proven to be an efficient wrapper feature selection technique which continues to be widely utilized in many applications, especially in classification of gene expression data. From the perspective of this data, not only the precision in classification but also the stability of Feature Selection method plays an important role. Nonetheless, the topic of stability is ignored in study of feature selection algorithms. To improve the stability of RFE method, a fusion of Information Gain and RFE (IG-RFE-SVM) method is proposed in this paper. Experimental studies show that IG-RFE-SVM outperforms SVM-RFE method in terms of stability

A novel hybrid wrapper–filter approach based on genetic algorithm, particle swarm optimization for feature subset selection

The classification is one of the main technique of machine learning science. In many problems, the data sets have a high dimensionality that the existence of all features is not important to the purpose of the problem, and this will decrease the accuracy and performance of the algorithm. In this situation, the feature selection will play a significant role, and by eliminating unrelated features, the efficiency of the algorithm will be increased. A hybrid filter-wrapper method is proposed in the present study for feature subset selection established with integration of evolutionary based genetic algorithms (GA) and particle swarm optimization (PSO). The presented method mainly aims to reduce the complication of calculation and the search time expended to achieve an optimum solution to the high dimensional datasets feature selection problem. The proposed method, named smart HGP-FS, utilizes artificial neural network (ANN) in the fitness function. The filter and wrapper methods are integrated in order to take the benefit of filter technique acceleration and the wrapper technique vigor for selection of dataset efficacious characteristics. Some dataset characteristics are eliminated through the filter phase, which in turn reduces complex computations and search time in the wrapper phase. Comparisons have been made for the effectiveness of the proposed hybrid algorithm with the usability of three hybrid filter-wrapper methods, two pure wrapper algorithms, two pure filter procedures, and two traditional wrapper feature selection techniques. The findings obtained over real-world datasets show the efficiency of the presented algorithm. The outcomes of algorithm examination on five datasets reveal that the developed method is able to obtain a more accurate classification and to remove unsuitable and unessential characteristics more effectively relative to the other approaches.

Classification of foot drop gait characteristic due to lumbar radiculopathy using machine learning algorithms

Background: Recently, the study of walking gait has received significant attention due to the importance of identifying disorders relating to gait patterns. Characterisation and classification of different common gait disorders such as foot drop in an effective and accurate manner can lead to improved diagnosis, prognosis assessment, and treatment. However, currently visual inspection is the main clinical method to evaluate gait disorders, which is reliant on the subjectivity of the observer, leading to inaccuracies.Research question: This study examines if it is feasible to use commercial off-the-shelf Inertial measurement unit sensors and supervised learning methods to distinguish foot drop gait disorder from the normal walking gait pattern.Method: The gait data collected from 56 adults diagnosed with foot drop due to L5 lumbar radiculopathy (with MRI verified compressive pathology), and 30 adults with normal gait during multiple walking trials on a flat surface. Machine learning algorithms were applied to the inertial sensor data to investigate the feasibility of classifying foot drop disorder.Results: The best three performing results were 88.45%, 86.87% and 86.08% accuracy derived from the Random Forest, SVM, and Naive Bayes classifiers respectively. After applying the wrapper feature selection technique, the top performance was from the Random Forest classifier with an overall accuracy of 93.18%.Significance: It is demonstrated that the combination of inertial sensors and machine learning algorithms, provides a promising and feasible solution to differentiating L5 radiculopathy related foot drop from normal walking gait patterns. The implication of this finding is to provide an objective method to help clinical decision making.

A novel GA-ELM model for patient-specific mortality prediction over large-scale lab event data

Patient-specific mortality prediction models are an essential component of Clinical Decision Support Systems developed for caregivers in Intensive Care Units (ICUs), that enable timely decisions towards effective patient care and optimized ICU resource management. While high prediction accuracy is a fundamental requirement for any mortality prediction application, being able to so with minimal patient-specific data is a major plus point that can help in improving care delivery and cost optimization. Most existing scoring techniques and prediction models utilize a multitude of lab tests and patient events to predict mortality and also suffer from reduced performance when available patient data is less. In this paper, a Genetic Algorithm based Wrapper Feature Selection technique is proposed for determining most-optimal lab events that contribute predominantly to mortality, even for large-scale patient cohorts. Using this, an Extreme Learning Machine (ELM) based neural network is designed for predicting patient-specific ICU mortality. The proposed GA-ELM model was benchmarked against four popular traditional mortality scores and also state-of-the-art machine learning models for experimental validation. The GA-ELM model achieved promising results as it outperformed the traditional scoring systems by 11%–29% and state-of-the-art models by up to 14%, in terms of AUROC.

A Wrapper Feature Selection Technique for Improving Diagnosis of Breast Cancer

A Wrapper Feature Selection Technique for Improving Diagnosis of Breast Cancer

Significance of Hybrid Feature Selection Technique for Intrusion Detection Systems

Objectives: Intrusion detection is the need of technical world where data is generating and changing at a very rapid rate. In last decade feature selection is the science that has given a new perspective to research in the area of Intrusion Detection System. Objective of this paper is to perform an analysis and comparison of various feature selection techniques with a new technique of hybrid Particle Swarm Optimization (PSO). Statistical Analysis: In this paper well known filter and wrapper feature selection techniques have been explored along with a hybrid PSO technique on the standard KDDCup99 dataset. A comparative analysis is performed over four filter techniques and two wrapper based techniques. Four different classifiers are compared to select the one providing good accuracy on the dataset. Findings: The hybrid PSO feature selection technique gives significant improvement in prediction capability as compared to traditional feature selection approaches. Analysis shows that SVM classifier provides better classification results. SVM is used as classifier because of its high accuracy. The analysis over 4 filter and two wrapper techniques shows that Hybrid PSO provides better results with 98.6% accuracy and 24 feature subset. Application/Improvements: Analysis provides importance of hybrid PSO, which may be applied to not only intrusion detection but also various other areas where feature reduction is required.

Feature Selection on MR Images Using Genetic Algorithm with SVM and Naive Bayes Classifiers

Dementias are termed as neuropsychiatric disorders. Brain images of dementia patients can be obtained through magnetic resonance imaging systems. The relevant disease can be diagnosed by examining critical regions of those images. Certain brain characteristics such as the cortical volume, the thickness, and the surface area may vary among dementia types. These attributes can be expressed as numerical values using image processing techniques. In this study, the dataset involves T1 medical image sets of 63 samples. Each particular sample is labeled with one of the three dementia types: Alzheimer's disease, frontotemporal dementia, and vascular dementia. The image sets are processed to create different feature groups. These are cortical volumes, gray volumes, surface areas, and thickness averages. The main objective is seeking brain sections more effective in establishing the clinical diagnosis. In other words, searching an optimal feature subset process is carried out for each feature group. To that end, a wrapper feature selection technique namely genetic algorithm is used with Naive Bayes classifier and support vector machines. The test phase is performed by using 10-fold cross validation. Consequently, accuracy results up to 93.7% with different classifiers and feature selection parameters are shown.