Wrapper Feature Selection Approach Research Articles

Breast Cancer Diagnosis Using an Efficient CAD System Based on Multiple Classifiers.

Breast cancer is one of the major health issues across the world. In this study, a new computer-aided detection (CAD) system is introduced. First, the mammogram images were enhanced to increase the contrast. Second, the pectoral muscle was eliminated and the breast was suppressed from the mammogram. Afterward, some statistical features were extracted. Next, k-nearest neighbor (k-NN) and decision trees classifiers were used to classify the normal and abnormal lesions. Moreover, multiple classifier systems (MCS) was constructed as it usually improves the classification results. The MCS has two structures, cascaded and parallel structures. Finally, two wrapper feature selection (FS) approaches were applied to identify those features, which influence classification accuracy. The two data sets (1) the mammographic image analysis society digital mammogram database (MIAS) and (2) the digital mammography dream challenge were combined together to test the CAD system proposed. The highest accuracy achieved with the proposed CAD system before FS was 99.7% using the Adaboosting of the J48 decision tree classifiers. The highest accuracy after FS was 100%, which was achieved with k-NN classifier. Moreover, the area under the curve (AUC) of the receiver operating characteristic (ROC) curve was equal to 1.0. The results showed that the proposed CAD system was able to accurately classify normal and abnormal lesions in mammogram samples.

Optimal feature selection for islanding detection in distributed generation

The integration of the distributed power generation into a distribution system comes with several system problems. One of the teething problems related to system protection is islanding detection. Various anti-islanding techniques based on feature evaluation were proposed in the recent past. However, they overlook the need for justifying the selection of a particular detection feature among all the possible measures. In this study, a wrapper feature selection approach is proposed where a modified multi-objective differential evolution algorithm is coupled with a kernel-based extreme learning machine classifier. To select the optimum features, five standard objective functions have been considered, such as dependability, security, accuracy, F-measure, and the number of features. About 1864 cases have been generated from the designed IEEE 13 bus system to extract the sensitive features. IEEE 1547 standards have been considered while designing and testing the IEEE 13 bus system against islanding. The selected optimal features detect the islanded condition decisively for both synchronous and inverter-based distributed generators. The features also validate their performance under noisy environment accurately with lesser computational time.

An efficient binary Salp Swarm Algorithm with crossover scheme for feature selection problems

Searching for the (near) optimal subset of features is a challenging problem in the process of feature selection (FS). In the literature, Swarm Intelligence (SI) algorithms show superior performance in solving this problem. This motivated our attempts to test the performance of the newly proposed Salp Swarm Algorithm (SSA) in this area. As such, two new wrapper FS approaches that use SSA as the search strategy are proposed. In the first approach, eight transfer functions are employed to convert the continuous version of SSA to binary. In the second approach, the crossover operator is used in addition to the transfer functions to replace the average operator and enhance the exploratory behavior of the algorithm. The proposed approaches are benchmarked on 22 well-known UCI datasets and the results are compared with 5 FS methods: Binary Grey Wolf Optimizer (BGWO), Binary Gravitational Search Algorithms (BGSA), Binary Bat Algorithm (BBA), Binary Particle Swarm Optimization (BPSO), and Genetic Algorithm (GA). The paper also considers an extensive study of the parameter setting for the proposed technique. From the results, it is observed that the proposed approach significantly outperforms others on around 90% of the datasets.

Improving Detection Rate of the Network Intrusion Detection System Based on Wrapper Feature Selection Approach

Regarding the security of computer systems, the intrusion detection systems (IDSs) are essential components for the detection of attacks at the early stage. They monitor and analyze network traffics, looking for abnormal behaviors or attack signatures to detect intrusions in real time. A major drawback of the IDS is their inability to provide adequate sensitivity and accuracy, coupled with their failure in processing enormous data. The issue of classification time is greatly reduced with the IDS through feature selection. In this paper, a new feature selection algorithm based on Firefly Algorithm (FA) is proposed. In addition, the naive bayesian classifier is used to discriminate attack behaviour from normal behaviour in the network traffic. The FA selects the discriminating features from NSL-KDD dataset. The performance of the IDS in the detection of attacks was enhanced by the proposed model and compare with other models.

A hybrid-multi filter-wrapper framework to identify run-time behaviour for fast malware detection

Malicious software (malware) constitute one of the most pressing cyber threats intended to cripple critical infrastructure, render infected systems unusable, permanently erase data from storage systems. The number of malware has skyrocketed through the use of enormous malware development toolkit. Run-time analysis has recently been used to overcome the limitations of current detection engines due to code obfuscation techniques such as polymorphism and metamorphism. However run-time approaches face a critical challenge of processing a large number of run-time malware features which may fail to provide real time protection. In this paper, we propose a hybrid framework by using more than one complementary filters and a wrapper feature selection approach to identify the most significant run-time behavioural characteristics of malware. The novelty of the proposed framework is that it exploits the complementary characteristics of within-filters and between wrapper-filters by hybridizing discriminant, minimum redundant, and maximum relevant filters with the wrapper approach to integrate the knowledge from the intrinsic characteristics of the run-time behaviour of malware obtained by the filters into the wrapper selection process. We have verified the performance of the proposed approach through extensive experiments using large real malware datasets. The results of the experiments show that the proposed framework finds the most significant run-time characteristics of malware. When these are used in the detection engine, the computational performances and detection accuracies are also improved up to 99.499% compared to any existing techniques.

Whale optimization approaches for wrapper feature selection

Classification accuracy highly dependents on the nature of the features in a dataset which may contain irrelevant or redundant data. The main aim of feature selection is to eliminate these types of features to enhance the classification accuracy. The wrapper feature selection model works on the feature set to reduce the number of features and improve the classification accuracy simultaneously. In this work, a new wrapper feature selection approach is proposed based on Whale Optimization Algorithm (WOA). WOA is a newly proposed algorithm that has not been systematically applied to feature selection problems yet. Two binary variants of the WOA algorithm are proposed to search the optimal feature subsets for classification purposes. In the first one, we aim to study the influence of using the Tournament and Roulette Wheel selection mechanisms instead of using a random operator in the searching process. In the second approach, crossover and mutation operators are used to enhance the exploitation of the WOA algorithm. The proposed methods are tested on standard benchmark datasets and then compared to three algorithms such as Particle Swarm Optimization (PSO), Genetic Algorithm (GA), the Ant Lion Optimizer (ALO), and five standard filter feature selection methods. The paper also considers an extensive study of the parameter setting for the proposed technique. The results show the efficiency of the proposed approaches in searching for the optimal feature subsets.

A GA-LR wrapper approach for feature selection in network intrusion detection

Intrusions constitute one of the main issues in computer network security. Through malicious actions, hackers can have unauthorised access that compromises the integrity, the confidentiality, and the availability of resources or services. Intrusion detection systems (IDSs) have been developed to monitor and filter network activities by identifying attacks and alerting network administrators. Different IDS approaches have emerged using data mining, machine learning, statistical analysis, and artificial intelligence techniques such as genetic algorithms, artificial neural networks, fuzzy logic, swarm intelligence, etc. Due to the high dimensionality of the exchanged data, applying those techniques will be extremely time consuming. Feature selection is needed to select the optimal subset of features that represents the entire dataset to increase the accuracy and the classification performance of the IDS. In this work, we apply a wrapper approach based on a genetic algorithm as a search strategy and logistic regression as a learning algorithm for network intrusion detection systems to select the best subset of features. The experiment will be conducted on the KDD99 dataset and the UNSW-NB15 dataset. Three different decision tree classifiers are used to measure the performance of the selected subsets of features. The obtained results are compared with other feature selection approaches to verify the efficiency of our proposed approach.

Resting State Effective Connectivity Allows Auditory Hallucination Discrimination

Hallucinations are elusive phenomena that have been associated with psychotic behavior, but that have a high prevalence in healthy population. Some generative mechanisms of Auditory Hallucinations (AH) have been proposed in the literature, but so far empirical evidence is scarce. The most widely accepted generative mechanism hypothesis nowadays consists in the faulty workings of a network of brain areas including the emotional control, the audio and language processing, and the inhibition and self-attribution of the signals in the auditive cortex. In this paper, we consider two methods to analyze resting state fMRI (rs-fMRI) data, in order to measure effective connections between the brain regions involved in the AH generation process. These measures are the Dynamic Causal Modeling (DCM) cross-covariance function (CCF) coefficients, and the partially directed coherence (PDC) coefficients derived from Granger Causality (GC) analysis. Effective connectivity measures are treated as input classifier features to assess their significance by means of cross-validation classification accuracy results in a wrapper feature selection approach. Experimental results using Support Vector Machine (SVM) classifiers on an rs-fMRI dataset of schizophrenia patients with and without a history of AH confirm that the main regions identified in the AH generative mechanism hypothesis have significant effective connection values, under both DCM and PDC evaluation.

A Novel Wrapper Approach for Feature Selection in Object-Based Image Classification Using Polygon-Based Cross-Validation

Feature selection is becoming a major component of object-based classification as numerous features of segmented object become available. Although common feature selection methods in object-based classification are acknowledged, wrapper-based methods remain an issue due to the diversity of accuracy assessment methods. This letter presents a new wrapper approach using polygon-based cross validation (CV) to overcome possible bias of object-based accuracy assessment for object-based classification. The new method is a two-step wrapper-based feature selection that involves the integration of: 1) feature importance rank using gain ratio and 2) feature subset evaluation using a polygon-based tenfold CV within a support vector machine (SVM) classifier. Several high-resolution images, including both unmanned aerial vehicle images and ISPRS (International Society for Photogrammetry and Remote Sensing) benchmark test data, were used to test the proposed method. Results show that, with the proposed polygon-based CV SVM wrapper, the mean overall accuracy is significantly higher than with an object-based CV SVM wrapper. Furthermore, the proposed method shows potential for comprehensively considering all types of features instead of only spectral features.

LAWRA: a layered wrapper feature selection approach for network attack detection

AbstractThe feature selection phase in network attack detection is mostly classifier based, while clustering techniques are used for labeling and creating compact training datasets. Because clustering finds natural groupings in the data, in this paper, a clustering‐based layered wrapper feature selection approach, LAWRA, has been proposed for selecting appropriate features for attack detection. The existing layered feature selection approaches in attack detection are unable to give results with high precision and recall because of the dependence on classifier accuracy, fitness value, and so on. Hence, in this paper, LAWRA uses external cluster validity indices, F‐measure, and Fowlkes–Mallows index, for feature selection. The two indices are the harmonic and geometric mean of precision and recall, respectively. Each index identifies features that give high precision and high recall of the attack detection algorithm. The first layer of LAWRA identifies the feature subset that best distinguishes between normal and attack instances and the second layer identifies the best cooperating features using cooperative game theory. Experiments have been conducted on NSL‐KDD dataset, and LAWRA has been compared with the existing approaches using different classifiers. The results show that LAWRA gives better overall accuracy and F‐measure value than the other approaches. Copyright © 2015 John Wiley & Sons, Ltd.

Hybrid Feature Selection Based Weighted Least Squares Twin Support Vector Machine Approach for Diagnosing Breast Cancer, Hepatitis, and Diabetes

There is a necessity for analysis of a large amount of data in many fields such as healthcare, business, industries, and agriculture. Therefore, the need of the feature selection (FS) technique for the researchers is quite evident in many fields of science, especially in computer science. Furthermore, an effective FS technique that is best suited to a particular learning algorithm is of great help for the researchers. Hence, this paper proposes a hybrid feature selection (HFS) based efficient disease diagnostic model for Breast Cancer, Hepatitis, and Diabetes. A HFS is an efficient method that combines the positive aspects of both Filter and Wrapper FS approaches. The proposed model adopts weighted least squares twin support vector machine (WLSTSVM) as a classification approach, sequential forward selection (SFS) as a search strategy, and correlation feature selection (CFS) to evaluate the importance of each feature. This model not only selects relevant feature subset but also efficiently deals with the data imbalance problem. The effectiveness of the HFS based WLSTSVM approach is examined on three well-known disease datasets taken from UCI repository with the help of predictive accuracy, sensitivity, specificity, and geometric mean. The experiment confirms that our proposed HFS based WLSTSVM disease diagnostic model can result in positive outcomes.

A wrapper approach for feature selection based on Bat Algorithm and Optimum-Path Forest

Besides optimizing classifier predictive performance and addressing the curse of the dimensionality problem, feature selection techniques support a classification model as simple as possible. In this paper, we present a wrapper feature selection approach based on Bat Algorithm (BA) and Optimum-Path Forest (OPF), in which we model the problem of feature selection as an binary-based optimization technique, guided by BA using the OPF accuracy over a validating set as the fitness function to be maximized. Moreover, we present a methodology to better estimate the quality of the reduced feature set. Experiments conducted over six public datasets demonstrated that the proposed approach provides statistically significant more compact sets and, in some cases, it can indeed improve the classification effectiveness.

Customer attrition in retailing: An application of Multivariate Adaptive Regression Splines

The profit resulting from customer relationship is essential to ensure companies viability, so an improvement in customer retention is crucial for competitiveness. As such, companies have recognized the importance of customer centered strategies and consequently customer relationship management (CRM) is often at the core of their strategic plans. In this context, a priori knowledge about the risk of a given customer to mitigate or even end the relationship with the provider is valuable information that allows companies to take preventive measures to avoid defection. This paper proposes a model to predict partial defection, using two classification techniques: Logistic regression and Multivariate Adaptive Regression Splines (MARS). The main objective is to compare the performance of MARS with Logistic regression in modeling customer attrition. This paper considers the general form of Logistic regression and Logistic regression combined with a wrapper feature selection approach, such as stepwise approach. The empirical results showed that MARS performs better than Logistic regression when variable selection procedures are not used. However, MARS loses its superiority when Logistic regression is conducted with stepwise feature selection.

Development of machine learning‐based real time scheduling systems: using ensemble based on wrapper feature selection approach

There are two items that significantly enhance the generalisation ability (i.e. classification accuracy) of machine learning‐based classifiers: feature selection (including parameter optimisation) and an ensemble of the classifiers. Accordingly, the objective in this study is to develop an ensemble of classifiers based on a genetic algorithm (GA) wrapper feature selection approach for real time scheduling (RTS). The proposed approach can better enhance the generalisation ability of the RTS knowledge base (i.e. classifier) in comparison with three classical machine learning‐based classifier RTS systems, including the GA‐based wrapper feature selection mechanism, in terms of the prediction accuracy of 10‐fold cross validation as measured according to all the performance criteria. The proposed ensemble classifier RTS also provides better system performance than the three machine learning‐based RTS systems, including the GA‐based wrapper feature selection mechanism and heuristic dispatching rules, under all the performance criteria, over a long period in a flexible manufacturing system (FMS) case study.

English

Recently, Gene expression profiling by microarray technique has been effectively utilized for classification and diagnostic guess- ing of cancer nodules in the field of medical sciences. But the techniques used for cancer classification is still in its lower level. There are various drawbacks in the existing classification techniques such as low testing accuracy, high training time, unreliability, etc. Moreover, mi- croarray data consists of a high degree of noise. Gene ranking techniques such as T-Score, ANOVA, etc are later proposed to overcome those problems. But those approaches will sometimes wrongly predict the rank when large database is used. To overcome these issues, this paper mainly focuses on the development of an effective feature selection and classification technique for microarray gene expression cancer diagnosis for provide significant accuracy, reliability and less error rate. In this paper, Wrapper feature selection approach called the GA-FSVML approach is used for the effective feature selection of genes. In FSVML, the RBF kernel function in SVM is trained using modi- fied Levenberg Marquadt algorithm. This approach proposes a Fast SVM Learning (FSVML) technique for the classification tasks. The ex- periment is performed on lymphoma data set and the result shows the better accuracy of the proposed FSVML with GA-FSVML classifica- tion approach when compared to the standard existing approaches.

Identifying user preferences with Wrapper-based Decision Trees

Wrapper feature selection approaches are widely used to choose a small subset of relevant features from a dataset. However, Wrappers suffer from the fact that they only use a single classifier. The downside to this is that each classifier will have its own biases and will therefore select very different features. To overcome the biases of individual classifiers, we propose a new data mining method called Wrapper-based Decision Trees (WDT). The WDT method uses multiple classifiers for selecting relevant features and decision trees to visualize relationships among the selected features. We use the WDT to investigate the influences of the levels of computer experience on users’ preferences for the design of search engines. The benefit of using WDT lies within the fact that it can uncover the most accurate set of relevant features to help differentiate the preferences of users with diverse levels of computer experience. The results indicate that the users with varied levels of computer experiences have different preferences regarding the following features: the number of icons, the arrangement of search results, and the presentation of error messages. Such findings can be used to develop personalized search engines to accommodate users’ different levels of computer experience.

A new wrapper feature selection approach using neural network

This paper presents a new feature selection (FS) algorithm based on the wrapper approach using neural networks (NNs). The vital aspect of this algorithm is the automatic determination of NN architectures during the FS process. Our algorithm uses a constructive approach involving correlation information in selecting features and determining NN architectures. We call this algorithm as constructive approach for FS (CAFS). The aim of using correlation information in CAFS is to encourage the search strategy for selecting less correlated (distinct) features if they enhance accuracy of NNs. Such an encouragement will reduce redundancy of information resulting in compact NN architectures. We evaluate the performance of CAFS on eight benchmark classification problems. The experimental results show the essence of CAFS in selecting features with compact NN architectures.

Orthogonal linear discriminant analysis and feature selection for micro-array data classification

A novel method for micro-array data classification based on orthogonal linear discriminant analysis (ODA), sequential forward floating selection (SFFS) and support vector machine (SVM) is here proposed. In this paper, in order to avoid the constraint that the dimension of the ODA subspace is bounded by the number of classes, to increase the dimension of the subspace and to improve the accuracy, we combine the “original” features to obtain new features. We combine the features in groups of K, each new feature f is obtained by the projection that maps the K-dimensional feature space to a single dimension. A feature selection algorithm is applied to select the most relevant features. Since the new features space has only few hundreds of features an exhaustive wrapper feature selection approach is used to select the set of relevant features. Finally a radial basis function SVM is trained using these features. The obtained results are very encouraging, they improve the average predictive accuracy obtained using standard feature transform techniques. Particularly interesting are the results on a breast cancer dataset, to the best of our knowledge the proposed method is the first method that, using the genes information, permits to determine with high accuracy if a person might benefit from adjuvant chemotherapy.

Feature Subset Selection Problem using Wrapper Approach in Supervised Learning

Feature subset selection is of immense importance in the field of data mining. The increased dimensionality of data makes testing and training of general classification method difficult. Mining on the reduced set of attributes reduces computation time and also helps to make the patterns easier to understand. In this paper a wrapper approach for feature selection is proposed. As a part of feature selection step we used wrapper approach with Genetic algorithm as random search technique for subset generation ,wrapped with different classifiers/ induction algorithm namely decision tree C4.5, NaiveBayes, Bayes networks and Radial basis function as subset evaluating mechanism on four standard datasets namely Pima Indians Diabetes Dataset, Breast Cancer, Heart Stat log and Wisconsin Breast Cancer. Further the relevant attributes identified by proposed wrapper are validated using classifiers. Experimental results illustrate, employing feature subset selection using proposed wrapper approach has enhanced classification accuracy.

Data mining to estimate broiler mortality when exposed to heat wave

Heat waves usually result in losses of animal production since they are exposed to thermal stress inducing an increase in mortality and consequent economical losses. Animal science and meteorological databases from the last years contain enough data in the poultry production business to allow the modeling of mortality losses due to heat wave incidence. This research analyzes a database of broiler production associated to climatic data, using data mining techniques such as attribute selection and data classification (decision tree) to model the impact of heat wave incidence on broiler mortality. The temperature and humidity index (THI) was used for screening environmental data. The data mining techniques allowed the development of three comprehensible models for estimating specifically high mortality during broiler production. Two models yielded a classification accuracy of 89.3% by using Principal Component Analysis (PCA) and Wrapper feature selection approaches. Both models obtained a class precision of 0.83 for classifying high mortality. When the feature selection was made by the domain experts, the model accuracy reached 85.7%, while the class precision of high mortality was 0.76. Meteorological data and the calculated THI from meteorological stations were helpful to select the range of harmful environmental conditions for broilers 29 and 42 days old. The data mining techniques were useful for building animal production models.