Wrapper Feature Selection Algorithm Research Articles

Detection of chronic kidney disease using binary whale optimization algorithm

Chronic kidney disease (CKD), a medical illness, is characterized by a steady deterioration in kidney function. A disease's ability to be prevented and effectively significantly treated depends on early diagnosis. The addition of filter feature selection to the machine learning algorithm has been done to detect CKD. However, the quality of its feature subset is not optimal. Wrapper feature selection can improve the quality of these feature subsets. Therefore, we proposed wrapper feature selection and binary whale optimization algorithm (BWOA) to enhance the accuracy of early CKD detection. We also make data improvements to improve accuracy, namely the preprocessing process with the median and modus techniques. We used a public dataset of 250 medical records of kidney sufferers and 150 completely healthy people. There are 24 features in this dataset. The test results showed that adding BWOA feature selection can increase accuracy. The proposed method produced an accuracy of 100%. Further research on these methods can be used to develop expert systems for early detection of CKD.

Potential dual inhibitors of Hexokinases and mitochondrial complex I discovered through machine learning approach

Hexokinases (Hks) and mitochondrial complex I (MCI) are involved in the energy metabolism of cells; glycolysis/fermentation and oxidative phosphorylation. Both Hks and MCI are known to play critical roles in either division of metabolic plasticity which enables tumor progression and proliferation in the presence of chemotherapies. Therefore, targeting these enzymes are important in cancer drug resistance. Here, computational models for the prediction of inhibition of Hks were developed based on experimental data and an optimal feature subset that was selected by the Boruta algorithm (a wrapper feature selection algorithm coupled with random forest). Out of the seven models that were explored, a random forest classifier gave the best prediction (GA = 0.84, FNR = 0.12 and AUC = 0.96 for the external dataset). Fragmentation analysis led to the identification of the unique structural scaffolds that characterize hexokinase inhibitors and non-inhibitors. The best Hks inhibition model predicted that 23 molecules out of the 191 dataset of MCI actives (IC50 ≤ 10 µM) that were screened, have more than 60 % probability of exhibiting Hk inhibitory activity. Hence, they are possible dual inhibitors of both targets. Furthermore, the 23 molecules’ core structures are members of the scaffolds that are unique to Hk inhibitors earlier predicted by fragment analysis. The need for dual targeting agents in cancer therapy, particularly in combating cancer drug resistance, highlights the relevance of these findings.

A novel two-stage wrapper feature selection approach based on greedy search for text sentiment classification

Sentiment analysis is a crucial step in obtaining subjective data from online text sources. Nevertheless, the substantial challenge of high dimensionality prevails within text classification. Addressing this, dimension reduction emerges as a valuable approach to enhance the efficacy of classification in the domain of machine learning. The discerning removal of redundant features not only expedites training processes but also bolsters the achievement of accurate classifications. It is worth noting that the effectiveness of distinct feature selection methodologies can be contingent upon the unique attributes inherent in diverse datasets. Within the purview of this investigation, a novel two-stage approach is introduced, characterized by a greedy search-based wrapper feature selection algorithm. The underpinning of this algorithm involves leveraging the outcomes yielded by filter-based feature selection techniques to establish a prioritized sequence for the scrutiny of features within the proposed framework. This strategic sequencing harnesses the cumulative insights from a series of filter-based methodologies, thereby facilitating the curation of feature subsets that underscore pivotal attributes. Nonetheless, it is acknowledged that the greedy selection approach primarily favors features with high-ranking scores, and thus, it may not adequately evaluate the potential of feature combinations that involve low-scoring elements. An extensive experimental inquiry was conducted across widely recognized sentiment analysis datasets to assess the performance of the introduced methodology. The computational findings eloquently demonstrate that the proposed algorithm attains an average accuracy of 96.88% for nine public sentiment datasets and 94.43% for the Humir datasets when coupled with the multinomial Naive Bayes classifier. Furthermore, the experimental outcomes conspicuously establish the superiority of the proposed technique in state-of-the-art studies across the same set of nine sentiment datasets and the Humir datasets.

Evolution of the random subset feature selection algorithm for classification problem

Datasets often include excessive or irrelevant data that affect the performance and complexity of the machine learning model. Feature selection is one of the most effective dimension-reduction technics in datasets to solve the problem. Feature reduction can lead to increased classification accuracy and decreased computational costs. Wrapper feature selection algorithms are a popular and effective category of these methods which considering the learning method's feedback. One of the well-known algorithms in this area is the random subset feature selection algorithm (RSFS). This study proposes an improved version that has higher convergence speed, lower feature selection rate, and higher classification accuracy. Improvements include considering the features' interaction in the selective subsets, continuous evaluation of the selected features to avoid being stuck in the local optima and enhancing the method's evolution phase. To comprehensively evaluate the proposed algorithm, its performance was applied to 20 standard and well-known datasets from public resources and compared with various recent related methods. Experimental results demonstrate that the proposed method reduced classification errors using fewer features compared to other methods, achieving the highest ranking in the Friedman and Wilcoxon rank-sum tests.

Research on Feature Extraction of Underwater Acoustic Target Radiation Noise Based on Machine Learning Algorithm

Underwater acoustic target recognition is a very important technology in the field of underwater acoustics, with great economic and military value. Feature extraction technology for underwater acoustic target radiation noise signals is the key to achieving acoustic target recognition. This study aims at the feature extraction task of acoustic targets and extracts 10 types of 252-dimensional feature vectors from three domains: time domain, frequency domain, and auditory domain. Through 7 machine learning algorithms for classification and recognition experiments, the experimental results show that the recognition performance of the ensemble classifier is much better than that of a single classifier. For different types of features, this study combines three ensemble learning algorithms and feature selection algorithms to select the original 252-dimensional features. The feature selection experiment shows that the wrapper feature selection algorithm has the best effect, and the feature vector dimension can be reduced to 40 dimensions. The recognition accuracy rate is not less than 92.8%, which provides feature extraction guidance for acoustic target recognition based on feature extraction.

Hybrid feature selection based on SLI and genetic algorithm for microarray datasets.

One of the major problems in microarray datasets is the large number of features, which causes the issue of “the curse of dimensionality” when machine learning is applied to these datasets. Feature selection refers to the process of finding optimal feature set by removing irrelevant and redundant features. It has a significant role in pattern recognition, classification, and machine learning. In this study, a new and efficient hybrid feature selection method, called Garank&rand, is presented. The method combines a wrapper feature selection algorithm based on the genetic algorithm (GA) with a proposed filter feature selection method, SLI-γ. In Garank&rand, some initial solutions are built regarding the most relevant features based on SLI-γ, and the remaining ones are only the random features. Eleven high-dimensional and standard datasets were used for the accuracy evaluation of the proposed SLI-γ. Additionally, four high-dimensional well-known datasets of microarray experiments were used to carry out an extensive experimental study for the performance evaluation of Garank&rand. This experimental analysis showed the robustness of the method as well as its ability to obtain highly accurate solutions at the earlier stages of the GA evolutionary process. Finally, the performance of Garank&rand was also compared to the results of GA to highlight its competitiveness and its ability to successfully reduce the original feature set size and execution time.

Mining social media text for disaster resource management using a feature selection based on forest optimization

Resource management is an essential task that needs to be performed by the government or any disaster management agency during natural disasters. During these critical circumstances, people mostly depend upon a social media platform to share and collect information about the situation of the affected localities. The huge volume of real-time data can be useful in disaster assessment, response, and relief activities. We have presented a system which analyzes tweets during natural disasters and categorizes them according to the availability or need for general or medical resources along with their location information (if any) mentioned in the tweets. Several statistical classifiers are applied to show their usefulness for a better solution. Optimal feature representation is the heart of any machine learning based classification model. Here, we have applied a forest optimization-based wrapper feature selection algorithm to improve the classification accuracy. FIRE, SMERP, and CrisisLex dataset are used to evaluate our system and its effectiveness is demonstrated for smooth management of the resources. From the experimentation, it is found that forest optimization algorithm (FOA) wrapped multinomial naive bayes classifier gives an accuracy of 91.41 percent and f-measure of 88.33 percent on the FIRE dataset. The execution time of the model is quite less which will be very helpful for this challenging task.

Binary QUATRE Using Time-varying Transfer Functions

<p>QUasi-Affine TRansformation Evolutionary (QUATRE) is an excellent algorithm that generalized differential evolution algorithm to matrix form. There are two approaches of Binary QUATRE (BQUATRE) algorithm that convert the continuous search space to the binary one. The convergence process of meta-heuristic algorithms can be regarded as two stages: exploration and exploitation. Generally speaking, transfer function plays an important role in the binary metaheuristic algorithms. Therefore, four families of transfer functions are converted into the time-varying form to balance the exploration and exploitation. In particular, a new timevarying formula for the V-shaped, U-shaped and Z-shaped transfer functions is designed in this manuscript. Then, two new approaches of BQUATRE algorithm using the timevarying transfer functions are proposed. In addition, a wrapper feature selection algorithm is designed based on the proposed time-varying BQUATRE. In order to verify the performance of the proposed algorithms, the UCI repository is used in the experiment and the results show that the proposed algorithms are superior to the original BQUATRE and the state-of-the-art algorithms.</p> <p> </p>

BCOVIDOA: A Novel Binary Coronavirus Disease Optimization Algorithm for Feature Selection

The increased use of digital tools such as smart phones, Internet of Things devices, cameras, and microphones, has led to the produuction of big data. Large data dimensionality, redundancy, and irrelevance are inherent challenging problems when it comes to big data. Feature selection is a necessary process to select the optimal subset of features when addressing such problems. In this paper, the authors propose a novel Binary Coronavirus Disease Optimization Algorithm (BCOVIDOA) for feature selection, where the Coronavirus Disease Optimization Algorithm (COVIDOA) is a new optimization technique that mimics the replication mechanism used by Coronavirus when hijacking human cells. The performance of the proposed algorithm is evaluated using twenty-six standard benchmark datasets from UCI Repository. The results are compared with nine recent wrapper feature selection algorithms. The experimental results demonstrate that the proposed BCOVIDOA significantly outperforms the existing algorithms in terms of accuracy, best cost, the average cost (AVG), standard deviation (STD), and size of selected features. Additionally, the Wilcoxon rank-sum test is calculated to prove the statistical significance of the results.

Recursive elimination–election algorithms for wrapper feature selection

For classification tasks in machine learning, this paper proposes a brand-new wrapper feature selection algorithm prototype named recursive elimination–election (REE), which is conceived in a simple but exquisite structure inspired by the recursion technique in computer science. Prevalent metaheuristic methods such as differential evolution (DE), particle swarm optimization (PSO), etc., from evolutionary computation (EC) and swarm intelligence (SI) communities have recently been widely applied to feature selection research, but suffer from severe drawbacks including but not limited to low-efficient binary representation transformation, poor population diversity, excessive control parameter adjustments and sophisticated mechanisms. Instead, REE is organically constructed with an ordinary subset representation of feature indexes, simple operators, getting rid of extra control parameters. Specifically, REE is assembled of two basic recursive sub-algorithms, i.e., recursive random bisection elimination (RRBE) and recursive greedy binary election (RGBE), which somewhat embody the idea of “divide-and-conquer”. By inspecting smaller and potential feature subsets in recursive ways, better subsets are returned automatically. A comprehensive experimental study was conducted on 14 UCI and ASU benchmark datasets with feature sizes ranging from dozens to thousands by using REE together with 6 state-of-the-art metaheuristic algorithms for comparison. The results show that the proposed REE has competitive search ability for feature selection problems, and it is especially prominent in handling high-dimensional datasets. Therefore, REE is promising to become a wrapper feature selection search paradigm with low solution cost and high efficiency.

A Machine Learning Approach for Predicting Sustained Remission in Rheumatoid Arthritis Patients on Biologic Agents.

In this longitudinal study, patients with RA who started a biological disease-modifying antirheumatic drug (bDMARD) in a tertiary care center were analyzed. Demographic and clinical characteristics were collected at treatment baseline, 12-month, and 24-month follow-up. A wrapper feature selection algorithm was used to determine an attribute core set. Four different ML algorithms, namely, LR, random forest, K-nearest neighbors, and extreme gradient boosting, were then trained and validated with 10-fold cross-validation to predict 24-month sustained DAS28 (Disease Activity Score on 28 joints) remission. The performances of the algorithms were then compared assessing accuracy, precision, and recall. Our analysis included 367 patients (female 323/367, 88%) with mean age ± SD of 53.7 ± 12.5 years at bDMARD baseline. Sustained DAS28 remission was achieved by 175 (47.2%) of 367 patients. The attribute core set used to train algorithms included acute phase reactant levels, Clinical Disease Activity Index, Health Assessment Questionnaire-Disability Index, as well as several clinical characteristics. Extreme gradient boosting showed the best performance (accuracy, 72.7%; precision, 73.2%; recall, 68.1%), outperforming random forest (accuracy, 65.9%; precision, 65.6%; recall, 59.3%), LR (accuracy, 64.9%; precision, 62.6%; recall, 61.9%), and K-nearest neighbors (accuracy, 63%; precision, 61.5%; recall, 54.8%). We showed that ML models can be used to predict sustained remission in RA patients on bDMARDs. Furthermore, our method only relies on a few easy-to-collect patient attributes. Our results are promising but need to be tested on longitudinal cohort studies.

The crash injury severity prediction of traffic accident using an improved wrappers feature selection algorithm

The crash injury severity prediction of traffic accident is the prerequisite for driving decisions under dangerous conditions. This paper puts forward a new wrapper feature selection algorithm to improve the prediction accuracy of the injury severity. Firstly, based on Gini index and Mutual information, the importance of factors is analyzed by the comprehensive evaluation criteria, according to the importance score, the factors were selected by sequential backward selection. Secondly, three models, that is RF (Random Forest), C4.5 (C4.5 Decision Tree) and SVM (Support Vector Machine) are established to verify the prediction accuracy of selected subsets. In order to verify the proposed algorithm, based on the FARS(Fatality Analysis Reporting System) in the NHTSA(National Highway Traffic Safety Administration), a total of 6,295 rear-end accidents including 23 candidate factors were screened, the results demonstrate that the accuracy of the three models increases with the elimination of redundant variables, among them the SVM model has better predictive performance, the significant factors are obtained which include seat belts, airbag, the relative speed, vehicle weight, etc. finally the sensitivity analysis of significant factors is implemented and some suggestions are given to improve traffic safety.

Hybrid Machine Learning Classifiers for Indoor User Localization Problem

Wi-Fi technology is now everywhere either inside or outside buildings. Using Wi-fi technology introduces an indoor localization service(s) (ILS). Determining indoor user location is a hard and complex problem. Several applications highlight the importance of indoor user localization such as disaster management, health care zones, Internet of Things applications (IoT), and public settlement planning. The measurements of Wi-Fi signal strength (i.e., Received Signal Strength Indicator (RSSI)) can be used to determine indoor user location. In this paper, we proposed a hybrid model between a wrapper feature selection algorithm and machine learning classifiers to determine indoor user location. We employed the Minimum Redundancy Maximum Relevance (mRMR) algorithm as a feature selection to select the most active access point (AP) based on RSSI values. Six different machine learning classifiers were used in this work (i.e., Decision Tree (DT), Support Vector Machine (SVM), k-nearest neighbors (kNN), Linear Discriminant Analysis (LDA), Ensemble-Bagged Tree (EBaT), and Ensemble Boosted Tree (EBoT)). We examined all classifiers on a public dataset obtained from UCI repository. The obtained results show that EBoT outperforms all other classifiers based on accuracy value.

Intrusion Detection using Deep Learning Long Short-term Memory with Wrapper Feature Selection Method

Recently, many companies move to use cloud com-puting systems to enhance their performance and productivity. Using these cloud computing systems allows the execution of applications, data, and infrastructures on cloud platforms (i.e., online), which increase the number of attacks on such systems. As a resulting, building robust Intrusion detection systems (IDS) is needed. The main goal of IDS is to detect normal and abnormal network traffic. In this paper, we propose a hybrid approach between an Enhanced Binary Genetic Algorithms (EBGA) as a wrapper feature selection (FS) algorithm and Long Short-Term Memory (LSTM). A novel injection method to prevent premature convergence of the GA is proposed in this paper. An intelligent k-means algorithm is employed to examine the solution distribution in the search space. Once 80% of the solutions belong to one cluster, an injection method (i.e., add new solutions) is used to redistribute the solutions over the search space. EBGA will reduce the search space as a preprocessing step, while LSTM works as a binary classification method. UNSW-NB15, a real-world public dataset, is used in this work to evaluate the proposed system. The obtained results show the ability of feature selection method to enhance the overall performance of LSTM.

An Improved Binary Grey-Wolf Optimizer With Simulated Annealing for Feature Selection

This paper proposes improvements to the binary grey-wolf optimizer (BGWO) to solve the feature selection (FS) problem associated with high data dimensionality, irrelevant, noisy, and redundant data that will then allow machine learning algorithms to attain better classification/clustering accuracy in less training time. We propose three variants of BGWO in addition to the standard variant, applying different transfer functions to tackle the FS problem. Because BGWO generates continuous values and FS needs discrete values, a number of V-shaped, S-shaped, and U-shaped transfer functions were investigated for incorporation with BGWO to convert their continuous values to binary. After investigation, we note that the performance of BGWO is affected by the selection of the transfer function. Then, in the first variant, we look to reduce the local minima problem by integrating an exploration capability to update the position of the grey wolf randomly within the search space with a certain probability; this variant was abbreviated as IBGWO. Consequently, a novel mutation strategy is proposed to select a number of the worst grey wolves in the population which are updated toward the best solution and randomly within the search space based on a certain probability to determine if the update is either toward the best or randomly. The number of the worst grey wolf selected by this strategy is linearly increased with the iteration. Finally, this strategy is combined with IBGWO to produce the second variant of BGWO that was abbreviated as LIBGWO. In the last variant, simulated annealing (SA) was integrated with LIBGWO to search around the best-so-far solution at the end of each iteration in order to identify better solutions. The performance of the proposed variants was validated on 32 datasets taken from the UCI repository and compared with six wrapper feature selection methods. The experiments show the superiority of the proposed improved variants in producing better classification accuracy than the other selected wrapper feature selection algorithms.

Microblog User Geolocation by Extracting Local Words Based on Word Clustering and Wrapper Feature Selection

Existing methods always rely on statistical features to extract local words for microblog user geolocation. There are many non-local words in extracted words, which makes geolocation accuracy lower. Considering the statistical and semantic features of local words, this paper proposes a microblog user geolocation method by extracting local words based on word clustering and wrapper feature selection. First, ordinary words without positional indications are initially filtered based on statistical features. Second, a word clustering algorithm based on word vectors is proposed. The remaining semantically similar words are clustered together based on the distance of word vectors with semantic meanings. Next, a wrapper feature selection algorithm based on sequential backward subset search is proposed. The cluster subset with the best geolocation effect is selected. Words in selected cluster subset are extracted as local words. Finally, the Naive Bayes classifier is trained based on local words to geolocate the microblog user. The proposed method is validated based on two different types of microblog data - Twitter and Weibo. The results show that the proposed method outperforms existing two typical methods based on statistical features in terms of accuracy, precision, recall, and F1-score.

Modified Cuckoo Search-Support Vector Machine (MCS-SVM) Gene Selection and Classification for Autism Spectrum Disorder (ASD) Gene Expression

Autism Spectrum Disorder (ASD) is a neuro developmental disorder characterized by weakened social skills, impaired verbal and non-verbal interaction, and repeated behavior. ASD has increased in the past few years and the root cause of the symptom cannot yet be determined. In ASD with gene expression is analyzed by classification methods. For the selection of genes in ASD, statistical philtres and a wrapper-based Geometric Binary Particle Swarm Optimization-Support Vector Machine (GBPSO-SVM) algorithm have recently been implemented. However GBPSO has provides lesser accuracy, if the dataset samples are large and it cannot directly apply to multiple output systems. To overcome this issue, Modified Cuckoo Search-Support Vector Machine (MCS-SVM) based wrapper feature selection algorithm is proposed which improves the accuracy of the classifier in ASD. This work consists of three major steps, (i) preprocessing, (ii) gene selection, and (iii) classification. Firstly, preprocessing is performed by mean or median ratios close to unity was removed from original gene dataset; based on this samples are reduced from 54,613 to 9454. Secondly, gene selection is performed by using statistical filters and wrapper algorithm. Statistical filters methods like Wilcox on Rank Sum test (WRS), Class Correlation (COR) function and Two-sample T-test (TT) were applied in parallel to a ten-fold cross validation range of the most discriminatory genes. In the wrapper algorithm, Modified Cuckoo Search (MCS) is also proposed to gene selection. This step decreases the number of genes of the dataset by removing genes. Finally, SVM classifier combined forms of gene subsets for grading. The autism microarray dataset used in the analysis was downloaded from the benchmark public repository Gene Expression Omnibus (GEO) (National Center for Biotechnology Information (NCBI)). The classification methods are measured in terms of the metrics like precision, recall, f-measure and accuracy. Proposed MCS-SVM classifier achieves highest accuracy when compared Linear Regression (LR), and GBPSO-SVM classifiers.

Ensemble of heterogeneous classifiers for diagnosis and prediction of coronary artery disease with reduced feature subset

Background and Objective: Coronary artery disease (CAD) is considered one of the most prominent health issues causing high mortality in the world population. Hence, earlier diagnosis and prediction of CAD is essential for the proper medication of patients. The objective of this study is to develop a machine learning algorithm that will help in accurate diagnosis of CAD.Methods: In this paper, we have proposed a novel heterogeneous ensemble method combining three base classifiers viz., K-Nearest Neighbour, Random Forest, and Support Vector Machine for effective diagnosis of CAD. The results of base classifiers are combined using ensemble voting technique based on average-voting (AVEn), majority-voting (MVEn), and weighted-average voting (WAVEn) for prediction of CAD. The random forest-based Boruta wrapper feature selection algorithm and feature importance of SVM are used for relevant feature selection based on attribute importance and rank.Results: The proposed ensemble algorithm is developed using 5 features selected based on the feature importance and the performance of the algorithm is evaluated using the Z-Alizadeh Sani dataset. Further, the dataset is balanced using Synthetic Minority Over-sampling Technique and its performance is evaluated. The result analysis shows that the WAVEn algorithm achieves better classification accuracy, sensitivity, specificity and precision of 98.97%, 100%, 96.3% and 98.3% respectively for the original dataset. The WAVEn algorithm applied on the balanced dataset achieves 100% accuracy, sensitivity, specificity and precision in diagnosing CAD. To the best of author’s knowledge, the accuracy achieved by WAVEn is the highest accuracy when compared with the state-of-the-art algorithms in the literature for both original and balanced dataset.Conclusions: The statistical results prove the robustness of the WAVEn algorithm in reliably discriminating the CAD patients from healthy ones with high precision, and therefore it can be used for developing a decision support system for diagnosing CAD at an early stage.

A multi-objective algorithm for multi-label filter feature selection problem

Feature selection is an important data preprocessing method before classification. Multi-objective optimization algorithms have been proved an effective way to solve feature selection problems. However, there are few studies on multi-objective optimization feature selection methods for multi-label data. In this paper, a multi-objective multi-label filter feature selection algorithm based on two particle swarms (MOMFS) is proposed. We use mutual information to measure the relevance between features and label sets, and the redundancy between features, which are taken as two objectives. In order to avoid Particle Swarm Optimization (PSO) from falling into the local optimum and obtaining a false Pareto front, we employ two swarms to optimize the two objectives separately and propose an improved hybrid topology based on particle’s fitness value. Furthermore, an archive maintenance strategy is introduced to maintain the distribution of archive. In order to study the effectiveness of the proposed algorithm, we select five multi-label evaluation criteria and perform experiments on seven multi-label data sets. MOMFS is compared with classic single-objective multi-label feature selection algorithms, multi-objective filter and wrapper feature selection algorithms. The experimental results show that MOMFS can effectively reduce the multi-label data dimension and perform better than other approaches on five evaluation criteria.

A feature selection algorithm for intrusion detection system based on Pigeon Inspired Optimizer

Feature selection plays a vital role in building machine learning models. Irrelevant features in data affect the accuracy of the model and increase the training time needed to build the model. Feature selection is an important process to build Intrusion Detection System (IDS). In this paper, a wrapper feature selection algorithm for IDS is proposed. This algorithm uses the pigeon inspired optimizer to utilize the selection process. A new method to binarize a continuous pigeon inspired optimizer is proposed and compared to the traditional way for binarizing continuous swarm intelligent algorithms. The proposed algorithm was evaluated using three popular datasets: KDDCUP99, NLS-KDD and UNSW-NB15. The proposed algorithm outperformed several feature selection algorithms from state-of-the-art related works in terms of TPR, FPR, accuracy, and F-score. Also, the proposed cosine similarity method for binarizing the algorithm has a faster convergence than the sigmoid method.