Wrapper Feature Selection Method Research Articles

Optimizing prediction accuracy for early recurrent lumbar disc herniation with a directional mutation-guided SVM model

Percutaneous endoscopic lumbar discectomy (PELD) is one of the main means of minimally invasive spinal surgery, and is an effective means of treating lumbar disc herniation, but its early recurrence is still difficult to predict. With the development of machine learning technology, the auxiliary model based on the prediction of early recurrent lumbar disc herniation (rLDH) and the identification of causative risk factors have become urgent problems in current research. However, the screening ability of current models for key factors affecting the prediction of rLDH, as well as their predictive ability, needs to be improved. Therefore, this paper presents a classification model that utilizes wrapper feature selection, developed through the integration of an enhanced bat algorithm (BDGBA) and support vector machine (SVM). Among them, BDGBA increases the population diversity and improves the population quality by introducing directional mutation strategy and guidance-based strategy, which in turn allows the model to secure better subsets of features. Furthermore, SVM serves as the classifier for the wrapper feature selection method, with its classification prediction results acting as a fitness function for the feature subset. In the proposed prediction method, BDGBA is used as an optimizer for feature subset filtering and as an objective function for feature subset evaluation based on the classification results of the support vector machine, which improves the interpretability and prediction accuracy of the model. In order to verify the performance of the proposed method, this paper proves the performance of the model through global optimization experiments and prediction experiments on real data sets. The accuracy of the proposed rLDH prediction model is 93.49% and sensitivity is 88.33%. The experimental results show that Level of herniated disk, Modic change, Disk height, Disk length, and Disk width are the key factors for predicting rLDH, and the proposed method is an effective auxiliary diagnosis method.

Deep Curious Feature Selection: A Recurrent, Intrinsic-Reward Reinforcement Learning Approach to Feature Selection

Feature selection (FS) is an important step in the process of building machine learning-based models. The goal of the feature selection step is to find a small subset of features that will provide good prediction results by removing noisy, irrelevant, or repetitive features. Commonly used Wrapper methods use a machine learning model as a black box and its performance as the goal function for evaluating different features' subsets and selecting the best one. To avoid examining all possible subsets (an NP-hard problem), search algorithms are used to find the subsets to be examined, in a heuristic way. As exhaustive search methods are computationally complicated, most methods use simple and greedy search methods that yield only locally optimal results and are not sensitive to possible features interactions, which means that a feature may be chosen at the expense of two others that are more informative together. We analyze the problem of searching the features subset space with reference to two dimensions, namely, memory of past selected features subset and future selected features. We propose a new wrapper feature selection method based on the deep artificial curiosity framework, which implements intrinsic reward reinforcement learning, with long short-term memory unit (LSTM). This novel algorithm integrates these two elements of memory and future step. We show that our method, called the Deep Curious Feature Selection (DCFS) algorithm, deals with feature interactions, and provides a feature subset which improves the accuracy of learning models on artificial and real datasets.

Hybrid wrapper feature selection method based on genetic algorithm and extreme learning machine for intrusion detection

Intrusion detection systems play a critical role in the mitigation of cyber-attacks on the Internet of Things (IoT) environment. Due to the integration of many devices within the IoT environment, a huge amount of data is generated. The generated data sets in most cases consist of irrelevant and redundant features that affect the performance of the existing intrusion detection systems (IDS). The selection of optimal features plays a critical role in the enhancement of intrusion detection systems. This study proposes a sequential feature selection approach using an optimized extreme learning machine (ELM) with an SVM (support vector machine) classifier. The main challenge of ELM is the selection of the input parameters, which affect its performance. In this study, the genetic algorithm (GA) is used to optimize the weights of ELM to boost its performance. After the optimization, the algorithm is applied as an estimator in the sequential forward selection (wrapper technique) to select key features. The final obtained feature subset is applied for classification using SVM. The IoT_ToN network and UNSWNB15 datasets were used to test the model's performance. The performance of the model was compared with other existing state-of-the-art classifiers such as k-nearest neighbors, gradient boosting, random forest, and decision tree. The model had the best quality of the selected feature subset. The results indicate that the proposed model had a better intrusion detection performance with 99%, and 86% accuracy for IoT_ToN network dataset and UNSWNB15 datasets, respectively. The model can be used as a promising tool for enhancing the classification performance of IDS datasets.

Machine learning model for predicting immediate postoperative desaturation using spirometry signal data

Postoperative desaturation is a common post-surgery pulmonary complication. The real-time prediction of postoperative desaturation can become a preventive measure, and real-time changes in spirometry data can provide valuable information on respiratory mechanics. However, there is a lack of related research, specifically on using spirometry signals as inputs to machine learning (ML) models. We developed an ML model and postoperative desaturation prediction index (DPI) by analyzing intraoperative spirometry signals in patients undergoing laparoscopic surgery. We analyzed spirometry data from patients who underwent laparoscopic, robot-assisted gynecologic, or urologic surgery, identifying postoperative desaturation as a peripheral arterial oxygen saturation level below 95%, despite facial oxygen mask usage. We fitted the ML model on two separate datasets collected during different periods. (Datasets A and B). Dataset A (Normal 133, Desaturation 74) was used for the entire experimental process, including ML model fitting, statistical analysis, and DPI determination. Dataset B (Normal 20, Desaturation 4) was only used for verify the ML model and DPI. Four feature categories—signal property, inter-/intra-position correlation, peak value/interval variability, and demographics—were incorporated into the ML models via filter and wrapper feature selection methods. In experiments, the ML model achieved an adequate predictive capacity for postoperative desaturation, and the performance of the DPI was unbiased.

Hybrid distributed feature selection using particle swarm optimization-mutual information

Feature selection (FS) is a data preprocessing step in machine learning (ML) that selects a subset of relevant and informative features from a large feature pool. FS helps ML models improve their predictive accuracy at lower computational costs. Moreover, FS can handle the model overfitting problem on a high-dimensional dataset. A major problem with the filter and wrapper FS methods is that they consume a significant amount of time during FS on high-dimensional datasets. The proposed “HDFS(PSO-MI): hybrid distribute feature selection using particle swarm optimization-mutual information (PSO-MI)”, is a PSO-based hybrid method that can overcome the problem mentioned above. This method hybridizes the filter and wrapper techniques in a distributed manner. A new combiner is also introduced to merge the effective features selected from multiple data distributions. The effectiveness of the proposed HDFS(PSO-MI) method is evaluated using five ML classifiers, i.e., logistic regression (LR), k-NN, support vector machine (SVM), decision tree (DT), and random forest (RF), on various datasets in terms of accuracy and Matthew’s correlation coefficient (MCC). From the experimental analysis, we observed that HDFS(PSO-MI) method yielded more than 98%, 95%, 92%, 90%, and 85% accuracy for the unbalanced, kidney disease, emotions, wafer manufacturing, and breast cancer datasets, respectively. Our method shows promising results comapred to other methods, such as mutual information, gain ratio, Spearman correlation, analysis of variance (ANOVA), Pearson correlation, and an ensemble feature selection with ranking method (EFSRank).

Improved feature selection using a hybrid side-blotched lizard algorithm and genetic algorithm approach

<span lang="EN-US">Feature selection entails choosing the significant features among a wide collection of original features that are essential for predicting test data using a classifier. Feature selection is commonly used in various applications, such as bioinformatics, data mining, and the analysis of written texts, where the dataset contains tens or hundreds of thousands of features, making it difficult to analyze such a large feature set. Removing irrelevant features improves the predictor performance, making it more accurate and cost-effective. In this research, a novel hybrid technique is presented for feature selection that aims to enhance classification accuracy. A hybrid binary version of side-blotched lizard algorithm (SBLA) with genetic algorithm (GA), namely SBLAGA, which combines the strengths of both algorithms is proposed. We use a sigmoid function to adapt the continuous variables values into a binary one, and evaluate our proposed algorithm on twenty-three standard benchmark datasets. Average classification accuracy, average number of selected features and average fitness value were the evaluation criteria. According to the experimental results, SBLAGA demonstrated superior performance compared to SBLA and GA with regards to these criteria. We further compare SBLAGA with four wrapper feature selection methods that are widely used in the literature, and find it to be more efficient.</span>

End-to-end lightweight berry number prediction for supporting table grape cultivation

The advent of smart agriculture has revolutionized and streamlined various manual tasks in grape cultivation, one of which is berry thinning. This task necessitates experienced farmers to selectively remove a specific number of berries from the working bunch, as guided by the remaining number of berries in the bunch. In response, this paper introduces a novel real-time edge computing application that automates the process of counting the berries in a working bunch using a single 2D image. The proposed application employs YOLOv5-based object detection techniques (Jocher, 2021) to distinguish each working bunch and the visible and slightly occluded berries contained therein. The key contribution of this paper is to accurately predict the number of berries in the whole bunches including those not visible in a 2D image by harnessing the output from object detection to devise features based solely on bounding box information. In addition, the feature set is optimized by employing a wrapper feature selection method (Kohavi & John, 1997), in consideration of the limitations of edge computing devices. The eight selected features yield a mean absolute error (MAE) of 2.60 berries, tested on a dataset of 26,230 images. Only a slight increase over the initial 19-feature set, which achieved an MAE of 2.42 berries. Furthermore, the proposed approach has been successfully implemented and tested on an Android smartphone, the Sony Xperia 1 III, without the need for an internet connection. The overall computation time per image stands at an average of 0.333 s, confirming its potential for real-world application.

Filter and Wrapper Stacking Ensemble (FWSE): a robust approach for reliable biomarker discovery in high-dimensional omics data.

Selecting informative features, such as accurate biomarkers for disease diagnosis, prognosis and response to treatment, is an essential task in the field of bioinformatics. Medical data often contain thousands of features and identifying potential biomarkers is challenging due to small number of samples in the data, method dependence and non-reproducibility. This paper proposes a novel ensemble feature selection method, named Filter and Wrapper Stacking Ensemble (FWSE), to identify reproducible biomarkers from high-dimensional omics data. In FWSE, filter feature selection methods are run on numerous subsets of the data to eliminate irrelevant features, and then wrapper feature selection methods are applied to rank the top features. The method was validated on four high-dimensional medical datasets related to mental illnesses and cancer. The results indicate that the features selected by FWSE are stable and statistically more significant than the ones obtained by existing methods while also demonstrating biological relevance. Furthermore, FWSE is a generic method, applicable to various high-dimensional datasets in the fields of machine intelligence and bioinformatics.

Prediction of Pavement Overall Condition Index Based on Wrapper Feature-Selection Techniques Using Municipal Pavement Data

Municipalities make significant efforts with limited resources to collect pavement condition data. Overall Condition Index (OCI), which uses the pavement surface evaluation and rating manual to identify roadways needing repair, is a convenient and common way of pavement condition assessment. Many data used in assessing the OCI are collected from fieldwork. Some data features give little insight into road conditions, and one feature may provide similar information to another; thus, effective data-collection resources can be optimized by selecting which data feature to keep and which to discard. In addition, the OCI reflects how local agencies highlight the important variables driving their pavement-management investments. It is also a reflection of the triggers that they use to propose various treatment strategies. This research aimed to evaluate pavement distresses in West Des Moines, Iowa, using machine-learning methods, and determine which combination of distresses and their distress proportions can accurately predict the OCI class of a particular pavement type. The wrapper feature-selection methods were used, fitting their results to classification-tree models. Automatic feature selection, using Featurewiz, was also considered to select the desired number of variables. Feature parameters were screened for OCI prediction using the mean decrease impurity, and the results could be used to model classification that may be used for year-long predictions. Results showed that an effective OCI estimate methodology could be determined with significant accuracy with fewer features.

An Empirical Evaluation of Feature Selection Stability and Classification Accuracy

The performance of inductive learners can be negatively affected by high-dimensional datasets. To address this issue, feature selection methods are used. Selecting relevant features and reducing data dimensions is essential for having accurate machine learning models. Stability is an important criterion in feature selection. Stable feature selection algorithms maintain their feature preferences even when small variations exist in the training set. Studies have emphasized the importance of stable feature selection, particularly in cases where the number of samples is small and the dimensionality is high. In this study, we evaluated the relationship between stability measures, as well as, feature selection stability and classification accuracy, using the Pearson Product-Moment Correlation Coefficient. We conducted an extensive series of experiments using five filter and two wrapper feature selection methods, three classifiers for subset and classification performance evaluation, and eight real-world datasets taken from two different data repositories. We measured the stability of feature selection methods using a total of twelve stability metrics. Based on the results of correlation analyses, we have found that there is a lack of substantial evidence supporting a linear relationship between feature selection stability and classification accuracy. However, a strong positive correlation has been observed among several stability metrics.

Predicting the Energy Consumption of Commercial Buildings Based on Deep Forest Model and Its Interpretability

Building energy assessment models are considered to be one of the most informative methods in building energy efficiency design, and most of the current building energy assessment models have been developed based on machine learning algorithms. Deep learning models have proved their effectiveness in fields such as image and fault detection. This paper proposes a deep learning energy assessment framework with interpretability to support building energy efficiency design. The proposed framework is validated using the Commercial Building Energy Consumption Survey dataset, and the results show that the wrapper feature selection method (Sequential Forward Generation) significantly improves the performance of deep learning and machine learning models compared with the filtered (Mutual Information) and embedded (Least Absolute Shrinkage and Selection Operator) feature selection algorithms. Moreover, the Deep Forest model has an R2 of 0.90 and outperforms the Deep Multilayer Perceptron, the Convolutional Neural Network, the Backpropagation Neural Network, and the Radial Basis Function Network in terms of prediction performance. In addition, the model interpretability results reveal how the features affect the prediction results and the contribution of the features to the energy consumption in a single building sample. This study helps building energy designers assess the energy consumption of new buildings and develop improvement measures.

Wrapper and Hybrid Feature Selection Methods Using Metaheuristic Algorithm for Chest X-Ray Images Classification

Covid-19 virus has led to a tremendous pandemic in more than 200 countries across the globe, leading to severe impacts on the lives and health of a large number of people globally. The emergence of Omicron (SARS-CoV-2), which is a coronavirus 2 variant, an acute respiratory syndrome which is highly mutated, has again caused social limitations around the world because of infectious and vaccine escape mutations. One of the most significant steps in the fight against covid-19 is to identify those who were infected with the virus as early as possible, to start their treatment and to minimize the risk of transmission. Detection of this disease from radiographic and radiological images is perhaps one of the quickest and most accessible methods of diagnosing patients. In this study, a computer aided system based on deep learning is proposed for rapid diagnosis of COVID-19 from chest x-ray images. First, a dataset of 5380 Chest x-ray images was collected from publicly available datasets. In the first step, the deep features of the images in the dataset are extracted by using the dataset pre-trained convolutional neural network (CNN) model. In the second step, Differential Evolution (DE), Ant Colony Optimization (ACO) and Particle Swarm Optimization (PSO) algorithms were used for feature selection in order to find the features that are effective for classification of these deep features. Finally, the features obtained in two stages, Decision Tree (DT), Naive Bayes (NB), support vector machine (SVM), k-Nearest Neighbours (k-NN) and Neural Network (NN) classifiers are used for binary, triple and quadruple classification. In order to measure the success of the models objectively, 10 folds cross validation was used. As a result, 1000 features were extracted with the SqueezeNet CNN model. In the binary, triple and quadruple classification process using these features, the SVM method was found to be the best classifier. The classification successes of the SVM model are 96.02%, 86.84% and 79.87%, respectively. The results obtained from the classification process with deep feature extraction were achieved by selecting the features in the proposed method in less time and with less features. While the performance achieved is very good, further analysis is required on a larger set of COVID-19 images to obtain higher estimates of accuracy.

High-Dimensional Feature Selection Based on Improved Binary Ant Colony Optimization Combined with Hybrid Rice Optimization Algorithm

In the realm of high-dimensional data analysis, numerous fields stand to benefit from its applications, including the biological and medical sectors that are crucial for computer-aided disease diagnosis and prediction systems. However, the presence of a significant number of redundant or irrelevant features can adversely affect system accuracy and real-time diagnosis efficiency. To mitigate this issue, this paper proposes two innovative wrapper feature selection (FS) methods that integrate the ant colony optimization (ACO) algorithm and hybrid rice optimization (HRO). HRO is a recently developed metaheuristic that mimics the breeding process of the three-line hybrid rice, which is yet to be thoroughly explored in the context of solving high-dimensional FS problems. In the first hybridization, ACO is embedded as an evolutionary operator within HRO and updated alternately with it. In the second form of hybridization, two subpopulations evolve independently, sharing the local search results to assist individual updating. In the initial stage preceding hybridization, a problem-oriented heuristic factor assignment strategy based on the importance of the knee point feature is introduced to enhance the global search capability of ACO in identifying the smallest and most representative features. The performance of the proposed algorithms is evaluated on fourteen high-dimensional biomedical datasets and compared with other recently advanced FS methods. Experimental results suggest that the proposed methods are efficient and computationally robust, exhibiting superior performance compared to the other algorithms involved in this study.

Modeling dissolved oxygen concentration using machine learning techniques with dimensionality reduction approach.

Oxygen is crucial to keep the life cycle balance in any aspect. Aquatic life is highly influenced by the levels of dissolved oxygen (DO). This calls for not just constant monitoring of the DO in aquatic systems, but to generate an accurate prediction model for future levels of the DO. This study aims to propose an accurate prediction model for DO concentrations. The performance of the Random Forest (RF) and multilayer perceptron (MLP) algorithms was evaluated in generating the regression models. Moreover, the effect of dimensionality reduction of the data by the wrapper feature Selection method on the performance of the models was evaluated. The results showed that the RF regressor excelled MLP in performance with both the dataset of all variables and the dataset of reduced variables with the best performance achieved by the RF regressor by considering Pearson correlation coefficient (0.8052), Mean absolute error (0.8911), and root mean square error (1.2805) when trained by the dataset of reduced variables. As for the accuracy of the models, the estimation error deviation of both models declined significantly when trained by the reduced variables. When the accuracy of the prediction was increased by 0.95% by the RF regressor, the accuracy of the MLP was incremented by 5.7% when trained by the dataset of reduced variables. The results demonstrated the positive impact of the dimensionality reduction on the accuracy of both models. However, RF can be considered a robust regressor in predicting DO concentrations.

A binary chaotic horse herd optimization algorithm for feature selection

One of the most challenging and common problems in machine learning is the Feature Selection (FS) process, which reduces the dataset size by finding optimal subsets of features. The Horse Herd Optimization Algorithm (HOA) is a new metaheuristic algorithm created by modeling the herd behavior of horses and developed for large scale optimization problems. This paper proposes the binary version of the HOA as a wrapper FS method to solve the FS problem. The proposed algorithm is a binary chaotic horse herd optimization algorithm for feature selection (BCHOAFS). The proposed BCHOAFS is applied to select the optimal feature combination that maximizes classification accuracy while minimizing the number of selected features. Classifier algorithms from machine learning algorithms were used to test the accuracy of the reduced subsets. The proposed method was named binary horse herd optimization for feature selection (BHOAFS) before adding chaotic maps; the k-nearest neighbor (k-NN) and Support Vector Machine (SVM) were tested as separate classifiers. It has been seen that k-NN classification accuracy gives better results than SVM. The BHOAFS-kNN method using the k-NN classification was combined with five chaotic maps and named as BCHOAFS-Logistics, BCHOAFS-Piecewise, BCHOAFS-Singer, BCHOAFS-Sinusoidal, BCHOAFS-Tent. The BCHOAFS versions were run on datasets consisting of 18 different sizes and quality datasets (i.e., low scale, medium scale, and large scale) taken from the UCI repository and compared with state-of-the-art algorithms in previous studies. The results prove that the proposed version, especially with the BCHOAFS-Piecewise and the BCHOAFS-Singer chaotic map outperforms or competes with well-known methods. The proof of the proposed approach’s statistical significance has been validated using the Friedman Signed Rank test and post hoc Wilcoxon test. The novelty of BCHOAFS is that HOA, which is an optimization algorithm specially designed for large scale data, is the first binary chaotic-based algorithm developed for feature selection problems. It also proposes a new local search strategy called Similarity Measurement Function (SMF). As a result, versions of the proposed algorithm BCHOAFS can be used for the FS problem.

A novel relation aware wrapper method for feature selection

Feature selection, aiming at eliminating irrelevant and redundant features, is an important data preprocessing technology for downstream tasks, e.g., classification. With the explosive growth of data in various fields, some data are high-dimensional and contain critical and complex hidden relationships, which brings new challenges to feature selection: i) How to find out the underlying available relationships from the data, and ii) how to use the learned relations to better select features? To deal with these challenges, we propose a novel wrapper feature selection method named Relation Aware Feature Selection Method (ERASE), which can learn and use the underlying sample relations and feature relations for feature selection. Different from existing methods, our method jointly learns sample relationships and feature relationships through a graph of samples and trees of features. Furthermore, it uses the relations to select the optimal feature subset according to the new proposed Relation-based Sequence Floating Selection Strategy. Extensive experimental results on nine datasets from different domains demonstrate that our method achieves the best performance in most cases compared with other feature selection methods, including state-of-the-art wrapper methods.

A Multi-level Random Forest Model-Based Intrusion Detection Using Fuzzy Inference System for Internet of Things Networks

Intrusion detection (ID) methods are security frameworks designed to safeguard network information systems. The strength of an intrusion detection method is dependent on the robustness of the feature selection method. This study developed a multi-level random forest algorithm for intrusion detection using a fuzzy inference system. The strengths of the filter and wrapper approaches are combined in this work to create a more advanced multi-level feature selection technique, which strengthens network security. The first stage of the multi-level feature selection is the filter method using a correlation-based feature selection to select essential features based on the multi-collinearity in the data. The correlation-based feature selection used a genetic search method to choose the best features from the feature set. The genetic search algorithm assesses the merits of each attribute, which then delivers the characteristics with the highest fitness values for selection. A rule assessment has also been used to determine whether two feature subsets have the same fitness value, which ultimately returns the feature subset with the fewest features. The second stage is a wrapper method based on the sequential forward selection method to further select top features based on the accuracy of the baseline classifier. The selected top features serve as input into the random forest algorithm for detecting intrusions. Finally, fuzzy logic was used to classify intrusions as either normal, low, medium, or high to reduce misclassification. When the developed intrusion method was compared to other existing models using the same dataset, the results revealed a higher accuracy, precision, sensitivity, specificity, and F1-score of 99.46%, 99.46%, 99.46%, 93.86%, and 99.46%, respectively. The classification of attacks using the fuzzy inference system also indicates that the developed method can correctly classify attacks with reduced misclassification. The use of a multi-level feature selection method to leverage the advantages of filter and wrapper feature selection methods and fuzzy logic for intrusion classification makes this study unique.

Drug–target interaction prediction based on protein features, using wrapper feature selection

Drug–target interaction prediction is a vital stage in drug development, involving lots of methods. Experimental methods that identify these relationships on the basis of clinical remedies are time-taking, costly, laborious, and complex introducing a lot of challenges. One group of new methods is called computational methods. The development of new computational methods which are more accurate can be preferable to experimental methods, in terms of total cost and time. In this paper, a new computational model to predict drug–target interaction (DTI), consisting of three phases, including feature extraction, feature selection, and classification is proposed. In feature extraction phase, different features such as EAAC, PSSM and etc. would be extracted from sequence of proteins and fingerprint features from drugs. These extracted features would then be combined. In the next step, one of the wrapper feature selection methods named IWSSR, due to the large amount of extracted data, is applied. The selected features are then given to rotation forest classification, to have a more efficient prediction. Actually, the innovation of our work is that we extract different features; and then select features by the use of IWSSR. The accuracy of the rotation forest classifier based on tenfold on the golden standard datasets (enzyme, ion channels, G-protein-coupled receptors, nuclear receptors) is as follows: 98.12, 98.07, 96.82, and 95.64. The results of experiments indicate that the proposed model has an acceptable rate in DTI prediction and is compatible with the proposed methods in other papers.

Late acceptance hill climbing aided chaotic harmony search for feature selection: An empirical analysis on medical data

In today’s era of data-driven digital society, there is a huge demand for optimized solutions that essentially reduce the cost of operation, thereby aiming to increase productivity. Processing a huge amount of data, like the Microarray based gene expression data, using machine learning and data mining algorithms has certain limitations in terms of memory and time requirements. This would be more concerning, when a dataset comes with redundant and non-important information. For example, many report-based medical datasets have several non-informative attributes which mislead the classification algorithms. To this end, researchers have been developing several feature selection algorithms that try to discard the redundant information from the raw datasets before feeding them to machine learning algorithms. Metaheuristic based optimization algorithms provide an excellent option to solve feature selection problems. In this paper, we propose a music-inspired harmony search (HS) algorithm based wrapper feature selection method. At the beginning, we use a chaotic mapping to initialize the population of the HS algorithm in order to better coverage of the search space. Further to complement the inferior exploitation of the HS algorithm, we integrate it with the Late Acceptance Hill Climbing (LAHC) method. Thus the combination of these two algorithms provides a good balance between the exploration and exploitation of the HS algorithm. We evaluate the proposed feature selection method on 15 UCI datasets and the obtained results are found to be better than many state-of-the-art methods both in terms of the classification accuracy and the number of features selected. To evaluate the effectiveness of our algorithm, we utilize a combination of precision, recall, F1 score, fitness value, and execution time as performance indicators. These metrics enable us to obtain a comprehensive assessment of the algorithm’s abilities and limitations. We also apply our method on 3 microarray based gene expression datasets used for prediction of cancer to ensure the scalability and robustness as a feature selection method in real-life scenarios. In addition to this, we test our approach using the COVID-19 dataset, and it performs better than several metaheuristic based optimization techniques.

An Exhaustive Wrapper Method for Feature Selection in Large Dimensional Datasets (WFS)

In this paper, a novel algorithm for randomly selecting a small subset of features from a dataset is presented. Using different combinations of features across a number of trials, the algorithm discovers the best subsets of features. When these subsets of features are obtained, the classification accuracies produced by three classifiers (K-Nearest Neighbor, Support Vector Machines, and Random Forest) are considered to evaluate the performance criterion of the proposed wrapper-based method. Further, to improve the classification accuracy and reduce the cardinality of the selected feature sets, an exhaustive feature selection method (the wrapper method) is used. The proposed algorithm is simulated on eighteen datasets, and the results are compared with those reported using nine comparable algorithms using three classifiers to justify the performance of the proposed algorithm. The average classification accuracies of eighteen datasets achieved are 88.66% in K-NN, 89.88% in SVM, and 89.14% in RF classifier with at most 10 features. The proposed algorithm archives better CA compared to nine comparable algorithms and the results of the experiments prove the proposed algorithm's performance is better in selecting the most effective features compared to other algorithms.