Evolutionary Feature Selection Research Articles

Predicting the Risk of Overweight and Obesity in Madrid—A Binary Classification Approach with Evolutionary Feature Selection

In this paper, we experimented with a set of machine-learning classifiers for predicting the risk of a person being overweight or obese, taking into account his/her dietary habits and socioeconomic information. We investigate with ten different machine-learning algorithms combined with four feature-selection strategies (two evolutionary feature-selection methods, one feature selection from the literature, and no feature selection). We tackle the problem under a binary classification approach with evolutionary feature selection. In particular, we use a genetic algorithm to select the set of variables (features) that optimize the accuracy of the classifiers. As an additional contribution, we designed a variant of the Stud GA, a particular structure of the selection operator of individuals where a reduced set of elitist solutions dominate the process. The genetic algorithm uses a direct binary encoding, allowing a more efficient evaluation of the individuals. We use a dataset with information from more than 1170 people in the Spanish Region of Madrid. Both evolutionary and classical feature-selection methods were successfully applied to Gradient Boosting and Decision Tree algorithms, reaching values up to 79% and increasing the average accuracy by two points, respectively.

Evolutionary deep feature selection for compact representation of gigapixel images in digital pathology

Despite the recent progress in Deep Neural Networks (DNNs) to characterize histopathology images, compactly representing a gigapixel whole-slide image (WSI) via salient features to enable computational pathology is still an urgent need and a significant challenge. In this paper, we propose a novel WSI characterization approach to represent, search and classify biopsy specimens using a compact feature vector (CFV) extracted from a multitude of deep feature vectors. Since the non-optimal design and training of deep networks may result in many irrelevant and redundant features and also cause computational bottlenecks, we proposed a low-cost stochastic method to optimize the output of pre-trained deep networks using evolutionary algorithms to generate a very small set of features to accurately represent each tissue/biopsy. The performance of the proposed method has been assessed using WSIs from the publicly available TCGA image data. In addition to acquiring a very compact representation (i.e., 11,000 times smaller than the initial set of features), the optimized features achieved 93% classification accuracy resulting in 11% improvement compared to the published benchmarks. The experimental results reveal that the proposed method can reliably select salient features of the biopsy sample. Furthermore, the proposed approach holds the potential to immensely facilitate the adoption of digital pathology by enabling a new generation of WSI representation for efficient storage and more user-friendly visualization.

Strip Hardness Prediction in Continuous Annealing Using Multiobjective Sparse Nonlinear Ensemble Learning With Evolutionary Feature Selection

In the iron and steel industry, the hardness of steel strips is one of the key performance indicators to evaluate strip quality and guide production for the continuous annealing production line (CAPL). However, the hardness cannot be measured online in the actual production process. Consequently, the precise prediction of the strip hardness based on practical data becomes one of the key tasks during production. In this article, a multiobjective sparse nonlinear ensemble learning with evolutionary feature selection (MOSNE-EFS) method is proposed, which is data-driven modeling of the soft sensor. The method mainly consists of two stages: 1) the construction of individual learners based on multiobjective feature selection learning (MOFSL) and 2) the selection and ensemble of individual learners based on sparse nonlinear ensemble learning via differential evolution (SNEL-DE). The final ensemble model obtained by SNEL-DE is used as the prediction model for strip hardness in CAPL. The proposed method is evaluated with industrial production data. Experimental results indicate that the two strategies, i.e., evolutionary feature selection and sparse nonlinear ensemble, are effective in improving the accuracy and robustness of the prediction model, and further comparison results demonstrate the superiority of the MOSNE-EFS model over the other existing methods. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Many quality metrics in the iron and steel industry cannot be online checked, which causes great difficulties in process monitoring, control, and operation optimization. The proposed multiobjective sparse nonlinear ensemble learning with evolutionary feature selection method can help practitioners to construct quality prediction models of many other similar production lines, such as hot rolling and cold rolling, and thus, better process monitoring, control, and optimization of product quality can be achieved.

An Evolutionary Multitasking-Based Feature Selection Method for High-Dimensional Classification

Feature selection (FS) is an important data preprocessing technique in data mining and machine learning, which aims to select a small subset of information features to increase the performance and reduce the dimensionality. Particle swarm optimization (PSO) has been successfully applied to FS due to being efficient and easy to implement. However, most of the existing PSO-based FS methods face the problems of trapping into local optima and computationally expensive high-dimensional data. Multifactorial optimization (MFO), as an effective evolutionary multitasking paradigm, has been widely used for solving complex problems through implicit knowledge transfer between related tasks. Inspired by MFO, this study proposes a novel PSO-based FS method to solve high-dimensional classification via information sharing between two related tasks generated from a dataset. To be specific, two related tasks about the target concept are established by evaluating the importance of features. A new crossover operator, called assortative mating, is applied to share information between these two related tasks. In addition, two mechanisms, which are variable-range strategy and subset updating mechanism, are also developed to reduce the search space and maintain the diversity of the population, respectively. The results show that the proposed FS method can achieve higher classification accuracy with a smaller feature subset in a reasonable time than the state-of-the-art FS methods on the examined high-dimensional classification problems.

Prediction of Delivery Within 7 Days After Diagnosis of Early Onset Preeclampsia Using Machine-Learning Models.

BackgroundEarly onset preeclampsia (eoPE) is a hypertensive disorder of pregnancy with endothelial dysfunction manifested before 34 weeks where expectant management is usually attempted. However, the timing of hospitalization, corticosteroids, and delivery remain a challenge. We aim to develop a prediction model using machine-learning tools for the need for delivery within 7 days of diagnosis (model D) and the risk of developing hemolysis, elevated liver enzymes, and low platelets (HELLP) syndrome or abruptio placentae (model HA).Materials and MethodsA retrospective cohort of singleton pregnancies with eoPE and attempted expectant management between 2014 and 2020. A Mono-objective Genetic Algorithm based on supervised classification models was implemented to develop D and HA models. Maternal basal characteristics and data gathered during eoPE diagnosis: gestational age, blood pressure, platelets, creatinine, transaminases, angiogenesis biomarkers (soluble fms-like tyrosine kinase-1, placental growth factor), and ultrasound data were pooled for analysis. The most relevant variables were selected by bio-inspired algorithms. We developed basal models that solely included demographic characteristics of the patient (D1, HA1), and advanced models adding information available at diagnosis of eoPE (D2, HA2).ResultsWe evaluated 215 eoPE cases and 47.9% required delivery within 7 days. The median time-to-delivery was 8 days. Basal models were better predicted by K-nearest-neighbor in D1, which had a diagnostic precision of 0.68 ± 0.09, with 63.6% sensitivity (Sn), 71.4% specificity (Sp), 70% positive predictive value (PPV), and 65.2% negative predictive value (NPV) using 13 variables and HA1 of 0.77 ± 0.09, 60.4% Sn, 80% Sp, 50% PPV, and 87.9% NPV. Models at diagnosis were better developed by support vector machine (SVM) using 18 variables, where D2’s precision improved to 0.79 ± 0.05 with 77.3% Sn, 80.1% Sp, 81.5% PPV, and 76.2% NPV, and HA2 had a precision of 0.79 ± 0.08 with 66.7% Sn, 82.8% Sp, 51.6% PPV, and 90.3% NPV.ConclusionAt the time of diagnosis of eoPE, SVM with evolutionary feature selection process provides good predictive information of the need for delivery within 7 days and development of HELLP/abruptio placentae, using maternal characteristics and markers that can be obtained routinely. This information could be of value when assessing hospitalization and timing of antenatal corticosteroid administration.

A Multiobjective Evolutionary Nonlinear Ensemble Learning With Evolutionary Feature Selection for Silicon Prediction in Blast Furnace.

In the blast furnace ironmaking process, accurate prediction of silicon content in molten iron is of great significance for maintaining stable furnace conditions, improving hot metal quality, and reducing energy consumption. However, most of the current research works employ linear correlation coefficient methods to select input features in modeling, which may not fully take the nonlinear and coupling relationships between features into account. Therefore, this article considers the input feature selection issue of silicon content prediction model from a new perspective and proposes a multiobjective evolutionary nonlinear ensemble learning model with evolutionary feature selection mechanism (MOENE-EFS), in which extreme learning machine is adopted as the base learner. MOENE-EFS takes the input feature scheme of each base learner as well as their network structure and parameters as decision variables and proposes a modified nondominated sorting differential evolution algorithm to optimize two conflicting objectives, i.e., accuracy and diversity of base learners, simultaneously. Through the optimization, a set of Pareto optimal base learners with high accuracy and strong diversity can be obtained. Moreover, different from the linear ensemble methods commonly used in classical evolutionary ensemble learning, this article proposes a nonlinear ensemble method to combine the obtained base learners based on differential evolution. Experimental results indicate that the two proposed strategies, i.e., evolutionary feature selection and nonlinear ensemble, are very effective in improving the accuracy and stability of the prediction model. MOENE-EFS also outperforms the other prediction models in both benchmark data and practical industrial data. Furthermore, analysis on the input features of all Pareto optimal base learners shows that the evolutionary feature selection is capable of selecting essential features and is consistent with human experience, which indicates it is a promising method to deal with the input feature selection issue in silicon content prediction.

Effective memetic algorithm for multilabel feature selection using hybridization-based communication

Multilabel feature selection, which is used to select features relevant to multiple labels, has demonstrated its effectiveness in prediction time and learning accuracy. A memetic algorithm that uses a feature subset refining process has been verified to outperform existing genetic algorithms in identifying an optimal feature subset. However, as the refinement process is consistently applied to all solutions of feature subsets, similar feature subsets can be over-produced, thereby limiting the synergy of hybridization. Here, we propose an evolutionary multilabel feature selection algorithm that searches the final feature subset using multiple populations to prevent limiting the synergy of hybridization. A new hybridization-based communication process refines solutions originated from each best solution of multiple populations, then, randomly distributes the produced solutions. With this approach, the proposed method circumvents the degradation of search capability and keeps the synergy of hybridization. Our experimental results indicate that the proposed method could identify better feature subsets than conventional methods.

Intrusion detection in cyber–physical environment using hybrid Naïve Bayes—Decision table and multi-objective evolutionary feature selection

Researchers are motivated to build effective Intrusion Detection Systems because of the implications of malicious actions in computing, communication, and cyber–physical systems (IDSs). In order to develop signature-based intrusion detection techniques that are suitable for use in cyber–physical environments, state-of-the-art supervised learning algorithms are devised. The main contribution of this research is the introduction of a signature-based intrusion detection model that is based on a hybrid Decision Table and Naive Bayes technique. In addition, the contribution of the suggested method is evaluated by comparing it to the existing literature in the field. In the preprocessing stage, Multi-Objective Evolutionary Feature Selection (MOEFS) feature selection has been used to select only five attack features from the recent CICIDS017 dataset. Keeping in view the class imbalance nature of CICIDS2017 dataset, adequate attack samples has been selected with more weightage to the attack classes having a smaller number of instances in the dataset. A hybrid of Decision Table and Naive Bayes models were combined to train and detect intrusions. Detection of botnets, port scans, Denial of Service (DoS)/Distributed Denial of Service (DDoS) attacks, such as Golden-Eye, Hulk, Slow httptest, slowloris, Heartbleed, Brute Force attacks, such as Patator (FTP), Patator (SSH), and Web attacks such as Infiltration, Web Brute Force, SQL Injection, and XSS, are all successfully detected by the proposed hybrid detection model. The proposed approach shows an accuracy of 96.8% using five features of CICIDS2017, which is higher than the accuracy of methods discussed in the literatures.

Distributed Evolutionary Feature Selection for Big Data Processing

Feature selection has become a powerful dimensional reduction strategy and an effective tool in handling high-dimensional data. Feature selection aims to reduce the dimension of the feature space, to speed up and reduce the cost of the learning model and that by selecting the most relevant feature subset to data mining and machine learning tasks. The selection of optimal feature subset is an optimization problem that proved to be NP-hard. Metaheuristics are traditionally used to deal with NP-hard problems since they are well known for solving complex and real-world problems in reasonable period of time. Genetic algorithm (GA) is one of the most popular metaheuristics algorithms, which proved to be effective for an accurate feature selection task. However, in the last few decades, data have become progressively larger in both numbers of instances and features. This paradigm is being popularly termed as Big Data. With the tremendous growth of dataset sizes, most current feature selection algorithms and exceptionally GA become unscalable. To improve the scalability of a feature selection algorithm on big data, the distributed computing strategy is always adopted such as MapReduce model and Hadoop system. In this paper, we first present a review for the most recent works which handle the use of Parallel Genetic algorithm in large datasets. Then, we will propose a new Parallel Genetic algorithm based on the Coarse-grained parallelization model (island model). The parallelization of the process and the distribution of the partitioning of data will be performed using Hadoop system with an Amazon cluster. The performance and the scalability of the proposed method were theoretically and empirically compared to the existing feature selection methods when handling large-scale datasets and results confirm the effectiveness of our proposed method.

The design of evolutionary feature selection operator for the micro-expression recognition

The design of evolutionary feature selection operator for the micro-expression recognition

Predicting the severity of COVID‐19 patients using a multi‐threaded evolutionary feature selection algorithm

AbstractThe COVID‐19 pandemic has huge effects on the global community and an extreme burden on health systems. There are more than 185 million confirmed cases and 4 million deaths as of July 2021. Besides, the exponential rise in COVID‐19 cases requires a quick prediction of the patients' severity for better treatment. In this study, we propose a Multi‐threaded Genetic feature selection algorithm combined with Extreme Learning Machines (MG‐ELM) to predict the severity level of the COVID‐19 patients. We conduct a set of experiments on a recently published real‐world dataset. We reprocess the dataset via feature construction to improve the learning performance of the algorithm. Upon comprehensive experiments, we report the most impactful features and symptoms for predicting the patients' severity level. Moreover, we investigate the effects of multi‐threaded implementation with statistical analysis. In order to verify the efficiency of MG‐ELM, we compare our results with traditional and state‐of‐the‐art techniques. The proposed algorithm outperforms other algorithms in terms of prediction accuracy.

Mining and Interpretation of Critical Aspects of Infant Health Status Using Multi-Objective Evolutionary Feature Selection Approaches

The rate of infant mortality (IMR) in a population under one year of age is a marker for infant mortality. It is a major sensitive marker of a community&#x2019;s overall physical health. Protecting the lives of newborns has become a challenging issue in public health, development programs, and humanitarian initiatives. Almost 10.1&#x0025; infants died in the United States of America (USA) in 2021. Therefore, this paper aims to extract and understand the various influential factors causing infant deaths in the USA. A crowding distance-based multi-objective ant lion optimization (MOALO-CD) is proposed here with statistical evidence for feature selection. The proposed technique is compared with competitive metaheuristic models such as multi-objective genetic algorithm based on crowding distance (MOGA-CD), multi-objective filter approaches, and recursive feature elimination. Various machine learning classifiers are applied to the selected feature subset obtained from MOALO-CD on the USA&#x2019;s infant dataset. Extensive experimental results indicate that the proposed model outperforms the existing metaheuristic approaches in terms of Generational Distance, Inverted Generational Distance, Spread, and Hyper volume. Also, the comparative analysis of various machine learning models reveals that random forest achieves significantly better performance on the feature subset obtained from MOALO-CD.

What an “Ehm” Leaks About You: Mapping Fillers into Personality Traits with Quantum Evolutionary Feature Selection Algorithms

This work shows that fillers - short utterances like “ehm” and “uhm” - allow one to predict whether someone is above median along the Big-Five personality traits. The experiments have been performed over a corpus of 2,988 fillers uttered by 120 different speakers in spontaneous conversations. The results show that the prediction accuracies range between 74 and 82 percent depending on the particular trait. The proposed approach includes a feature selection step - based on Quantum Evolutionary Algorithms - that has been used to detect the personality markers, i.e., the subset of the features that better account for the prediction outcomes and, indirectly, for the personality of the speakers. The results show that only a relatively few features tend to be consistently selected, thus acting as reliable personality markers.

Combined adaptive neuro‐fuzzy inference system and genetic algorithm for e‐learning resilience assessment during COVID‐19 pandemic

AbstractGiven the growing use of e‐learning and expansion of internet‐based infrastructure during COVID‐19 epidemic, the need for a resilient approach to e‐learning systems is deeply felt. This article introduces a combined technique utilizing adaptive neuro‐fuzzy inference system (ANFIS) and genetic algorithm (GA), named ANFIS‐GA, to evaluate e‐learning resilience. In the proposed ANFIS model, 22 features from five main factors including individual, technology, content, agility, and assessment/support factors are used as fuzzy inputs, while the e‐learning resilience is considered as a single output of the model. To select the most significant features for the evaluation of the e‐learning resilience, an evolutionary feature selection based on GA is used. The proposed ANFIS‐GA model has been successfully developed for evaluation of e‐learning resilience in virtual Iranian university. According to the obtained results, agility is the most important factor, and then, technology and assessment/support factors have the next priorities to evaluate e‐learning resilience in virtual Iranian university. Statistical analysis demonstrated that there is no significant difference between the experts' opinion and the resilience obtained via the proposed model. The proposed ANFIS‐GA model can be used in any educational institution to evaluate the improvement of resilience in e‐learning.

An Intelligent Hybrid Sentiment Analyzer for Personal Protective Medical Equipments Based on Word Embedding Technique: The COVID-19 Era

Due to the accelerated growth of symmetrical sentiment data across different platforms, experimenting with different sentiment analysis (SA) techniques allows for better decision-making and strategic planning for different sectors. Specifically, the emergence of COVID-19 has enriched the data of people’s opinions and feelings about medical products. In this paper, we analyze people’s sentiments about the products of a well-known e-commerce website named Alibaba.com. People’s sentiments are experimented with using a novel evolutionary approach by applying advanced pre-trained word embedding for word presentations and combining them with an evolutionary feature selection mechanism to classify these opinions into different levels of ratings. The proposed approach is based on harmony search algorithm and different classification techniques including random forest, k-nearest neighbor, AdaBoost, bagging, SVM, and REPtree to achieve competitive results with the least possible features. The experiments are conducted on five different datasets including medical gloves, hand sanitizer, medical oxygen, face masks, and a combination of all these datasets. The results show that the harmony search algorithm successfully reduced the number of features by 94.25%, 89.5%, 89.25%, 92.5%, and 84.25% for the medical glove, hand sanitizer, medical oxygen, face masks, and whole datasets, respectively, while keeping a competitive performance in terms of accuracy and root mean square error (RMSE) for the classification techniques and decreasing the computational time required for classification.

A Clustering-Guided Integer Brain Storm Optimizer for Feature Selection in High-Dimensional Data

For high-dimensional data with a large number of redundant features, existing feature selection algorithms still have the problem of “curse of dimensionality.” In view of this, the paper studies a new two-phase evolutionary feature selection algorithm, called clustering-guided integer brain storm optimization algorithm (IBSO-C). In the first phase, an importance-guided feature clustering method is proposed to group similar features, so that the search space in the second phase can be reduced obviously. The second phase applies oneself to finding optimal feature subset by using an improved integer brain storm optimization. Moreover, a new encoding strategy and a time-varying integer update method for individuals are proposed to improve the search performance of brain storm optimization in the second phase. Since the number of feature clusters is far smaller than the size of original features, IBSO-C can find an optimal feature subset fast. Compared with several existing algorithms on some real-world datasets, experimental results show that IBSO-C can find feature subset with high classification accuracy at less computation cost.

Automated detection of bioimages using novel deep feature fusion algorithm and effective high-dimensional feature selection approach

The classification of bioimages plays an important role in several biological studies, such as subcellular localisation, phenotype identification and other types of histopathological examinations. The objective of the present study was to develop a computer-aided bioimage classification method for the classification of bioimages across nine diverse benchmark datasets. A novel algorithm was developed, which systematically fused the features extracted from nine different convolution neural network architectures. A systematic fusion of features boosts the performance of a classifier but at the cost of the high dimensionality of the fused feature set. Therefore, non-discriminatory and redundant features need to be removed from a high-dimensional fused feature set to improve the classification performance and reduce the time complexity. To achieve this aim, a method based on analysis of variance and evolutionary feature selection was developed to select an optimal set of discriminatory features from the fused feature set. The proposed method was evaluated on nine different benchmark datasets. The experimental results showed that the proposed method achieved superior performance, with a significant reduction in the dimensionality of the fused feature set for most bioimage datasets. The performance of the proposed feature selection method was better than that of some of the most recent and classical methods used for feature selection. Thus, the proposed method was desirable because of its superior performance and high compression ratio, which significantly reduced the computational complexity.

Evolutionary ensemble feature selection learning for image‐based assessment of lymphedema arm volume

SummaryArm lymphedema is a side effect of surgery and radiation therapy in patients with breast cancer. There are several methods to measure lymphedema arm volume, such as water displacement (WD) and circumferential measurement (CM). Although these techniques have a satisfactory level of accuracy, they have some limitations for clinical use in practical applications. To overcome these drawbacks, a horizontal‐vertical image scanning (HVIS) tool, is presented as a valid method for arm volume measurement. To predict the arm volume, an evolutionary ensemble feature selection learning (named EEFSL) comprising different base learners (KNN, SVM, ANFIS, Decision Tree, and Naïve Bayes) is presented. In order to improve the performance of the EEFSL model, its hyperparameters are optimized using genetic algorithm (GA). The hyperparameters include the weights of different base learners and the specific features for each base learner. The fitness function of GA is to maximize the correlation between predicted and actual arm volumes. The proposed method was successfully developed for arm measurement of 60 arms from 30 women with arm lymphedema. Obtained results indicate that the proposed HVIS‐EEFSL method is a reliable and valid technique to measure arm volume of patients with lymphedema, suitable for daily clinical use instead of WD and CM.

Evolutionary Algorithm-based Feature Selection for an Intrusion Detection System

Keeping computer reliability to confirm reliable, secure, and truthful correspondence of data between different enterprises is a major security issue. Ensuring information correspondence over the web or computer grids is always under threat of hackers or intruders. Many techniques have been utilized in intrusion detections, but all have flaws. In this paper, a new hybrid technique is proposed, which combines the Ensemble of Feature Selection (EFS) algorithm and Teaching Learning-Based Optimization (TLBO) techniques. In the proposed, EFS-TLBO method, the EFS strategy is applied to rank the features for choosing the ideal best subset of applicable information, and the TLBO is utilized to identify the most important features from the produced datasets. The TLBO algorithm uses the Extreme Learning Machine (ELM) to choose the most effective attributes and to enhance classification accuracy. The performance of the recommended technique is evaluated in a benchmark dataset. The experimental outcomes depict that the proposed model has high predictive accuracy, detection rate, false-positive rate, and requires less significant attributes than other techniques known from the literature.

Enhanced Evolutionary Feature Selection and Ensemble Method for Cardiovascular Disease Prediction.

Cardiovascular Disease (CVD) is one among the main factors for the increase in mortality rate worldwide. The analysis and prediction of this disease is yet a highly formidable task in medical data analysis. Recent advancements in technology such as Big Data, Artificial Intelligence and the need for automated models have paved the way for developing a more reliable and efficient model for predicting heart disease. Several researches have been carried out in predicting heart diseases but the focus on choosing the important attributes that play a significant role in predicting CVD is inadequate. Hence the choice of right features for the classification and the diagnosis of the heart disease is important. The core aim of this work is to identify and select the important features and machine learning methodologies that can enhance the prediction capability of the classification models for accurately predicting CVD. The results show that the proposed enhanced evolutionary feature selection with the hybrid ensemble model outperforms the existing approaches in terms of precision, recall and accuracy. The experimental outcomes show that the proposed approach attains the maximum classification accuracy of 93.65% for statlog dataset, 82.81% for SPECTF dataset and 84.95% for coronary heart disease dataset. The proposed classification model performance is demonstrated using ROC curve against state-of-the-art methods in machine learning.