Evolutionary Feature Selection Research Articles

Authenticity markers of cultivated blueberries (Vaccinium sp.) and European wild blueberry (V. myrtillus) assessed by hyperspectral imaging and machine learning algorithms

The present study was undertaken to tackle food fraud in the food chain by developing a novel panel of analytical fingerprints to distinguish the fruit of bilberry (Vaccinium myrtillus), a wild European blueberry grown in Finland, from their commercial cultivated counterparts based on V. corymbosum or other originally American Vaccinium species. The analytical fingerprints were derived using hyperspectral imaging (HSI). The cultivated domestic blueberry samples from Finland (N = 188), those imported (N = 338), and domestic Finnish wild bilberry fruit (N = 247) were imaged using Specim IQ and FX17 HSI cameras. The fruit HSI data were then classified using seven machine learning model types. The supervised evolutionary feature selection algorithm was used to recognize the most separating subset of wavelengths (nm predictors, colors) between the fruit class labels, which were 1) wild bilberry fruit vs. cultivated blueberry fruit and 2) domestic cultivated blueberry fruit vs. imported cultivated blueberry fruit. Moreover, we ran 190,822 individual models and 54,137 predictor combinations to determine whether subsets or all wavelengths better distinguished fruit class labels. Overall, the best model type, the rbf kernel support vector machine (rbf-SVM), showed excellent prediction performance when classifying wild bilberries vs. cultivated blueberries with HSI data obtained using both camera types (4.1% and 3.2% holdout set errors). Also the rbf-SVM performed well when classifying domestic cultivated blueberries vs. those imported with HSI data obtained using an IQ camera (2.3% holdout set error). Our results showed that hyperspectral imaging could predict food fraud attempts concerning the Vaccinium fruit of different origin and species.

Multi-population differential evolution approach for feature selection with mutual information ranking

Feature selection is a crucial aspect of data preprocessing because of the significant effect of redundant features on classification performance and the extensive computational resources required. Evolutionary algorithm-based feature-selection methods have shown remarkable results in determining the optimal feature subset. To further enhance classification performance, this paper proposes a novel multi-population differential evolution approach for feature selection with mutual information ranking (MI-MPODE). Firstly, population preprocessing guided by mutual information is employed to reduce the dimensionality of the initial feature space. Then, the feature subset obtained from mutual information serves as the initial population for MI-MPODE. MI-MPODE incorporates a novel multi-population information-sharing mechanism, with common individuals from three layers contributing to inter-subpopulation information sharing. Additionally, an individual enhancement strategy is proposed to handle the variations of individuals in the population, and a Lens imaging opposition-based learning method is adopted to improve the algorithm’s optimization capability. MI-MPODE is compared with several state-of-the-art evolutionary algorithm-based feature selection methods through experimental comparisons. The results show that MI-MPODE outperforms all comparison algorithms on more than half of the datasets, with a significant reduction in the number of features used, demonstrating a significant advantage over competitors.

Developing a crash severity model based on multi objective evolutionary feature selection approaches

The main purpose of this article is assessing the capability of multiobjective optimisation evolutionary algorithms (MOEAs) along with machine learning method on predicting the crash injury severity with the approach of reducing input variables. Dataset of crashes which were occurred in Iran was analysed over the period of 2016–2020. The data were classified into two classes in terms of injury severity: fatal and non-fatal datasets. At first, general prediction models were created by inserting 31 available input variables into support vector machine (SVM) method based on two imbalanced and obtained balanced datasets. A cluster based under sampling technique was used to obtained balanced data. Then evolutionary algorithms of two objectives optimisation algorithms namely non-dominated sorting genetic algorithm (NSGA-II), multi-objective evolutionary algorithm based on decomposition (MOEA/D), and pareto envelope-based selection algorithm (PESA-II) along with SVM were used to select features that significantly affect the severity of crashes and developed the simplified models. Then the simplified developed model was compared to the model which was prepared with random forest (RF) method in feature selection. It is shown that the simplified model developed by combined MOEAs and SVM is more appropriate to accurately predict the crash severity, considering the number of input variables in comparison to RF.

A Survey on Evolutionary Multiobjective Feature Selection in Classification: Approaches, Applications, and Challenges

A Survey on Evolutionary Multiobjective Feature Selection in Classification: Approaches, Applications, and Challenges

Evolutionary computation for feature selection in classification: A comprehensive survey of solutions, applications and challenges

Feature selection (FS), as one of the most significant preprocessing techniques in the fields of machine learning and pattern recognition, has received great attention. In recent years, evolutionary computation has become a popular technique for handling FS problems due to its superior global search performance. In this paper, a comprehensive review of evolutionary computation research on the FS problems is presented. Firstly, a new taxonomy for the basic components of evolutionary feature selection algorithms (EFSs) is proposed, including encoding strategy, population initialization, population updating, local search, multi-FS hybrid and ensemble. Secondly, we summarize the latest research progress of EFSs on some new and complex scenarios, including large-scale high-dimensional data, multi-objective/metric scenario, multi-label data, distributed storage data, multi-task scenario, multi-modal scenario, interpretable FS and stable FS, etc. Moreover, this survey provides also an in-depth analysis of real-world applications of EFSs, such as hyperspectral band selection, bioinformatics gene selection, text classification and fault detection, etc. Finally, several opportunities for future work are pointed out.

Surrogate-Assisted and Filter-Based Multiobjective Evolutionary Feature Selection for Deep Learning.

Feature selection (FS) for deep learning prediction models is a difficult topic for researchers to tackle. Most of the approaches proposed in the literature consist of embedded methods through the use of hidden layers added to the neural network architecture that modify the weights of the units associated with each input attribute so that the worst attributes have less weight in the learning process. Other approaches used for deep learning are filter methods, which are independent of the learning algorithm, which can limit the precision of the prediction model. Wrapper methods are impractical with deep learning due to their high computational cost. In this article, we propose new attribute subset evaluation FS methods for deep learning of the wrapper, filter and wrapper-filter hybrid types, where multiobjective and many-objective evolutionary algorithms are used as search strategies. A novel surrogate-assisted approach is used to reduce the high computational cost of the wrapper-type objective function, while the filter-type objective functions are based on correlation and an adaptation of the reliefF algorithm. The proposed techniques have been applied in a time series forecasting problem of air quality in the Spanish south-east and an indoor temperature forecasting problem in a domotic house, with promising results compared to other FS techniques used in the literature.

Surrogate-assisted multi-objective evolutionary feature selection of generation-based fixed evolution control for time series forecasting with LSTM networks

Surrogate-assisted multi-objective evolutionary algorithms are powerful techniques to solve computationally expensive multi-objective optimization problems. In this paper, we propose a direct fitness replacement method with generation-based fixed evolution control to implement a multi-objective evolutionary algorithm that uses a surrogate model for wrapper-type feature selection, where long short-term memory is established as the learning algorithm. The importance of the work and its benefits lie in the need to reduce the excessive computational time required by conventional wrapper-type feature selection methods based on multi-objective evolutionary algorithms and LSTM networks, maintaining or improving the predictive capacity of the models. We analyze the use of incremental learning to update the surrogate model, in comparison with the conventional non-incremental learning approach. We applied these methods in real-life time series forecasting of air quality, indoor temperature in a smart building and oil temperature in electricity transformers. Multi-step ahead predictions of the forecast models obtained with different meta-learners of the surrogate model were compared by using the Diebold–Mariano statistical test on a multi-criteria performance metric. The proposed method outperformed other approaches for feature selection including, among others, methods based on surrogate-assisted multi-objective evolutionary algorithms developed by the authors in previous research, other surrogate-assisted deterministic methods for feature selection and the conventional wrapper-type feature selection method based on LSTM, improving the prediction on test dataset by 23.98%, 34.61% and 13.77%, respectively.

A novel probabilistic approach for driving the sustainable energy circular economy: Innovating efficiency in renewable resource markets

Our research introduces a novel hybrid model aimed at driving the sustainable economy by innovating efficiency in forecasting renewable resource market trends for recovery. The fusion of Hierarchical Recurrent Neural Networks (RNNs) and LeNet-5 Convolutional Neural Networks (CNNs) capitalizes on the distinct advantages of both architectures. RNNs capture temporal dependencies, while CNNs identify spatial patterns, providing a holistic understanding of market dynamics. This symbiosis significantly enhances predictive accuracy, empowering stakeholders with more reliable insights into the trajectory of renewable resource markets. Efficient feature selection is achieved through the incorporation of the Modified Hill Climbing (MHC) algorithm. As an evolutionary feature selection technique, MHC dynamically refines input features during the learning process, adapting to the evolving nature of renewable resource markets. This adaptive feature selection not only improves computational efficiency and interpretability but also ensures a focus on the most relevant information, aligning with the goal of driving sustainable circular economic recovery. The fortified hybrid model, harmonizing RNNs, CNNs, and MHC, showcases heightened robustness and generalization across diverse market conditions. By effectively merging deep learning architectures with an evolutionary feature selection process, the model adeptly captures intricate patterns while mitigating overfitting. This contribution underscores the applicability of the proposed approach to real-world scenarios, positioning it as a catalyst for innovative efficiency in renewable resource markets—a pivotal driver for the recovery of the sustainable economy. Our research marks a crucial advancement in the realm of reliable and adaptable forecasting models, essential for informed decision-making in the dynamic landscape of renewable energy markets.

A Dynamic Tasking-Based Evolutionary Algorithm for Bi-Objective Feature Selection

Feature selection in classification is a complex optimization problem that cannot be solved in polynomial time. Bi-objective feature selection, aiming to minimize both selected features and classification errors, is challenging due to the conflict between objectives, while one of the most effective ways to tackle this is to use multi-objective evolutionary algorithms. However, very few of these have ever reflected an evolutionary multi-tasking framework, despite the implicit parallelism offered by the population-based search characteristic. In this paper, a dynamic multi-tasking-based multi-objective evolutionary algorithm (termed DTEA) is proposed for handling bi-objective feature selection in classification, which is not only suitable for datasets with relatively lower dimensionality of features, but is also suitable for datasets with relatively higher dimensionality of features. The role and influence of multi-tasking on multi-objective evolutionary feature selection were studied, and a dynamic tasking mechanism is proposed to self-adaptively assign multiple evolutionary search tasks by intermittently analyzing the population behaviors. The efficacy of DTEA is tested on 20 classification datasets and compared with seven state-of-the-art evolutionary algorithms. A component contribution analysis was also conducted by comparing DTEA with its three variants. The empirical results show that the dynamic-tasking mechanism works efficiently and enables DTEA to outperform other algorithms on most datasets in terms of both optimization and classification.

Role of education and awareness programs in fostering energy conservation behavior in cities: Empowering urban sustainability using deep learning approach

This paper introduces a pioneering framework that emphasizes the critical role of education in promoting energy conservation behavior within urban environments. By synergizing deep learning methodologies with education-awareness formulations and algorithm, the framework aims to bolster energy conservation security in cities. A central aspect of this framework is the development of an advanced Intrusion Detection System (IDS) model, utilizing Restricted Boltzmann Machines (RBMs) tailored to enhance energy consumption security. Through six strategic integrations of educational components, the IDS not only identifies but also mitigates potential threats to energy systems with unprecedented efficiency. Additionally, the research introduces a novel evolutionary feature selection method based on Memetic Algorithms (MAs), which strategically identifies crucial features for RBM training, ensuring optimal model performance. The incorporation of Memetic Algorithms facilitates individual learning and problem-specific knowledge transmission, further enhancing the adaptive capabilities of the system. This multifaceted approach underscores the pivotal role of education in fostering urban sustainability, pushing the boundaries of energy conservation security. By proactively responding to emerging threats, the proposed framework contributes significantly to the overarching goal of sustainable and secure urban energy management.

Design and Adjustment of Optimizing Athletes' Training Programs Using Machine Learning Algorithms

The adjustment of optimizing athletes' training programs using machine learning involves leveraging data-driven approaches to enhance training regimens and performance outcomes for athletes. By analyzing various factors such as athletes' physiological data, training logs, performance metrics, and external conditions, machine learning algorithms can identify patterns, correlations, and optimal training strategies. These insights enable coaches and sports scientists to tailor training programs more effectively to individual athletes' needs, goals, and abilities. By continuously adapting and refining training plans based on real-time feedback and data analysis, machine learning helps optimize athletes' preparation, recovery, and overall performance, ultimately maximizing their potential and success in competitive sports. This paper explores novel methodologies and machine-learning techniques aimed at optimizing athletes' training programs. With the increasing demand for peak performance and injury prevention in sports, there is a growing need for data-driven approaches to tailor training regimens effectively. One such methodology, the Optimized Adjustment Evolutionary Computing Feature Selection (OA-EC-FS), is investigated for its ability to select relevant features crucial for enhancing athletes' performance across various sports disciplines. Additionally, machine learning algorithms are employed to classify athletes' training programs based on selected features, enabling coaches and trainers to make informed decisions to maximize performance outcomes.

CBDA: Chaos-based binary dragonfly algorithm for evolutionary feature selection

The goal of feature selection in machine learning is to simultaneously maintain more classification accuracy, while reducing lager amount of attributes. In this paper, we firstly design a fitness function that achieves both objectives jointly. Then we come up with a chaos-based binary dragonfly algorithm (CBDA) that incorporates several improvements over the conventional dragonfly algorithm (DA) for developing a wrapper-based feature selection method to solve the fitness function. Specifically, the CBDA innovatively introduces three improved factors, namely the chaotic map, evolutionary population dynamics (EPD) mechanism, and binarization strategy on the basis of conventional DA to balance the exploitation and exploration capabilities of the algorithm and make it more suitable to handle the formulated problem. We conduct experiments on 24 well-known data sets from the UCI repository with three ablated versions of CBDA targeting different components of the algorithm in order to explain their contributions in CBDA and also with five established comparative algorithms in terms of fitness value, classification accuracy, CPU running time, and number of selected features. The results show that the proposed CBDA has remarkable advantages in most of the tested data sets.

An evolutionary feature selection method based on probability-based initialized particle swarm optimization

An evolutionary feature selection method based on probability-based initialized particle swarm optimization

Feature Selection Techniques for Enhancing Credit Card Fraud Detection Performance: A Hybrid Metaheuristic Approach Using Nature-Inspired Algorithms

Abstract: The surge in credit card usage has led to a parallel increase in fraudulent transactions, necessitating advanced detection systems. Traditional methods encounter challenges with high-dimensional and imbalanced datasets. This paper proposes a comprehensive approach integrating metaheuristic algorithms and deep learning techniques to enhance fraud detection accuracy. The first contribution, the Rock Hyrax Swarm Optimization Feature Selection (RHSOFS) algorithm, draws inspiration from the collective behaviors of rock hyrax swarms to effectively select relevant features from high-dimensional datasets. RHSOFS identifies an optimal subset of features critical for fraud detection through supervised machine learning. Complementing this, the second contribution leverages a hybrid deep learning model. Beginning by organizing transactional data and constructing a Logical Graph of Behavior Profile (LGBP) to abstract transaction details, the Modified Butterfly Optimization Algorithm (MBOA) selects important features from the dataset. The hybrid model, integrating Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN), establishes rational connections between transactional characteristics, enhancing detection performance. To evaluate the approach, comparative efficiency analyses against existing methods, including Differential Evolutionary Feature Selection (DEFS), Genetic Algorithm Feature Selection (GAFS), Particle Swarm Optimization Feature Selection (PSOFS), and Ant Colony Optimization Feature Selection (ACOFS), are conducted. Results demonstrate the superiority of the approach in terms of both reliability and recognition rate, validated through rigorous statistical testing. The proposed hybrid approach signifies a significant advancement in credit card fraud detection systems, offering enhanced accuracy and efficiency in combating fraudulent activities amidst the growing complexity of transactional datasets.

Evolutionary Multi-Objective Feature Selection Algorithms on Multiple Smart Sustainable Community Indicator Datasets

The conceptual fusion of smart city and sustainability indicators has inspired the emergence of the smart sustainable city (SSC). Given the early stage of development in this field, most SSC studies have been primarily theoretical. Notably, existing empirical studies have overlooked the crucial aspect of feature engineering in the context of SSC, despite its significance in advancing SSC initiatives. This paper introduces an approach advocating for feature subset selection to maximize prediction accuracy and minimize computational time across diverse SSC indicators encompassing socio-cultural, economic, environmental, and governance categories. The study systematically collected multiple datasets on SSC indicators, covering various themes within the SSC framework. Employing six carefully chosen multiple-objective evolutionary feature selection algorithms, the research selected feature subsets. These subsets were then utilized in modeling algorithms to predict SSC indicators. The proposal enhanced prediction accuracy for life expectancy, online shopping intentions, energy consumption, air quality, water quality, and traffic flow for a smart and sustainable city by minimizing the subset features. The findings underscore the efficacy of feature subset selection in generating minimal features, thereby enhancing both prediction accuracy and computational efficiency in the realm of SSC indicators. For researchers aiming to develop sustainable systems for real-time data monitoring within SSC, the identified subset features offer a valuable resource, negating the necessity for extensive dataset collection. The provided SSC datasets are anticipated to serve as a catalyst, inspiring researchers to embark on empirical studies that explore SSC development from diverse perspectives, ultimately contributing to a more profound understanding of the SSC dynamics.

Improving arabic signature authentication with quantum inspired evolutionary feature selection

Improving arabic signature authentication with quantum inspired evolutionary feature selection

Evolutionary feature selection based on hybrid bald eagle search and particle swarm optimization

Feature selection is a complicated multi-objective optimization problem with aims at reaching to the best subset of features while remaining a high accuracy in the field of machine learning, which is considered to be a difficult task. In this paper, we design a fitness function to jointly optimize the classification accuracy and the selected features in the linear weighting manner. Then, we propose two hybrid meta-heuristic methods which are the hybrid basic bald eagle search-particle swarm optimization (HBBP) and hybrid chaos-based bald eagle search-particle swarm optimization (HCBP) that alleviate the drawbacks of bald eagle search (BES) by utilizing the advantages of particle swarm optimization (PSO) to efficiently optimize the designed fitness function. Specifically, HBBP is proposed to overcome the disadvantages of the originals (i.e., BES and PSO) and HCBP is proposed to further improve the performance of HBBP. Moreover, a binary optimization is utilized to effectively transfer the solution space from continuous to binary. To evaluate the effectiveness, 17 well-known data sets from the UCI repository are employed as well as a set of well-established algorithms from the literature are adopted to jointly confirm the effectiveness of the proposed methods in terms of fitness value, classification accuracy, computational time and selected features. The results support the superiority of the proposed hybrid methods against the basic optimizers and the comparative algorithms on the most tested data sets.

Efficient credit card fraud detection using evolutionary hybrid feature selection and random weight networks

In the realm of financial security, the detection and prevention of credit card fraud has become paramount. With the ever-increasing reliance on digital transactions, the risk of fraudulent activities targeting credit card systems has grown significantly. To combat this, sophisticated techniques are required to swiftly identify and mitigate potential threats. Machine learning, a cornerstone of modern data analysis, has emerged as a powerful tool in this pursuit. By leveraging vast datasets and employing advanced algorithms, machine learning enables the automated scrutiny of transactions, distinguishing between legitimate and fraudulent activities with remarkable precision. This paper introduces an intelligent method for credit card fraud detection that relies on Competitive Swarm Optimization (CSO) and Random Weight Network (RWN). Additionally, the system includes an automated hybrid feature selection capability to identify the most pertinent features during the detection process. The experimental outcomes validate that this system can attain outstanding results in G-Mean, RUC, and Recall values.

Probe Population Based Initialization and Genetic Pool Based Reproduction for Evolutionary Bi-Objective Feature Selection

Probe Population Based Initialization and Genetic Pool Based Reproduction for Evolutionary Bi-Objective Feature Selection

Benefiting From Single-Objective Feature Selection to Multiobjective Feature Selection: A Multiform Approach.

Evolutionary multiobjective feature selection (FS) has gained increasing attention in recent years. However, it still faces some challenges, for example, the frequently appeared duplicated solutions in either the search space or the objective space lead to the diversity loss of the population, and the huge search space results in the low search efficiency of the algorithm. Minimizing the number of selected features and maximizing the classification performance are two major objectives in FS. Usually, the fitness function of a single-objective FS problem linearly aggregates these two objectives through a weighted sum method. Given a predefined direction (weight) vector, the single-objective FS task can explore the specified direction or area extensively. Different direction vectors result in different search directions in the objective space. Motivated by this, this article proposes a multiform framework, which solves a multiobjective FS task combined with its auxiliary single-objective FS tasks in a multitask environment. By setting different direction vectors, promising feature subsets from single-objective FS tasks can be utilized, to boost the evolutionary search of the multiobjective FS task. By comparing with five classical and state-of-the-art multiobjective evolutionary algorithms, as well as four well-performing FS algorithms, the effectiveness and efficiency of the proposed method are verified via extensive experiments on 18 classification datasets. Furthermore, the effectiveness of the proposed method is also investigated in a noisy environment.