Feature Subset Selection Research Articles

Bowerbird courtship-inspired feature selection for efficient high-dimensional data analysis using a novel meta-heuristic

The challenge of feature selection in high-dimensional datasets is critical in enhancing machine learning model performance and interpretability while reducing computational costs. This paper introduces a novel feature selection method inspired by the unique courtship behaviors of bowerbirds, termed Bowerbird Courtship-Inspired Feature Selection (BBFS). Mimicking the decorative strategies and selective adaptation of bowerbirds, BBFS optimizes the selection of feature subsets through a meta-heuristic approach that efficiently balances exploration and exploitation. We compare the effectiveness of BBFS with established algorithms like Binary Cuckoo Search Optimization (BCSO) and Binary Grey Wolf Optimization (BGWO) across seven diverse datasets. Our results demonstrate that BBFS consistently selects fewer features and maintains or enhances classification accuracy using Decision Trees and Naive Bayes classifiers. This study underscores the potential of BBFS as a scalable and robust tool for high-dimensional data analysis, offering significant improvements over traditional and contemporary feature selection methods.Graphical

A hybrid critical channels and optimal feature subset selection framework for EEG fatigue recognition

Fatigue driving is one of the potential factors threatening road safety, and monitoring drivers’ mental state through electroencephalography (EEG) can effectively prevent such risks. In this paper, a new model, DE-GFRJMCMC, is proposed for selecting critical channels and optimal feature subsets from EEG data to improve the accuracy of fatigue driving recognition. The model is validated on the SEED-VIG dataset. The model first selects critical EEG channels using the Differential Evolution (DE) algorithm, extracting important electrode channel information to enhance recognition accuracy. These electrode channels are used to construct a Functional Brain Network (FBN), from which the topological feature set is extracted. Empirical Mode Decomposition (EMD) is then applied to extract the intrinsic mode components as network nodes, thereby reducing the influence of the number of electrode channels on the brain functional network. The topological features extracted from these components form the suboptimal feature set. To minimize redundant information, we propose an improved Reversible Jump Markov Chain Monte Carlo (RJMCMC) algorithm for selecting the optimal feature subset, ensuring both the efficiency and accuracy of fatigue recognition. The optimal feature subsets were input into various classifiers, and the results showed that the K-Nearest Neighbor (KNN)-based classifier achieved the highest recognition accuracy of 96.11% ± 0.43%, demonstrating the method’s stability and robustness. Compared to similar studies, this model shows superior performance in fatigue driving recognition, which is of significant value for research on fatigue driving detection and prevention.

A novel ANN-based feature subset selection in multi-scale granular ball neighborhood decision tables

A novel ANN-based feature subset selection in multi-scale granular ball neighborhood decision tables

Effective Data Classification using Interval Neutrosophic Covering Rough Sets based on Neighborhoods for FinTech Applications

Neutrosophic set (NS) is particularly appropriate in applications where data is incomplete, unclear, or inconsistent, which offers an effectual means for analyzing and exhibiting complex mechanisms. An NS is a mathematical technique to manage uncertainty, indeterminacy, and imprecision. It enlarges classical sets, IF sets, and fuzzy sets by presenting three degrees such as indeterminacy (I), false (F), and truth (T). Financial technology (Fintech) plays an essential part in advancing modern society, technology, economies, and various fields. Financial crisis prediction (FCP) plays a crucial role in shaping economic outcomes. Past research has predominantly focused on using deep learning (DL), machine learning (ML), and statistical methods to forecast the financial stability of business. In this manuscript, we focus on the development of Effective Data Classification using Interval Neutrosophic Covering Rough Sets based on Neighborhoods and Multi-Strategy Improved Butterfly Optimization (EDCINCRS-MSIBO) Algorithm for FinTech Applications. It contains distinct kinds of stages such as data normalization, feature selection, classification, and parameter tuning. In the EDCINCRS-MSIBO technique, a min-max normalization-based data pre-processing model to scale the raw data into a uniform format. For feature subset selection, the whale optimizer algorithm (WOA) is employed to reduce the dimensionality and improve model efficiency by selecting the most relevant features. In addition, the EDCINCRS-MSIBO technique takes place interval neutrosophic covering rough sets (INCRS) classifier is utilized for detection and classification of a financial crisis. Finally, a multi-strategy improved butterfly optimization algorithm (MSIBOA) is exploited for the optimum parameter adjustment of the INCRS model. To confirm the better predictive solution of the EDCINCRS-MSIBO model, a wide range of simulations are executed on the two benchmark databases. The comparative result analysis displays the encouraging outcomes of the EDCINCRS-MSIBO method on the existing techniques

A Comparative Analysis of Metaheuristic Feature Selection Methods in Software Vulnerability Prediction

Background: Early identification of software vulnerabilities is an intrinsic step in achieving software security. In the era of artificial intelligence, software vulnerability prediction models (VPMs) are created using machine learning and deep learning approaches. The effectiveness of these models aids in increasing the quality of the software. The handling of imbalanced datasets and dimensionality reduction are important aspects that affect the performance of VPMs. Aim: The current study applies novel metaheuristic approaches for feature subset selection. Method: This paper performs a comparative analysis of forty-eight combinations of eight machine learning techniques and six metaheuristic feature selection methods on four public datasets. Results: The experimental results reveal that VPMs productivity is upgraded after the application of the feature selection methods for both metrics-based and text-mining-based datasets. Additionally, the study has applied Wilcoxon signed-rank test to the results of metrics-based and text-features-based VPMs to evaluate which outperformed the other. Furthermore, it discovers the best-performing feature selection algorithm based on AUC for each dataset. Finally, this paper has performed better than the benchmark studies in terms of F1-Score. Conclusion: The results conclude that GWO has performed satisfactorily for all the datasets.

Optimizing Input Selection for Cardiac Model Training and Inference: An Efficient 3D CNN-Based Approach to Automate Coronary Angiogram Video Selection

Optimizing Input Selection for Cardiac Model Training and Inference: An Efficient 3D CNN-Based Approach to Automate Coronary Angiogram Video Selection

AI-Driven Optimization of Renewable Energy Storage Systems in Smart Cities Using Improved Binary Genetic Algorithm

Today, the utilization and management of renewable energy have become integral to the development of smart cities. This paper explores the application of Artificial Intelligence (AI) in analyzing energy storage and renewable energy systems within smart city contexts. We introduce a joint optimization method that combines two-stage input feature selection and parameter training in traditional prediction models. This method integrates physical, statistical, and AI modeling techniques. Additionally, we conduct comprehensive experiments on feature input selection using a Binary Genetic Algorithm (BGA) and two-stage model parameter optimization based on a Natural Evolution Strategies (NES). The experimental results provide strong support for the conclusions of this study. Finally, we simulate the full charge process of energy storage in a smart city and predict the full charge data for batteries B0005, B0006, B0007, and B0029. The results indicate that the NES-BGA-ELM approach outperforms the BGA-ELM method in terms of error reduction.

Towards a New MI-Driven Methodology for Predicting the Prices of Cryptocurrencies

Forecasting the price of cryptocurrencies is a notoriously hard and significant problem, due to the rapid market growth and high volatility. In this article, we propose a methodology for predicting future values of cryptocurrency exchange rates by developing a Non-linear Autoregressive with Exogenous Inputs (NARX) prediction model that uses the most adequate external information. The exogenous variables considered are historical values of the exchange rate and a series of technical indicators. The selection of the most relevant external inputs is based on the computation of the mutual information indicator and estimated using the k-nearest neighbor method. The methodology employs a fine-tuned Long Short-Term Memory (LSTM) neural network as the regressor. We have used quantitative and trend accuracy measures to compare the proposed method against other state-of-the-art LSTM-based models. In addition, regarding the input selection process, the proposed approach was compared against the most commonly used one, which is based on the cross-correlation coefficient. A long series of experiments and statistical analyses proved that the proposed methodology is highly accurate and the resulting model outperforms the state-of-the-art LSTM-based models.

Class-specific feature selection for enhancing explainability in ensemble classification models

AbstractFeature selection techniques aim at finding a relevant subset of features that perform equally or better than the original set of features at explaining the behavior of data. Typically, features are extracted from feature ranking or subset selection techniques, and the performance is measured by classification or regression tasks. However, while selected features may not have equal importance for the task, they do have equal importance for each class. The fundamental idea of the class-specific concept resides in the understanding that the significance of each feature can vary from one class to another. This contrasts with the traditional class-independent approach, which evaluates the importance of attributes collectively for all classes. For example, in tumor prediction scenarios, each type of tumor (class) may be associated with a distinct subset of relevant features. These features possess significant discriminatory power, enabling the differentiation of one tumor type from others (classes). This class-specific perspective offers a more effective approach to classification tasks by recognizing and leveraging the unique characteristics of each class. A novel deep one-versus-each strategy is introduced, which offers advantages from the point of view of explainability (feature selection) and decomposability (classification). In addition, the class-specific relevance matrix is presented, from which some more sophisticated classification schemes can be derived, such as the three-layer class-specific scheme. These schemes have the great advantage to combine independent classification units (e.g., neural networks) that use a reduced number of features to target each class. The potential for further advancements in this area is wide and will open new horizons for exploring novel research directions in interdisciplinary fields, particularly in complex, multiclass hyperdimensional contexts (e.g., in genomics).

Assessing pest control treatments from phenology models and field data.

Efficacy of insecticides is often determined from apparent yield loses due to a target pest. However, pests can affect yields even when controls work as expected. Further, most pest populations are monitored through adult counts without procedures to assess dynamics of immature stages. Here, we propose a framework to assess the efficacy of control treatments from adult counts in non-experimental setups based on the shifts in temporal patterns of adult emergence caused during the residual period of treatments applied to kill immatures. We use phenology models scaled to field counts to track the stage structure of pest populations across a season and produce reference population trajectories with and without the treatment. Field-collected trajectories are then classified as with or without an effective control through a time-sequential probability ratio test. The method was evaluated using pheromone trap captures of codling moth, Cydia pomonella, and four of the most widely implemented treatment programs in apple and pear orchards. Simulations revealed that when field-collected trajectories are classified as treated with a control, there is 70% chance that the treatment program is > 50% effective, or that the program is < 66% effective when field-collected trajectories are classified as untreated, provided the trajectories are made of ≥ 15 pheromone traps. This framework is a powerful, evidence-based tool to optimize the selection of inputs and application protocols for pest control and could be applied to virtually any pest that can be sampled regularly and whose phenology can be modeled as a function of degree-days. © 2024 The Author(s). Pest Management Science published by John Wiley & Sons Ltd on behalf of Society of Chemical Industry.

Predicting creep life of CrMo pressure vessel steel using machine learning models with optimal feature subset selection

The data-driven approach for creep life prediction typically integrates numerous characteristics, including material compositions, manufacturing details, and service conditions, into machine learning models. In this study, a machine learning-based creep life prediction approach with optimal feature subset selection is established for 2.25Cr1Mo pressure vessel steel. Before model training and testing, six critical features that significantly impact the creep life of 2.25Cr1Mo steel are selected, specifically the applied stress, temperature, and chemical compositions consisting of Cr, Ni, Mn, and Mo. Various machine learning algorithms, along with the traditional L–M method, are utilized for model training and performance evaluation. Additionally, the developed models undergo validation using experimental data independent of the training and testing datasets to assess their generalization abilities. The results reveal that, among all tested models, the support vector regression (SVR) model, coupled with the optimal feature subset, demonstrates superior prediction accuracy and generalization capability. Finally, the creep life prediction model exhibiting optimal performance is deployed into a software application, leveraging the Python programming language. This predictor tool facilitates rapid and precise creep life predictions for 2.25Cr1Mo pressure vessel steel, relying solely on a limited amount of input information, and provides a clear and visual presentation of the prediction results.

An Effective Hybrid Feature Selection Method Based on an Improved Artificial GTO Algorithm for Medical Datasets

Feature subset selection is considered as the most essential pre-processing step. Metaheuristic approaches may be employed to discover a solution to difficulties in feature selection, which can be viewed as an optimisation problem. The aim of the system is to provide a hybrid binary metaheuristic algorithm that combines gorilla troop optimisation and genetic algorithm to handle the feature selection issue effectively. This new method is called GTO-GA. To ensure that the optimisation technique converges fast and properly and to enhance the exploration process, the GA were used. The suggested technique is tested for stability and robustness using 16 medical datasets taken from the Kaggle and UCI repositories. To evaluate the chosen features’ performance in classification issues further. The results show that the algorithm outperforms 10 top-tier optimisation methods, including PSO, ALO, the original GTO and the SCA algorithm. The results highlighted the statistical difference, superiority and importance of the suggested feature selection strategies.

Hybrid multivariate time series prediction system fusing transfer entropy and local relative density

Kernel extreme learning machine (KELM), as a natural extension of ELM to kernel learning, has been successfully applied to solve various multivariate time series prediction (MTSP) tasks. Nevertheless, the high-dimensional and nonlinear properties of prediction information against the background of big data bring great challenges to the application of KELM. Recognizing these challenges, this paper develops a KELM-based hybrid MTSP system, aiming to address the effective mining of potential relationships among variables and sample significance. Our system is initiated by devising a feature evaluation mechanism that leverages transfer entropy and directed graph theory, effectively capturing the intricate interactions and intrinsic influences among variables. Next, we introduce a robust local relative density concept to gauge the significance level of different samples in KELM learning, and develop a more efficient KELM. Diverging from previous MTSP methodologies, the developed prediction system is capable of automatically discovering potential relationships between input features and modelling, and simultaneously realizes feature subset selection and modelling learning. Empirical evidence drawn from real-world datasets substantiates the effectiveness and practicality of our proposed system. The results not only validate our approach but also highlight its theoretical and practical superiority over existing state-of-the-art methods.

Spatial-temporal characteristics and the importance of environmental factors in relation to algal blooms in coastal seas

In the prediction of algal blooms with the artificial intelligence (AI) technology, the proper selection of the indices is crucial to the quality of the results. Although eutrophication and climate change have been identified as the main contributors to algal blooms, limited knowledge of the relationships among environmental factors hinders the accurate and correct AI prediction. To explore the importance of environmental factors in the occurrence of algal blooms in coastal seas and facilitate the selection of inputs for the prediction model, this study examines observed water quality data and analyzes their relationship using machine learning methods. It was found that nutrients, water temperature, and salinity play important roles in controlling algal blooms in coastal seas, and their influence varies depending on locations. The high-risk cases of algal bloom mainly occur in environments with medium salinity (22–28 psu), particularly in open estuaries with high nutrient levels. According to the feature importance analysis, water temperature presented the most significant impact on algal bloom risk compared to other environmental factors. The variations of water temperature and Chl-a concentration are synchronous when the water temperature is below 32 °C. Additionally, the relationships between inorganic nutrients (NO3−-NO2− and Ortho-P) and Chl-a exhibit asynchronous across multi-temporal scales. The uptake of inorganic nutrients occurs in large quantities within the 24 h prior to the boom in Chl-a concentration. This demonstrates that variations in concentration of inorganic nutrients can be considered as crucial factors in algal bloom predictions, and their significant decrease can be served as an important signal of an impending algal bloom. This study enhances our understanding of the mechanisms behind algal blooms and facilitates their management and prediction strategies in coastal seas.

A Gentle Introduction and Application of Feature-Based Clustering with Psychological Time Series

Psychological researchers and practitioners collect increasingly complex time series data aimed at identifying differences between the developments of participants or patients. Past research has proposed a number of dynamic measures that describe meaningful developmental patterns for psychological data (e.g., instability, inertia, linear trend). Yet, commonly used clustering approaches are often not able to include these meaningful measures (e.g., due to model assumptions). We propose feature-based time series clustering as a flexible, transparent, and well-grounded approach that clusters participants based on the dynamic measures directly using common clustering algorithms. We introduce the approach and illustrate the utility of the method with real-world empirical data that highlight common ESM challenges of multivariate conceptualizations, structural missingness, and non-stationary trends. We use the data to showcase the main steps of input selection, feature extraction, feature reduction, feature clustering, and cluster evaluation. We also provide practical algorithm overviews and readily available code for data preparation, analysis, and interpretation.

Unveiling Productivity Dynamics Among Regional Rural Banks in India

This study employs the non-parametric Malmquist productivity index to evaluate the total factor productivity (TFP) change among the regional rural banks (RRBs) in India from 2016 to 2023. The analysis of productivity change among the RRBs holds intrinsic significance due to its potential impact on overall performance and to the delivery of quality services among the rural population. To evaluate the productivity change under varying combinations of input and output selections, two distinct approaches were used: the intermediation approach and the production approach. The empirical results reveal substantial variation in average TEP estimates across the RRBs under consideration throughout the sample period. Furthermore, the observed productivity changes predominantly emanated from technological change (TC) rather than efficiency improvements. Additionally, we found that the smaller RRBs were performing better than their larger counterparts. These empirical findings bear noteworthy policy implications, offering valuable insights for policymakers and regulators to formulate effective strategies to ensure the competitiveness and sustainability of RRBs in India.

Quantum computing and quantum-inspired techniques for feature subset selection: a review

Quantum computing and quantum-inspired techniques for feature subset selection: a review

Coal and Gangue Detection Networks with Compact and High-Performance Design.

The efficient separation of coal and gangue remains a critical challenge in modern coal mining, directly impacting energy efficiency, environmental protection, and sustainable development. Current machine vision-based sorting methods face significant challenges in dense scenes, where label rewriting problems severely affect model performance, particularly when coal and gangue are closely distributed in conveyor belt images. This paper introduces CGDet (Coal and Gangue Detection), a novel compact convolutional neural network that addresses these challenges through two key innovations. First, we proposed an Object Distribution Density Measurement (ODDM) method to quantitatively analyze the distribution density of coal and gangue, enabling optimal selection of input and feature map resolutions to mitigate label rewriting issues. Second, we developed a Relative Resolution Object Scale Measurement (RROSM) method to assess object scales, guiding the design of a streamlined feature fusion structure that eliminates redundant components while maintaining detection accuracy. Experimental results demonstrate the effectiveness of our approach; CGDet achieved superior performance with AP50 and AR50 scores of 96.7% and 99.2% respectively, while reducing model parameters by 46.76%, computational cost by 47.94%, and inference time by 31.50% compared to traditional models. These improvements make CGDet particularly suitable for real-time coal and gangue sorting in underground mining environments, where computational resources are limited but high accuracy is essential. Our work provides a new perspective on designing compact yet high-performance object detection networks for dense scene applications.

Chaos Game Optimization with stacked LSTM sequence to sequence autoencoder for malware detection in IoT cloud environment

Malware detection in Internet of Things (IoT) cloud platforms is a crucial security system for securing data and devices' integrity, secrecy, and availability. IoT devices are linked to cloud-based services offering storage, calculating, and analytics abilities. However, these devices are also exposed to malware attacks that could cause significant damage. Malware detection in IoT cloud platforms involves analyzing and identifying potential threats like Trojans, viruses, ransomware, and worms. It is done through several processes, including behavior-based detection, signature-based detection, and anomaly-based detection. The study proposes a Chaos Game Optimization with improved deep learning for Malware Detection (CGOIDL-MD) technique in the IoT cloud platform. The proposed CGOIDL-MD technique majorly concentrates on the automated detection and classification of malware in the IoT cloud framework. The CGOIDL-MD method applies the CGO-based feature subset selection (CGO-FSS) approach to select features. Besides, the stacked long short-term memory sequence-to-sequence autoencoder (SLSTM-SSAE) approach was exploited for malware classification and detection. Moreover, the arithmetic optimization algorithm (AOA) technique was exploited for the hyperparameter selection technique. The simulation outcomes of the CGOIDL-MD technique were tested on the malware dataset, and the outcome can be studied from different perspectives. The experimentation outcomes illustrate the betterment of the CGOIDL-MD technique under various measures.

Improving multi-view ensemble learning with Round-Robin feature set partitioning

Multi-view Ensemble Learning (MEL) techniques have shown remarkable success in improving the accuracy and resilience of classification algorithms by combining multiple base classifiers trained over different perspectives of a dataset, known as views. One crucial factor affecting ensemble performance is the selection of diverse and informative feature subsets. Feature Set Partitioning (FSP) methods address this challenge by creating distinct views of features for each base classifier. In this context, we propose the Round-Robin Feature Set Partitioning (RR-FSP) technique, which introduces a novel approach to feature allocation among views. This novel approach evenly distributes highly correlated features across views, thereby enhancing ensemble diversity, promoting balanced feature utilization, and encouraging the more equitable distribution of correlated features, RR-FSP contributes to the advancement of MEL techniques. Through experiments on various datasets, we demonstrate that RR-FSP offers improved classification accuracy and robustness, making it a valuable addition to the arsenal of FSP techniques for MEL.