Integrating Feature Selection Research Articles

Enhanced forest fire susceptibility mapping by integrating feature selection genetic algorithm and bagging-based support vector machine with artificial neural networks

Enhanced forest fire susceptibility mapping by integrating feature selection genetic algorithm and bagging-based support vector machine with artificial neural networks

Predicting Antidiabetic Peptide Activity: A Machine Learning Perspective on Type 1 and Type 2 Diabetes.

Diabetes mellitus (DM) presents a critical global health challenge, characterized by persistent hyperglycemia and associated with substantial economic and health-related burdens. This study employs advanced machine-learning techniques to improve the prediction and classification of antidiabetic peptides, with a particular focus on differentiating those effective against T1DM from those targeting T2DM. We integrate feature selection with analysis methods, including logistic regression, support vector machines (SVM), and adaptive boosting (AdaBoost), to classify antidiabetic peptides based on key features. Feature selection through the Lasso-penalized method identifies critical peptide characteristics that significantly influence antidiabetic activity, thereby establishing a robust foundation for future peptide design. A comprehensive evaluation of logistic regression, SVM, and AdaBoost shows that AdaBoost consistently outperforms the other methods, making it the most effective approach for classifying antidiabetic peptides. This research underscores the potential of machine learning in the systematic evaluation of bioactive peptides, contributing to the advancement of peptide-based therapies for diabetes management.

Financial distress prediction using an improved particle swarm optimization wrapper feature selection method and tree boosting ensemble

Financial distress prediction (FDP) is a complex task involving both feature selection and model construction. While many studies have addressed these challenges individually, there is a lack of FDP models that integrate feature selection into the overall model building process. To address the issues of fast convergence and local optimization in particle swarm optimization (PSO), this study proposes a modified PSO algorithm with three improvement strategies. This improved PSO, known as IPSO, is embedded within state-of-the-art tree boosting ensemble models for feature selection. IPSO continuously evolves toward the optimal feature subset, preventing premature convergence and avoiding local optima. To construct the FDP model, four state-of-the-art tree boosting models are combined using an improved majority voting ensemble strategy. The integration of IPSO enhances the prediction performance of the model. Comparative experiments are conducted on samples of Chinese listed companies under short-term and long-term prediction scenarios to evaluate the proposed model. The results demonstrate that IPSO significantly improves the prediction performance, and the ensemble strategy enhances the robustness of the predictions.

Integrating Feature Selection with Machine Learning for Accurate Reservoir Landslide Displacement Prediction

Accurate prediction of reservoir landslide displacements is crucial for early warning and hazard prevention. Current machine learning (ML) paradigms for predicting landslide displacement demonstrate superior performance, while often relying on various feature engineering techniques, such as decomposing into different temporal lags and feature selection. This study investigates the impact of various feature selection techniques on the performance of ML algorithms for landslide displacement prediction. The Shuping and Baishuihe landslides in China’s Three Gorges Reservoir Area are used to comprehensively benchmark four prevalent ML algorithms. Both static ML models, including backpropagation neural network (BPNN), support vector machine (SVM), and dynamic models, such as long short-term memory (LSTM), and gated recurrent unit (GRU), are included. Each ML model is evaluated under three feature engineering techniques: raw multivariate time series, and feature selection under maximal information coefficient-partial autocorrelation function (MIC-PACF), or grey relational analysis-PACF (GRA-PACF). The results demonstrate that appropriate feature selection methods could significantly improve the performance of static ML models. In contrast, dynamic models effectively leverage inherent capabilities in capturing temporal dynamics within raw multivariate time series, seeing marginal gains with extensive feature engineering compared to no feature selection strategy. The optimal feature selection approach varies based on the ML model and specific landslide, highlighting the importance of case-specific assessments. The findings in this study offer guidance on integrating feature selection techniques with different machine learning models to maximize the robustness and generalizability of data-driven landslide displacement prediction frameworks.

Integrated feature selection and ensemble learning for heart disease detection: a 2-tier approach with ALAN and ET-ABDF machine learning model

Integrated feature selection and ensemble learning for heart disease detection: a 2-tier approach with ALAN and ET-ABDF machine learning model

Dual contrastive learning for multi-view clustering

Self-representation subspace clustering, based on the self-representation property of linear subspaces, has demonstrated superior performance in subspace clustering. However, many existing methods learn self-representation in the embedding space, which fails to accurately capture the clustering structure of the data. Additionally, in the data embedding process, only the shared features among clusters are removed, and the discriminative features that distinguish different clusters cannot be effectively selected. To address these limitations, we propose a contrastive learning-based self-representation subspace clustering method named Multi-view Contrastive Subspace Clustering (MCSC). This method integrates feature selection and self-representation learning into a unified framework. It learns a shared self-representation coefficient in both the embedding space and the original space using contrastive learning, which allows for a more accurate description of the linear relationship between samples and a better depiction of the inter-sample structure simultaneously. Moreover, we employ the ℓ1,2-norm for feature selection, which eliminates shared redundancies while preserving important features specific to each cluster, making the samples more distinguishable. Extensive experiments on six datasets verify the validity of our proposed model.

Responsible AI for cardiovascular disease detection: Towards a privacy-preserving and interpretable model

Background and objectiveCardiovascular disease (CD) is a major global health concern, affecting millions with symptoms like fatigue and chest discomfort. Timely identification is crucial due to its significant contribution to global mortality. In healthcare, artificial intelligence (AI) holds promise for advancing disease risk assessment and treatment outcome prediction. However, machine learning (ML) evolution raises concerns about data privacy and biases, especially in sensitive healthcare applications. The objective is to develop and implement a responsible AI model for CD prediction that prioritize patient privacy, security, ensuring transparency, explainability, fairness, and ethical adherence in healthcare applications. MethodsTo predict CD while prioritizing patient privacy, our study employed data anonymization involved adding Laplace noise to sensitive features like age and gender. The anonymized dataset underwent analysis using a differential privacy (DP) framework to preserve data privacy. DP ensured confidentiality while extracting insights. Compared with Logistic Regression (LR), Gaussian Naïve Bayes (GNB), and Random Forest (RF), the methodology integrated feature selection, statistical analysis, and SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) for interpretability. This approach facilitates transparent and interpretable AI decision-making, aligning with responsible AI development principles. Overall, it combines privacy preservation, interpretability, and ethical considerations for accurate CD predictions. ResultsOur investigations from the DP framework with LR were promising, with an area under curve (AUC) of 0.848 ± 0.03, an accuracy of 0.797 ± 0.02, precision at 0.789 ± 0.02, recall at 0.797 ± 0.02, and an F1 score of 0.787 ± 0.02, with a comparable performance with the non-privacy framework. The SHAP and LIME based results support clinical findings, show a commitment to transparent and interpretable AI decision-making, and aligns with the principles of responsible AI development. ConclusionsOur study endorses a novel approach in predicting CD, amalgamating data anonymization, privacy-preserving methods, interpretability tools SHAP, LIME, and ethical considerations. This responsible AI framework ensures accurate predictions, privacy preservation, and user trust, underscoring the significance of comprehensive and transparent ML models in healthcare. Therefore, this research empowers the ability to forecast CD, providing a vital lifeline to millions of CD patients globally and potentially preventing numerous fatalities.

Reservoir properties inversion using attention-based parallel hybrid network integrating feature selection and transfer learning

The inversion of subsurface reservoir properties is of profound significance to the oil and gas energy development and utilization. The strong heterogeneity and complex pore structure of underground reservoirs pose a challenge to efficient oil and gas energy development and utilization across the world, which increases the necessity of developing an efficient reservoir properties inversion method. However, traditional model-driven methods are confronted with the challenges of strong nonlinearity and geological heterogeneity. Moreover, previous studies rarely emphasized the importance of nonlinear feature selection and transfer learning (TL). Aiming to address the research gaps, a reservoir properties inversion method was proposed by combining random forest (RF) feature selection, bidirectional temporal convolutional network (BiTCN), bidirectional gated recurrent units (BiGRU) network, multi-head attention (MHA) mechanism and TL strategy. First, the RF was used to screen the features with significant correlation with the target reservoir properties. Thereafter, combining BiTCN and BiGRU network to leverage their complementary strengths, a parallel dual branch feature learning network was constructed to learn richer geological information from logging data. Meanwhile, MHA was introduced to fuse the output features of the dual network structure. Finally, the fused features were passed through the fully connected module to output the inversion results. TL was used to associate the correlation between reservoir properties and model inversion to improve the inversion performance. The application results with actual field data showed that the proposed method was accurate and robust in reservoir properties inversion. This study can provide a new way for reliable reservoir properties inversion and promote the application of artificial intelligence in data-driven energy science.

Identifying miRNA as biomarker for breast cancer subtyping using association rule

— This paper presents a comprehensive study focused on breast cancer subtyping, utilizing a multifaceted approach that integrates feature selection, machine learning classifiers, and miRNA regulatory networks. The feature selection process begins with the CFS algorithm, followed by the Apriori algorithm for association rule generation, resulting in the identification of significant features tailored to Luminal A, Luminal B, HER-2 enriched, and Basal-like subtypes. The subsequent application of Random Forest (RF) and Support Vector Machine (SVM) classifiers yielded promising results, with the SVM model achieving an overall accuracy of 76.60 % and the RF model demonstrating robust performance at 80.85 %. Detailed accuracy metrics revealed strengths and areas for refinement, emphasizing the potential for optimizing subtype-specific recall. To explore the regulatory landscape in depth, an analysis of selected miRNAs was conducted using MIENTURNET, a tool for visualizing miRNA-target interactions. While FDR analysis raised concerns for HER-2 and Basal-like subtypes, Luminal A and Luminal B subtypes showcased significant miRNA-gene interactions. Functional enrichment analysis for Luminal A highlighted the role of Ovarian steroidogenesis, implicating specific miRNAs such as hsa-let-7c-5p and hsa-miR-125b-5p as potential diagnostic biomarkers and regulators of Luminal A breast cancer. Luminal B analysis uncovered associations with the MAPK signaling pathway, with miRNAs like hsa-miR-203a-3p and hsa-miR-19a-3p exhibiting potential diagnostic and therapeutic significance. In conclusion, this integrative approach combines machine learning techniques with miRNA analysis to provide a holistic understanding of breast cancer subtypes. The identified miRNAs and associated pathways offer insights into potential diagnostic biomarkers and therapeutic targets, contributing to the ongoing efforts to improve breast cancer diagnostics and personalized treatment strategies.

Employing a Convolutional Neural Network to Classify Sleep Stages from EEG Signals Using Feature Reduction Techniques

One of the most essential components of human life is sleep. One of the first steps in spotting abnormalities connected to sleep is classifying sleep stages. Based on the kind and frequency of signals obtained during a polysomnography test, sleep phases can be separated into groups. Accurate classification of sleep stages from electroencephalogram (EEG) signals plays a crucial role in sleep disorder diagnosis and treatment. This study proposes a novel approach that combines feature selection techniques with convolutional neural networks (CNNs) to enhance the classification performance of sleep stages using EEG signals. Firstly, a comprehensive feature selection process was employed to extract discriminative features from raw EEG data, aiming to reduce dimensionality and enhance the efficiency of subsequent classification using mutual information (MI) and analysis of variance (ANOVA) after splitting the dataset into two sets—the training set (70%) and testing set (30%)—then processing it using the standard scalar method. Subsequently, a 1D-CNN architecture was designed to automatically learn hierarchical representations of the selected features, capturing complex patterns indicative of different sleep stages. The proposed method was evaluated on a publicly available EDF-Sleep dataset, demonstrating superior performance compared to traditional approaches. The results highlight the effectiveness of integrating feature selection with CNNs in improving the accuracy and reliability of sleep stage classification from EEG signals, which reached 99.84% with MI-50. This approach not only contributes to advancing the field of sleep disorder diagnosis, but also holds promise for developing more efficient and robust clinical decision support systems.

PlantMine: A Machine-Learning Framework to Detect Core SNPs in Rice Genomics.

As a fundamental global staple crop, rice plays a pivotal role in human nutrition and agricultural production systems. However, its complex genetic architecture and extensive trait variability pose challenges for breeders and researchers in optimizing yield and quality. Particularly to expedite breeding methods like genomic selection, isolating core SNPs related to target traits from genome-wide data reduces irrelevant mutation noise, enhancing computational precision and efficiency. Thus, exploring efficient computational approaches to mine core SNPs is of great importance. This study introduces PlantMine, an innovative computational framework that integrates feature selection and machine learning techniques to effectively identify core SNPs critical for the improvement of rice traits. Utilizing the dataset from the 3000 Rice Genomes Project, we applied different algorithms for analysis. The findings underscore the effectiveness of combining feature selection with machine learning in accurately identifying core SNPs, offering a promising avenue to expedite rice breeding efforts and improve crop productivity and resilience to stress.

Innovative framework for accurate and transparent forecasting of energy consumption: A fusion of feature selection and interpretable machine learning

The study presents a novel framework integrating feature selection (FS) and machine learning (ML) techniques to forecast inland national energy consumption (EC) in the United Kingdom across all energy sources. This innovative framework strategically combines three FS approaches with five interpretable ML models using Shapley Additive Explanations (SHAP), with the dual goal of enhancing accuracy and transparency in EC predictions. By meticulously selecting the most pertinent features from diverse features—including meteorological conditions, socioeconomic parameters, and historical consumption patterns of different primary fuels—the proposed framework enhances the robustness of the forecasting model. This is achieved through benchmarking three FS approaches: ensemble filter, wrapper, and a hybrid ensemble filter-wrapper. In addition, we introduce a novel ensemble filter FS, synthesizing outcomes from multiple base FS methods to make well-informed decisions about feature retention. Experimental results underscore the efficacy of integrating both wrapper and ensemble filter-wrapper FS approaches with interpretable ML models, ensuring the forecasting process remains comprehensible and interpretable while utilizing a manageable number of features (four to eight). In addition, experimental results indicate that different feature subsets are usually selected for each combined FS approach and ML model. This study not only demonstrates the framework's capability to provide accurate forecasts but also establishes it as a valuable tool for policymakers and energy analysts.

Using machine learning algorithms based on patient admission laboratory parameters to predict adverse outcomes in COVID-19 patients

Amidst the global COVID-19 pandemic, the urgent need for timely and precise patient prognosis assessment underscores the significance of leveraging machine learning techniques. In this study, we present a novel predictive model centered on routine clinical laboratory test data to swiftly forecast patient survival outcomes upon admission. Our model integrates feature selection algorithms and binary classification algorithms, optimizing algorithmic selection through meticulous parameter control. Notably, we developed an algorithm coupling Lasso and SVM methodologies, achieving a remarkable area under the ROC curve of 0.9277 with the use of merely 8 clinical laboratory parameters collected upon admission. Our primary contribution lies in the utilization of straightforward laboratory parameters for prognostication, circumventing data processing intricacies, and furnishing clinicians with an expeditious and precise prognostic assessment tool.

Assessing Performance Across Various Machine Learning Algorithms with Integrated Feature Selection for Fetal Heart Classification

Assessing Performance Across Various Machine Learning Algorithms with Integrated Feature Selection for Fetal Heart Classification

Integrated Feature Selection and Hyperparameter Optimization for Multi-Label Classification of Medical Conditions

Integrated Feature Selection and Hyperparameter Optimization for Multi-Label Classification of Medical Conditions

Enhancement in the safety and reliability of Pressurized Water reactors using Machine Learning approach

The increasing interest in artificial intelligence and automation within the nuclear industry stems from the hope of elevating the safety and reliability of nuclear power plants by minimizing the impact of human errors. This paper explores the capabilities of machine-learning based fault detection and diagnosis (FDD) models in accurately classifying transient events in a nuclear power plant using the Classification Learners Application from MATLAB. The scope of the research is narrowed to the identification of transient events of the same nature occurring at various locations within the plant. To accomplish this, the Generic Pressurized Water Reactor (GPWR) from the Western Services Cooperation (WSC) was used to simulate 14 transients in the primary system and 10 transients in the secondary system, resulting in two distinct datasets of 108,000 and 79,200 observations. Nine types of classifiers, including Decision Trees, Discriminant Analysis, Support Vector Machines, Logistic Regression, Nearest Neighbors, Naive Bayes, Kernel Approximation, Ensembles, and Neural Networks, with a total of 33 predefined models, were trained and validated. The K-Nearest Neighbors model achieved the highest accuracy of 97% in the primary system, while the Efficient Linear Discriminant and Logistic Regression models achieved the highest average accuracy of 99% in the secondary system. The assessment of the top-performing models was conducted through a comprehensive analysis employing key classification model evaluation metrics, such as the confusion matrix, accuracy, precision, recall, and the F1 score. In an effort to optimize the models, the study integrated feature selection using the minimum Redundancy Maximum Relevance (mRMR) algorithm. Additionally, the investigation explored different validation schemes, including both holdout and k-fold cross-validation schemes, resulting in a substantial reduction in training time without significantly compromising the overall accuracy of the models. The optimized models in this study demonstrated remarkable prediction accuracies, consistently exceeding 94%. Despite the complexity of the models and the intricacies involved in the training process, the training times ranged from 10 to 1800 s, reflecting the efficiency and robustness of the implemented optimization techniques. The results of this study underscore the potential of machine learning models in identifying transient events in nuclear reactors, showcasing their promising capabilities while maintaining low computational and execution costs. This suggests their efficacy in optimizing safe and efficient operational practices within nuclear facilities.

Partial discharge pattern recognition algorithm of overhead covered conductors based on feature optimization and bidirectional LSTM‐GRU

AbstractRecognition of partial discharge (PD) patterns is essential for insulation diagnosis of covered conductors in overhead lines. Current research has not sufficiently addressed the complex background noise in real environments, and most detection methods depend primarily on feature engineering or deep learning, suggesting potential for improvement in accuracy and efficiency. This has led the authors to propose a PD pattern recognition algorithm that integrates feature selection and deep learning. This algorithm incorporates the design of a discrete wavelet denoising function specifically tailored to the characteristics of PD for data preprocessing. It employs Bayesian optimization algorithms and light gradient boosting machines for characterizing corona discharge features. Furthermore, it develops multi‐scale clustering features and phase‐resolved features for feature fusion, and constructs insightful features based on the light gradient boosting machine. Finally, a novel deep learning model is formulated, demonstrating exceptional detection performance for early faults in covered conductors. Experimental results show that this algorithm attains an Matthews correlation coefficient score of 0.814, a 13.2% improvement over the baseline algorithm's 0.719, and a speed increase of 39.18%. The final accuracy amounts to 97.85%. This algorithm demonstrates exceptional performance in detecting early insulation faults in conductors.

A robust intrusion detection system based on a shallow learning model and feature extraction techniques.

The escalating prevalence of cybersecurity risks calls for a focused strategy in order to attain efficient resolutions. This study introduces a detection model that employs a tailored methodology integrating feature selection using SHAP values, a shallow learning algorithm called PV-DM, and machine learning classifiers like XGBOOST. The efficacy of our suggested methodology is highlighted by employing the NSL-KDD and UNSW-NB15 datasets. Our approach in the NSL-KDD dataset exhibits exceptional performance, with an accuracy of 98.92%, precision of 98.92%, recall of 95.44%, and an F1-score of 96.77%. Notably, this performance is achieved by utilizing only four characteristics, indicating the efficiency of our approach. The proposed methodology achieves an accuracy of 82.86%, precision of 84.07%, recall of 77.70%, and an F1-score of 80.20% in the UNSW-NB15 dataset, using only six features. Our research findings provide substantial evidence of the enhanced performance of the proposed model compared to a traditional deep-learning model across all performance metrics.

Robust Ranking Kernel Support Vector Machine via Manifold Regularized Matrix Factorization for Multi-Label Classification

Multi-label classification has been extensively researched and utilized for several decades. However, the performance of these methods is highly susceptible to the presence of noisy data samples, resulting in a significant decrease in accuracy when noise levels are high. To address this issue, we propose a robust ranking support vector machine (Rank-SVM) method that incorporates manifold regularized matrix factorization. Unlike traditional Rank-SVM methods, our approach integrates feature selection and multi-label learning into a unified framework. Within this framework, we employ matrix factorization to learn a low-rank robust subspace within the input space, thereby enhancing the robustness of data representation in high-noise conditions. Additionally, we incorporate manifold structure regularization into the framework to preserve manifold relationships among low-rank samples, which further improves the robustness of the low-rank representation. Leveraging on this robust low-rank representation, we extract a resilient low-rank features and employ them to construct a more effective classifier. Finally, the proposed framework is extended to derive a kernelized ranking approach, for the creation of nonlinear multi-label classifiers. To effectively solve this non-convex kernelized method, we employ the augmented Lagrangian multiplier (ALM) and alternating direction method of multipliers (ADMM) techniques to obtain the optimal solution. Experimental evaluations conducted on various datasets demonstrate that our framework achieves superior classification results and significantly enhances performance in high-noise scenarios.

Unsupervised Discriminative Feature Selection via Contrastive Graph Learning.

Due to many unmarked data, there has been tremendous interest in developing unsupervised feature selection methods, among which graph-guided feature selection is one of the most representative techniques. However, the existing feature selection methods have the following limitations: (1) All of them only remove redundant features shared by all classes and neglect the class-specific properties; thus, the selected features cannot well characterize the discriminative structure of the data. (2) The existing methods only consider the relationship between the data and the corresponding neighbor points by Euclidean distance while neglecting the differences with other samples. Thus, existing methods cannot encode discriminative information well. (3) They adaptively learn the graph in the original or embedding space. Thus, the learned graph cannot characterize the data's cluster structure. To solve these limitations, we present a novel unsupervised discriminative feature selection via contrastive graph learning, which integrates feature selection and graph learning into a uniform framework. Specifically, our model adaptively learns the affinity matrix, which helps characterize the data's intrinsic and cluster structures in the original space and the contrastive learning. We minimize l1,2 -norm regularization on the projection matrix to preserve class-specific features and remove redundant features shared by all classes. Thus, the selected features encode discriminative information well and characterize the discriminative structure of the data. Generous experiments indicate that our proposed model has state-of-the-art performance.