Ensemble Feature Research Articles

Degradation index‐based prediction for remaining useful life using multivariate sensor data

AbstractThe prediction of remaining useful life (RUL) is a critical component of prognostic and health management for industrial systems. In recent decades, there has been a surge of interest in RUL prediction based on degradation data of a well‐defined degradation index (DI). However, in many real‐world applications, the DI may not be readily available and must be constructed from complex source data, rendering many existing methods inapplicable. Motivated by multivariate sensor data from industrial induction motors, this paper proposes a novel prognostic framework that develops a nonlinear DI, serving as an ensemble of representative features, and employs a similarity‐based method for RUL prediction. The proposed framework enables online prediction of RUL by dynamically updating information from the in‐service unit. Simulation studies and a case study on three‐phase industrial induction motors demonstrate that the proposed framework can effectively extract reliability information from various channels and predict RUL with high accuracy.

A privacy-preserving platform oriented medical healthcare and its application in identifying patients with candidemia

Federated learning (FL) has emerged as a significant method for developing machine learning models across multiple devices without centralized data collection. Candidemia, a critical but rare disease in ICUs, poses challenges in early detection and treatment. The goal of this study is to develop a privacy-preserving federated learning framework for predicting candidemia in ICU patients. This approach aims to enhance the accuracy of antifungal drug prescriptions and patient outcomes. This study involved the creation of four predictive FL models for candidemia using data from ICU patients across three hospitals in China. The models were designed to prioritize patient privacy while aggregating learnings across different sites. A unique ensemble feature selection strategy was implemented, combining the strengths of XGBoost’s feature importance and statistical test p values. This strategy aimed to optimize the selection of relevant features for accurate predictions. The federated learning models demonstrated significant improvements over locally trained models, with a 9% increase in the area under the curve (AUC) and a 24% rise in true positive ratio (TPR). Notably, the FL models excelled in the combined TPR + TNR metric, which is critical for feature selection in candidemia prediction. The ensemble feature selection method proved more efficient than previous approaches, achieving comparable performance. The study successfully developed a set of federated learning models that significantly enhance the prediction of candidemia in ICU patients. By leveraging a novel feature selection method and maintaining patient privacy, the models provide a robust framework for improved clinical decision-making in the treatment of candidemia.

A refined approach to early movie box office prediction leveraging ensemble learning and feature encoding

Predicting the revenue of a movie prior to its release presents a significant challenge. The ability to predict pre-release revenue enables movie production companies to devise effective marketing strategies and mitigate the risks associated with potential box office failures. The primary hurdles in this endeavor stem from managing the myriad factors influencing box office outcomes and accurately forecasting a movie's revenue before it becomes available to the public. To overcome these challenges, we introduce a sophisticated Early Movie Box Office Prediction Model that incorporates Ensemble Learning and Feature Encoding techniques. This model amalgamates multiple foundational models, utilizing regression and decision trees to forecast box office revenues. Our composite model demonstrates superior performance over its constituent models, achieving an impressive accuracy rate of 91.4%.

DNN-BP: a novel framework for cuffless blood pressure measurement from optimal PPG features using deep learning model.

Continuous blood pressure (BP) provides essential information for monitoring one's health condition. However, BP is currently monitored using uncomfortable cuff-based devices, which does not support continuous BP monitoring. This paper aims to introduce a blood pressure monitoring algorithm based on only photoplethysmography (PPG) signals using the deep neural network (DNN). The PPG signals are obtained from 125 unique subjects with 218 records and filtered using signal processing algorithms to reduce the effects of noise, such as baseline wandering, and motion artifacts. The proposed algorithm is based on pulse wave analysis of PPG signals, extracted various domain features from PPG signals, and mapped them to BP values. Four feature selection methods are applied and yielded four feature subsets. Therefore, an ensemble feature selection technique is proposed to obtain the optimal feature set based on major voting scores from four feature subsets. DNN models, along with the ensemble feature selection technique, outperformed in estimating the systolic blood pressure (SBP) and diastolic blood pressure (DBP) compared to previously reported approaches that rely only on the PPG signal. The coefficient of determination ( ) and mean absolute error (MAE) of the proposed algorithm are 0.962 and 2.480 mmHg, respectively, for SBP and 0.955 and 1.499 mmHg, respectively, for DBP. The proposed approach meets the Advancement of Medical Instrumentation standard for SBP and DBP estimations. Additionally, according to the British Hypertension Society standard, the results attained Grade A for both SBP and DBP estimations. It concludes that BP can be estimated more accurately using the optimal feature set and DNN models. The proposed algorithm has the potential ability to facilitate mobile healthcare devices to monitor continuous BP.

Homogeneous Ensemble Feature Selection for Mass Spectrometry Data Prediction in Cancer Studies

The identification of important proteins is critical for the medical diagnosis and prognosis of common diseases. Diverse sets of computational tools have been developed for omics data reduction and protein selection. However, standard statistical models with single-feature selection involve the multi-testing burden of low power with limited available samples. Furthermore, high correlations among proteins with high redundancy and moderate effects often lead to unstable selections and cause reproducibility issues. Ensemble feature selection in machine learning (ML) may identify a stable set of disease biomarkers that could improve the prediction performance of subsequent classification models and thereby simplify their interpretability. In this study, we developed a three-stage homogeneous ensemble feature selection (HEFS) approach for both identifying proteins and improving prediction accuracy. This approach was implemented and applied to ovarian cancer proteogenomics datasets comprising (1) binary putative homologous recombination deficiency (HRD)- positive or -negative samples; (2) multiple mRNA classes (differentiated, proliferative, immunoreactive, mesenchymal, and unknown samples). We conducted and compared various ML methods with HEFS including random forest (RF), support vector machine (SVM), and neural network (NN) for predicting both binary and multiple-class outcomes. The results indicated that the prediction accuracies varied for both binary and multiple-class classifications using various ML approaches with the proposed HEFS method. RF and NN provided better prediction accuracies than simple Naive Bayes or logistic models. For binary outcomes, with a sample size of 122 and nine selected prediction proteins using our proposed three-stage HEFS approach, the best ensemble ML (Treebag) achieved 83% accuracy, 85% sensitivity, and 81% specificity. For multiple (five)-class outcomes, the proposed HEFS-selected proteins combined with Principal Component Analysis (PCA) in NN resulted in prediction accuracies for multiple-class classifications ranging from 75% to 96% for each of the five classes. Despite the different prediction accuracies of the various models, HEFS identified consistent sets of proteins linked to the binary and multiple-class outcomes.

Optimizing hybrid ensemble feature selection strategies for transcriptomic biomarker discovery in complex diseases.

Biomedical research takes advantage of omic data, such as transcriptomics, to unravel the complexity of diseases. A conventional strategy identifies transcriptomic biomarkers characterized by expression patterns associated with a phenotype by relying on feature selection approaches. Hybrid ensemble feature selection (HEFS) has become increasingly popular as it ensures robustness of the selected features by performing data and functional perturbations. However, it remains difficult to make the best suited choices at each step when designing such approaches. We conducted an extensive analysis of four possible HEFS scenarios for the identification of Stage IV colorectal, Stage I kidney and lung and Stage III endometrial cancer biomarkers from transcriptomic data. These scenarios investigate the use of two types of feature reduction by filters (differentially expressed genes and variance) conjointly with two types of resampling strategies (repeated holdout by distribution-balanced stratified and random stratified) for downstream feature selection through an aggregation of thousands of wrapped machine learning models. Based on our results, we emphasize the advantages of using HEFS approaches to identify complex disease biomarkers, given their ability to produce generalizable and stable results to both data and functional perturbations. Finally, we highlight critical issues that need to be considered in the design of such strategies.

A Review of: Ensemble Feature Selection Scheme-Based Performance Evaluation of Several Classifiers for Sentiment Analysis

A Review of: Ensemble Feature Selection Scheme-Based Performance Evaluation of Several Classifiers for Sentiment Analysis

Evaluate the Sentiment Analysis Performance of Several Classifiers Using Ensemble Feature Selection Method

Evaluate the Sentiment Analysis Performance of Several Classifiers Using Ensemble Feature Selection Method

Oligomer Formation by Physiologically Relevant C-Terminal Isoforms of Amyloid β-Protein.

Alzheimer's disease (AD) is a neurological disorder associated with amyloid β-protein (Aβ) assembly into toxic oligomers. In addition to the two predominant alloforms, Aβ1-40 and Aβ1-42, other C-terminally truncated Aβ peptides, including Aβ1-38 and Aβ1-43, are produced in the brain. Here, we use discrete molecular dynamics (DMD) and a four-bead protein model with amino acid-specific hydropathic interactions, DMD4B-HYDRA, to examine oligomer formation of Aβ1-38, Aβ1-40, Aβ1-42, and Aβ1-43. Self-assembly of 32 unstructured monomer peptides into oligomers is examined using 32 replica DMD trajectories for each of the four peptides. In a quasi-steady state, Aβ1-38 and Aβ1-40 adopt similar unimodal oligomer size distributions with a maximum at trimers, whereas Aβ1-42 and Aβ1-43 oligomer size distributions are multimodal with the dominant maximum at trimers or tetramers, and additional maxima at hexamers and unidecamers (for Aβ1-42) or octamers and pentadecamers (for Aβ1-43). The free energy landscapes reveal isoform- and oligomer-order specific structural and morphological features of oligomer ensembles. Our results show that oligomers of each of the four isoforms have unique features, with Aβ1-42 alone resulting in oligomers with disordered and solvent-exposed N-termini. Our findings help unravel the structure-function paradigm governing oligomers formed by various Aβ isoforms.

Evaluating and Comparing AI Models for Hourly Energy Demand Prediction

This study investigates the application of various AI models to predict energy demand, comparing the performance of four specific models: Decision Tree, Random Forest, Gradient Boosting, and Linear Regression. The evaluation of these models' prediction performance reveals that ensemble methods like Random Forest and Gradient Boosting exhibit promising generalization capabilities, while the Decision Tree model shows high training accuracy but suffers from overfitting. The discussion underscores the importance of ensemble techniques and feature engineering optimization in mitigating overfitting and enhancing forecast accuracy. Furthermore, the study highlights the potential of AI-driven approaches to promote sustainability and resilience in energy systems, emphasizing the need for further optimization and collaboration among stakeholders to achieve a cleaner, more sustainable energy future.

Student Opinion Mining About Instructor Using Optimized Ensemble Machine Learning Model and Feature Fusion

Student Opinion Mining About Instructor Using Optimized Ensemble Machine Learning Model and Feature Fusion

Improving Code Smell Detection by Reducing Dimensionality Using Ensemble Feature Selection and Machine Learning

Improving Code Smell Detection by Reducing Dimensionality Using Ensemble Feature Selection and Machine Learning

PRFold-TNN: Protein Fold Recognition With an Ensemble Feature Selection Method Using PageRank Algorithm Based on Transformer.

Understanding the tertiary structures of proteins is of great benefit to function in many aspects of human life. Protein fold recognition is a vital and salient means to know protein structure. Until now, researchers have successively proposed a variety of methods to realize protein fold recognition, but the novel and effective computational method is still needed to handle this problem with the continuous updating of protein structure databases. In this study, we develop a new protein structure dataset named AT and propose the PRFold-TNN model for protein fold recognition. Firstly, different types of feature extraction methods including AAC, HMM, HMM-Bigram and ACC are selected to extract corresponding features for protein sequences. Then an ensemble feature selection method based on PageRank algorithm integrating various tree-based algorithms is used to screen the fusion features. Ultimately, the classifier based on the Transformer model achieves the final prediction. Experiments show that the prediction accuracy is 86.27% on the AT dataset and 88.91% on the independent test set, indicating that the model can demonstrate superior performance and generalization ability in the problem of protein fold recognition. Furthermore, we also carry out research on the DD, EDD and TG benchmark datasets, and make them achieve prediction accuracy of 88.41%, 97.91% and 95.16%, which are at least 3.0%, 0.8% and 2.5% higher than those of the state-of-the-art methods. It can be concluded that the PRFold-TNN model is more prominent.

Improving the performance and interpretability on medical datasets using graphical ensemble feature selection.

A major hindrance towards using Machine Learning (ML) on medical datasets is the discrepancy between a large number of variables and small sample sizes. While multiple feature selection techniques have been proposed to avoid the resulting overfitting, overall ensemble techniques offer the best selection robustness. Yet, current methods designed to combine different algorithms generally fail to leverage the dependencies identified by their components. Here, we propose Graphical Ensembling (GE), a graph-theory-based ensemble feature selection technique designed to improve the stability and relevance of the selected features. Relying on four datasets, we show that GE increases classification performance with fewer selected features. For example, on rheumatoid arthritis patient stratification, GE outperforms the baseline methods by 9% Balanced Accuracy while relying on fewer features. We use data on sub-cellular networks to show that the selected features (proteins) are closer to the known disease genes, and the uncovered biological mechanisms are more diversified. By successfully tackling the complex correlations between biological variables, we anticipate that GE will improve the medical applications of ML. https://github.com/ebattistella/auto_machine_learning.

Ensemble multi-view feature set partitioning method for effective multi-view learning

Ensemble multi-view feature set partitioning method for effective multi-view learning

Hot rolled steel surface defect detection and classification using an automatic ensemble approach

This study introduces an ensemble-based Deep Neural Network (DNN) model for detecting defects on steel surfaces. The method suggested in this study classifies steel surface conditions into six possible fault categories, namely, crazing, inclusion, rolled in, pitted surface, scratches, and patches. The images undergo preprocessing and extraction of features in spatial and frequency domains using image segmentation techniques such as grey level difference method (GLDM), fast Fourier Transform (FFT), grey level co-occurrence matrix (GLCM), texture analysis and discrete wavelet transform (DWT). The ensembling of image features into a fused feature pool is carried out after the preprocessing of input images that are provided as input to a light-weight neural network model for training and testing. The performance of the model is comprehensively evaluated via an ablation study both before and after ensembling. In addition, the model capability is effectively analyzed using receiver operating characteristics (ROC) curve, confusion matrix from which classification accuracy of the model could be obtained and other parameters including precision and f1-score. It was observed that the proposed deep learning network presents phenomenally high accuracy of 99.72% for detection and classification of steel surface faults. This result was found to be superior when compared with the performance of the same neural network over each feature type individually. This study also compares the classification results of the model built based on the ensembled feature set with the results of various other classification approaches available in literature. The ensemble-based model could potentially be integrated into existing inspection systems for real-time, efficient and robust condition monitoring of steel surfaces.

Ensemble feature selection using q-rung orthopair hesitant fuzzy Hamacher, Einstein and Dombi Aggregation operators

The article aims in addressing the issue of ensemble feature selection problem by modeling it as a multi-criteria decision making technique. To build such a model, initially, aggregation operators such as weighted arithmetic, weighted geometric, ordered weighted arithmetic, ordered weighted geometric aggregation of Hamacher, Einstein and Dombi operators in the q-rung orthopair hesitant fuzzy environment are proposed. The properties of these operators are also discussed to provide a more elaborate understanding of them. Such an approach to ensemble feature selection has not yet been carried out in the literature which adds to the novelty of our work. Validation is provided through comparison of the performance metrics with existing and base feature selection methods and also by carrying out statistical tests. Through this article, a model for ensemble feature selection incorporating the advantages of Einstein, Hamacher, Dombi and q-rung orthopair hesitant fuzzy set was constructed which was reflected in the results.

ELIDS: Ensemble Feature Selection for Lightweight IDS against DDoS Attacks in Resource-Constrained IoT Environment

Distributed Denial of Service (DDoS) attacks are increasingly prevalent in today’s digital era, particularly due to the growth of unprotected Internet of Things (IoT) devices. This escalation in attacks necessitates the development of adaptive Intrusion Detection Systems (IDS) that are both efficient in detection and less resource intensive, suitable for resource-constraint IoT devices. Feature Selection (FS) techniques are most popular for developing efficient and lightweight IDS, but their reliance on a single method can be risky due to inherent dataset biasness, necessitating diverse and adaptable FS approaches. This paper proposes a new Ensemble FS approach, termed as Ensemble Feature Selection for Lightweight IDS (ELIDS), by leveraging the strength of seven different filter-based methods. ELIDS not only reduces features but also converges the selection towards the most significant features out of those identified by the individual FS methods. Considering several learning algorithms, robust classification models are built out of ELIDS that are comprehensively evaluated for performance and resumption through both in-domain and cross-domain testing. During in-domain testing, results indicate that classification models built out of ELIDS generally exhibit similar accuracy to those built out of the existing individual FS techniques, where peak accuracy rates of 99.8% are achieved. However, cross-domain testing reveals that classifiers built using individual FS methods fall noticeably in their accuracy whereas the proposed ELIDS show robustness by outperforming existing techniques by large margins. In-fact, ELIDS exhibits a higher accuracy rate by at-least 23.7% over existing solutions while considering Random Forest (RF) as the learning algorithm.

A physics-and-data co-driven material design strategy for multicomponent diboride ceramics

The vast chemical composition space of multicomponent ceramics provides opportunity for regulating their properties and performances. Other than trial-and-error and screening, a physics-and-data co-driven material design strategy was reported to efficiently explore chemical composition space through establishing prediction model between physical properties and chemical composition via physics-guided data mining method. Firstly, causal and correlative relationships were discriminated according to physicochemical knowledge. Secondly, ensemble features such as chemical formula and chemical bond were digitized by constructing descriptors for crystalline solids with atomic features including atomic weight and radius contained in the Periodic Table of elements. Finally, quantitative causal relations between material structure/properties and ensemble descriptors were mined out of small dataset using a least square regression method, on which physical prediction model established therefore for forward predicting structure/properties and inverse designing chemical composition. Choosing diboride ceramics as a case, those causal analytic relations of composition-structure and composition-property were successfully derived and illustrated to predict the value of structure and properties accompanying with quantitative relation between chemical pressure and chemical bonding ensemble descriptors. This method bridges the connection between atomic features and macroscopic properties via ensemble descriptors, which significantly accelerates the discovery of new materials.

Enhanced Bearing Fault Diagnosis Through Trees Ensemble Method and Feature Importance Analysis

Abstract Purpose This research introduces a groundbreaking method for bearing defect detection. It leverages ensemble machine learning (ML) models and conducts comprehensive feature importance analysis. The key innovation is the training and benchmarking of three tree ensemble models—Decision Tree (DT), Random Forest (RF), and Extreme Gradient Boosting (XGBoost)—on an extensive experimental dataset (QU-DMBF) collected from bearing tests with seeded defects of varying sizes on the inner and outer raceways under different operating conditions. Method The dataset was meticulously prepared with categorical variable encoding and Min–Max data normalization to ensure consistent class distribution and model accuracy. Implementing the ML models involved a grid search method for hyperparameter tuning, focusing on reporting the models’ accuracy. The study also explores applying ensemble methods and using supervised and unsupervised learning algorithms for bearing fault detection. It underscores the value of feature importance analysis in understanding the contributions of specific inputs to the model’s performance. The research compares the ML models to traditional methods and discusses their potential for advanced fault diagnosis in bearing systems. Results and Conclusions The XGBoost model, trained on data from actual bearing tests, outperformed the others, achieving 92% accuracy in detecting bearing health and fault location. However, a deeper analysis of feature importance reveals that the models weigh certain experimental conditions differently—such as sensor location and motor speed. This research’s primary novelties and contributions are comparative evaluation, experimental validation, accuracy benchmarking, and interpretable feature importance analysis. This comprehensive methodology advances the bearing health monitoring field and has significant practical implications for condition-based maintenance, potentially leading to substantial cost savings and improved operational efficiency.