Ensemble Learning Research Articles

Development of criteria to optimize manual smear review of automated complete blood counts using a machine learning model.

In this study, we aim to determine if machine learning can reduce manual smear review (MSR) rates while meeting or exceeding the performance of traditional MSR criteria. 9938 automated CBCs with paired MSRs were performed on samples from rhesus and cynomolgus macaques. The definition of a positive (abnormal) smear was determined. Two expert-derived MSR criteria were created: criteria adapted from published, standardized human laboratory criteria (Adapted International Consensus Guidelines[aICG]) and internally generated criteria (Center Consensus Guidelines[CCG]). An ensemble machine learning model was trained on an independent subset of the data to optimize the balanced accuracy of classification, a combined measure of sensitivity and specificity. The resulting machine learning model and the two expert-derived MSR criteria were applied to a test dataset, and their performance compared. aICG criteria demonstrated high sensitivity (80.8%) and MSR rate (74.2%) while CCG criteria demonstrated lower sensitivity (57.1%) and MSR rate (36.1%). The machine learning model integrated with CCG criteria had a superior combination of both sensitivity (76.8%) and MSR rate (45.1%) achieving a false negative rate of 1.6%. Machine learning in combination with expert-derived criteria can optimize the selection of samples for MSR thus decreasing MSR rates and labor efforts required for CBC performance.

Nanobody–antigen interaction prediction with ensemble deep learning and prompt-based protein language models

Nanobody–antigen interaction prediction with ensemble deep learning and prompt-based protein language models

Exploring Thematic Diversity in Classical Chinese Poetry: A Novel Dataset and a BERT-Enhanced Ensemble Learning Approach

Classical Chinese poetry, as an essential aspect of cultural heritage, exhibits rich theme diversity often overlooked in natural language processing research. To address this gap, we aim to explore the classification of thematic categories within this literary domain. We curate a dataset of 2,918 annotated poems spanning 7 common themes and propose a BERT-based ensemble learning approach for effective classification. Although this method integrates existing models, it achieves an accuracy and F1 score of over 72% in the 7-class task, surpassing established baselines, and providing a baseline for future research. The experimental findings reveal the effectiveness of ensemble strategies in improving individual base model performance and highlight the potential of the MLP-based ensemble technique. The study contributes to a deeper understanding of thematic categories and textual features in classical Chinese poetry and offers an automated classification system for classical Chinese poems.

Heterogeneous knowledge graph-driven subassembly identification with ensemble deep learning in Industry 4.0

In the context of Industry 4.0, model-based definition (MBD) has been an effective approach to creating 3D models contained all heterogeneous information needed to define a product, which proposes new challenges for the traditional subassembly identification method that only considers the geometric information of a product in assembly sequence planning. To bridge the gap, we propose a novel heterogeneous knowledge graph-driven subassembly identification method to enhance assembly sequence planning in the model-based systems engineering (MBSE) paradigm. Specifically, a heterogeneous knowledge graph is first constructed based on the shape information and engineering details of an MBD model. Next, an ensemble deep learning approach that combines graph neural networks with the community detection algorithm is proposed to effectively detect the subassembly from the MBD model. Finally, the feasibility and effectiveness of the proposed method are demonstrated through an example of car suspension subassembly identification, providing insight into the industrial implementation.

One score to rule them all: regularized ensemble polygenic risk prediction with GWAS summary statistics.

Ensemble learning has been increasingly popular for boosting the predictive power of polygenic risk scores (PRS), with almost every recent multi-ancestry PRS approach employing ensemble learning as a final step. Existing ensemble approaches rely on individual-level data for model training, which severely limits their real-world applications, especially in non-European populations without sufficient genomic samples. Here, we introduce a statistical framework to construct regularized ensemble PRS, which allows us to combine a large number of candidate PRS models using only summary statistics from genome-wide association studies. We demonstrate its robust and substantial improvement over many existing PRS models in both within- and cross-ancestry applications. We believe this is truly "one score to rule them all" due to its capability to continuously combine newly developed PRS models with existing models to improve prediction performance, which makes it a universal approach that should always be employed in future PRS applications.

Enhancing groundwater quality prediction through ensemble machine learning techniques.

Groundwater quality is assessed by conducting water sampling and laboratory analysis. Field-based measurements are costly and time-consuming. This study introduces a machine learning (ML)-based framework and innovative application of stacking ensemble learning model, for predicting groundwater quality in an unconfined aquifer located in northern Iran. The groundwater quality index (GWQI) from 250 wells was evaluated and classified. We considered various influential factors such as proximity to residential areas, evaporation, aquifer transmissivity, precipitation values, population density, distance to industrial centers, distance to water resources, and topography. Three different ML classifiers were employed to establish relationships between GWQI and the aforementioned factors: the AdaBoost classifier (ADA), quadratic discriminant analysis (QDA), and stacking ensemble learning (SEL). A novel model was introduced dubbed quadratic-ada-stacking ensemble learning (QA-SEL) to predict GWQI. The performance of these algorithms was evaluated through the receiver-operating characteristic (ROC) and multiple statistical efficiency indicators, including overall accuracy, precision, recall, and the F-1 score. All three ML algorithms displayed a high degree of accuracy in their GWQI predictions. Nonetheless, the QA-SEL method was identified as the most effective model due to its superior accuracy (overall accuracy, precision, recall = 0.95, 0.95, 0.96, ROC = 0.96, respectively). Following model optimization and testing, the QA-SEL model and a GIS were employed to map GWQI classes across the entire area. The produced GWQI map was validated by comparing the measured and predicted GWQI on the map. This study offers an economically efficient model for groundwater quality prediction, which can be replicated in other plains.

An enhanced deep intelligent model with feature fusion and ensemble learning for the fault diagnosis of rotating machinery

Vibration signals, serving as critical sources of information for monitoring the status of rotating machinery, demand effective extraction and rational utilization of its features to significantly enhance the accuracy and reliability of fault diagnosis. However, vibration signal features typically manifest as nonlinear and nonstationary, posing a significant challenge in industrial settings. To tackle this challenge, this article proposes an enhanced deep intelligent model based on feature fusion and ensemble learning for practical fault diagnosis of rotating machinery. First, a parallel network structure is introduced to comprehensively and accurately explore the fault characteristics of rotating machinery. This network comprises two branches: the first branch designs an improved one-dimensional convolutional neural network to extract locally robust features from raw signals; the second branch adopts variational mode decomposition to decompose raw signals into a set of intrinsic mode functions and extract comprehensive statistical features in both the time and frequency domains, significantly enhancing the signal representation capability. Subsequently, a deep neural network is used to extract more stable feature information. The features from the two branches are then fused, and the final network output is generated through a softmax regression function. Finally, ensemble learning uses a majority voting scheme to obtain more stable final outputs. To confirm the effectiveness of the proposed method, experiments are conducted on two laboratory cases and one industrial case. The experimental results demonstrate that the proposed method significantly improves fault diagnosis accuracy and reliability in controlled laboratory environments and real-world industrial applications, making it highly applicable for real-time monitoring and predictive maintenance of industrial machinery. These improvements can reduce maintenance costs and downtime, thus enhancing operational efficiency in various industrial settings.

NkEL: nearest k-labelsets ensemble for multi-label learning

NkEL: nearest k-labelsets ensemble for multi-label learning

Prediction of prolonged mechanical ventilation in the intensive care unit via machine learning: a COVID-19 perspective

Early recognition of risk factors for prolonged mechanical ventilation (PMV) could allow for early clinical interventions, prevention of secondary complications such as nosocomial infections, and effective triage of hospital resources. This study tested the hypothesis that an ensemble machine learning (ML) analysis of clinical data at time of intubation could identify patients at risk of PMV, using a COVID-19 dataset to classify patients into PMV (> 14 days) and non-PMV (≤ 14 days) groups. While several factors are known to cause PMV, including acid-base, weakness, and delirium, lesser-utilized but routinely measured parameters such as platelet count, glucose levels and fevers may also be relevant. Patient data from a single University Hospital were analyzed via the ML workflow to predict patients at risk of PMV and identify key clinical markers. Model performance was evaluated on a chronologically distinct cohort. The ML workflow identified patients at risk of PMV with AUROCTRAIN=0.960 (F1TRAIN = 0.935) and AUROCTEST=0.804 (F1TEST = 0.800). Top key features for classification included glucose, platelet count, temperature, LVEF, bicarbonate (arterial blood gas), and BMI. Data analysis at intubation time via the proposed workflow offers the potential to accurately predict patients at risk of PMV, with the goal to improve patient management and triage of hospital resources.

Surrounding Rock Squeezing Classification in Underground Engineering Using a Hybrid Paradigm of Generative Artificial Intelligence and Deep Ensemble Learning

Surrounding rock squeezing is a common geological disaster in underground excavation projects (e.g., TBM tunneling and deep mining), which has adverse effects on construction safety, schedule, and property. To predict the squeezing of the surrounding rock accurately and quickly, this study proposes a hybrid machine learning paradigm that integrates generative artificial intelligence and deep ensemble learning. Specifically, conditional tabular generative adversarial network is devised to solve the problems of data shortage and class imbalance for data augmentation at the data level, and the deep random forest is built based on the augmented data for subsequent squeezing classification. A total of 139 historical squeezing cases are collected worldwide to validate the efficacy of the proposed modeling paradigm. The results reveal that this paradigm achieves a prediction accuracy of 92.86% and a macro F1-score of 0.9292. In particular, the individual F1-scores on strong squeezing and extremely strong squeezing are more than 0.9, with excellent prediction reliability for high-intensity squeezing. Finally, a comparative analysis with traditional machine learning techniques is conducted and the superiority of this paradigm is further verified. This study provides a valuable reference for surrounding rock squeezing classification under a limited data environment.

Enhanced Semantic Natural Scenery Retrieval System Through Novel Dominant Colour and Multi‐Resolution Texture Feature Learning Model

ABSTRACTA conventional content‐based image retrieval system (CBIR) extracts image features from every pixel of the images, and its depiction of the feature is entirely different from human perception. Additionally, it takes a significant amount of time for retrieval. An optimal combination of appropriate image features is necessary to bridge the semantic gap between user queries and retrieval responses. Furthermore, users should require minimal interactions with the CBIR system to obtain accurate responses. Therefore, the proposed work focuses on extracting highly relevant feature information from a set of images in various natural image databases. Subsequently, a feature‐based learning/classification model is introduced before similarity measure calculations, aiming to minimise retrieval time and the number of comparisons. The proposed work analyses the learning models based on the retrieval system's performance separately for the following features: (i) dominant colour, (ii) multi‐resolution radial difference texture patterns, and a combination of both. The developed work is assessed with other techniques, and the results are reported. The results demonstrate that the implemented ensemble learning model‐based CBIR outperforms the recent CBIR techniques.

A neighborhood rough sets-based ensemble method, with application to software fault prediction

Software fault prediction (SFP) aims to detect fault-prone software modules, which is beneficial for allocating software testing resources and improving software quality. Recently, ensemble learning(EL)-based SFP methods have attracted much attention. Although many EL algorithms have been applied to SFP, they are still insufficient to generate multiple accurate and diverse base learners. Therefore, this paper presents a multi-modal EL algorithm (called NRSEL) based on neighborhood rough sets. In NRSEL, the technique of neighborhood approximate reduct (NAR) is used to implement the perturbation of attribute space and the bootstrap sampling technique is used to implement the perturbation of sample space. As a novel technique for the perturbation of attribute space, NAR stems from the concept of approximate reduct in rough sets. We also consider the application of NRSEL to SFP, and employ a hybrid scheme (called SMOTE-NRSEL) to handle the problem of imbalanced data in SFP. We compare SMOTE-NRSEL with existing EL algorithms using 20 public datasets. Experimental results indicate that SMOTE-NRSEL is effective for SFP. Compared with the baseline algorithms, on average, SMOTE-NRSEL improves the AUC, F1-score, and MCC by 3.09%, 3.18%, and 7.5%, respectively. Moreover, the results of three statistical tests (including the paired t-test, Friedman test, and Nemenyi test) indicate that SMOTE-NRSEL is significantly better than the baseline algorithms in most cases. This paper shows that NAR is a good choice for the perturbation of attribute space. With the help of NAR and the multi-modal perturbation strategy based on it, SMOTE-NRSEL can generate accurate and diverse base learners. The code is available at https://github.com/jiangfeng0278/NRSEL.

Ensemble Learning with Extremely Randomized k-Nearest Neighbors for Accurate and Efficient Classification

Ensemble Learning with Extremely Randomized k-Nearest Neighbors for Accurate and Efficient Classification

Predicting the Performance of Students Using Deep Ensemble Learning

Universities and schools rely heavily on the ability to forecast student performance, as it enables them to develop efficient strategies for enhancing academic results and averting student attrition. The automation of processes and the management of large datasets generated by technology-enhanced learning tools can facilitate the analysis and processing of these data, which provides crucial insights into the knowledge of students and their engagement with academic endeavors. The method under consideration aims to forecast the academic achievement of students through an ensemble of deep neural networks. The proposed method presents a new feature-ranking mechanism based on existing approaches. This mechanism is effective in identifying the most pertinent features and their correlation with the academic performance of students. The proposed method employs an optimization strategy to concurrently configure and train the deep neural networks within our ensemble system. Furthermore, the proposed ensemble model uses weighted voting among its learning components for more accurate prediction. Put simply, the suggested approach enhances the accuracy of academic performance predictions for students not only by employing weighted ensemble techniques, but also by optimizing the parameters of deep learning models. These experimental outcomes provide evidence that the proposed method outperformed the alternative approaches, accurately predicting student performance with a root-mean-square error (RMSE) value of 1.66, a Mean Absolute Percentage Error (MAPE) value of 9.75, and an R-squared value of 0.7430. These results show a significant improvement compared to the null model (RMSE = 4.05, MAPE = 24.89, and R-squared = 0.2897) and prove the efficiency of the techniques employed in the proposed method.

DETECTION OF DUST ON SOLAR PANELS WITH DEEP LEARNING

Solar energy is an environmentally friendly, clean, and sustainable alternative. The widespread use of this energy source offers excellent environmental and economic benefits. However, some factors affect the efficiency of solar panels. One of these factors is dust. When dust accumulates on the surface of solar panels, it can significantly reduce the efficiency of energy production. Therefore, detecting and quickly removing dust from solar panels is crucial. Managing this process with unmanned artificial intelligence systems, especially in large areas, will provide significant advantages in terms of time and cost. In recent years, convolutional neural networks have achieved significant success in image classification. In particular, transfer learning methods have proven their success in this field. In this study, we aim to solve a new task with limited data using pre-trained deep learning models (EfficientNetB3, ResNet50, MobileNet, VGG19, Xception, InceptionResNetV2, VGG16, ResNet101, DenseNet201, EfficientNetB7) to classify dirty and clean solar panels. These models were chosen because they each have different strengths and have performed well on various tasks. The models with the best performance among these models are combined to improve classification prediction. The proposed ensemble learning approach achieved 99.31% classification accuracy by considering the prediction results of the models with a voting approach. As a result, this approach aims to optimize the maintenance processes of solar energy systems, improve energy efficiency, and support sustainable energy use in the long term.

Exploring vision transformers and XGBoost as deep learning ensembles for transforming carcinoma recognition

Early detection of colorectal carcinoma (CRC), one of the most prevalent forms of cancer worldwide, significantly enhances the prognosis of patients. This research presents a new method for improving CRC detection using a deep learning ensemble with the Computer Aided Diagnosis (CADx). The method involves combining pre-trained convolutional neural network (CNN) models, such as ADaRDEV2I-22, DaRD-22, and ADaDR-22, using Vision Transformers (ViT) and XGBoost. The study addresses the challenges associated with imbalanced datasets and the necessity of sophisticated feature extraction in medical image analysis. Initially, the CKHK-22 dataset comprised 24 classes. However, we refined it to 14 classes, which led to an improvement in data balance and quality. This improvement enabled more precise feature extraction and improved classification results. We created two ensemble models: the first model used Vision Transformers to capture long-range spatial relationships in the images, while the second model combined CNNs with XGBoost to facilitate structured data classification. We implemented DCGAN-based augmentation to enhance the dataset’s diversity. The tests showed big improvements in performance, with the ADaDR-22 + Vision Transformer group getting the best results, with a testing accuracy of 93.4% and an AUC of 98.8%. In contrast, the ADaDR-22 + XGBoost model had an AUC of 97.8% and an accuracy of 92.2%. These findings highlight the efficacy of the proposed ensemble models in detecting CRC and highlight the importance of using well-balanced, high-quality datasets. The proposed method significantly enhances the clinical diagnostic accuracy and the capabilities of medical image analysis or early CRC detection.

MorphoGlia, an interactive method to identify and map microglia morphologies, demonstrates differences in hippocampal subregions of an Alzheimer's disease mouse model.

Microglia are dynamic central nervous system cells crucial for maintaining homeostasis and responding to neuroinflammation, as evidenced by their varied morphologies. Existing morphology analysis often fails to detect subtle variations within the full spectrum of microglial morphologies due to their reliance on predefined categories. Here, we present MorphoGlia, an interactive, user-friendly pipeline that objectively characterizes microglial morphologies. MorphoGlia employs a machine learning ensemble to select relevant morphological features of microglia cells, perform dimensionality reduction, cluster these features, and subsequently map the clustered cells back onto the tissue, providing a spatial context for the identified microglial morphologies. We applied this pipeline to compare the responses between saline solution (SS) and scopolamine (SCOP) groups in a SCOP-induced mouse model of Alzheimer's disease, with a specific focus on the hippocampal subregions CA1 and Hilus. Next, we assessed microglial morphologies across four groups: SS-CA1, SCOP-CA1, SS-Hilus, and SCOP-Hilus. The results demonstrated that MorphoGlia effectively differentiated between SS and SCOP-treated groups, identifying distinct clusters of microglial morphologies commonly associated with pro-inflammatory states in the SCOP groups. Additionally, MorphoGlia enabled spatial mapping of these clusters, identifying the most affected hippocampal layers. This study highlights MorphoGlia's capability to provide unbiased analysis and clustering of microglial morphological states, making it a valuable tool for exploring microglial heterogeneity and its implications for central nervous system pathologies.

Machine Learning Models Decoding the Association Between Urinary Stone Diseases and Metabolic Urinary Profiles

Background: Employing advanced machine learning models, we aim to identify biomarkers for urolithiasis from 24-h metabolic urinary abnormalities and study their associations with urinary stone diseases. Methods: We retrospectively recruited 468 patients at Peking Union Medical College Hospital who were diagnosed with urinary stone disease, including renal, ureteral, and multiple location stones, and had undergone a 24-h urine metabolic evaluation. We applied machine learning methods to identify biomarkers of urolithiasis from the urinary metabolite profiles. In total, 148 (34.02%) patients were with kidney stones, 34 (7.82%) with ureter stones, and 163 (34.83%) with multiple location stones, all of whom had detailed urinary metabolite data. Our analyses revealed that the Random Forest algorithm exhibited the highest predictive accuracy, with AUC values of 0.809 for kidney stones, 0.99 for ureter stones, and 0.775 for multiple location stones. The Super Learner Ensemble Method also demonstrated high predictive performance with slightly lower AUC values compared to Random Forest. Further analysis using multivariate logistic regression identified significant features for each stone type based on the Random Forest method. Results: We found that 24-h urinary magnesium was positively associated with both kidney stones and multiple location stones (OR = 1.195 [1.06–1.3525] and 1.3258 [1.1814–1.4949]) due to its high correlation with urinary phosphorus, while 24-h urinary creatinine was a protective factor for kidney stones and ureter stones, with ORs of 0.9533 [0.9117–0.996] and 0.8572 [0.8182–0.8959]. eGFR was a risk factor for ureter stones and multiple location stones, with ORs of 1.0145 [1.0084–1.0209] and 1.0148 [1.0077–1.0223]. Conclusion: Machine learning techniques show promise in revealing the links between urological stone disease and 24-h urinary metabolic data. Enhancing the prediction accuracy of these models leads to improved dietary or pharmacological prevention strategies.

Enhanced Diagnosis of Lung Cancer through an Ensemble Learning Model leveraging an Adaptive Optimization Algorithm

Early and accurate diagnosis of lung cancer is crucial to improving patient outcomes and survival rates. Machine and deep learning models have emerged as promising tools to improve the accuracy and efficiency of disease diagnosis. However, achieving optimal diagnostic performance remains a challenging task in medical research. This study integrates ensemble learning techniques with an adaptive optimization algorithm to enhance the accuracy of lung cancer diagnosis. By combining the predictive potential of multiple base classifiers, the ensemble-learning model improves overall performance and mitigates the weaknesses of individual classifiers. Additionally, the adaptive optimization algorithm dynamically adjusts the model parameters to optimize the classification performance. The effectiveness of the approach was evaluated using a comprehensive dataset that includes lung cancer images. Rigorous evaluation and comparison with state-of-the-art models showed that the proposed method achieved superior diagnostic performance, reaching an overall accuracy of 99%.

Eye Movement Classification using Feature Engineering and Ensemble Machine Learning

This paper explores the classification of gaze direction using electrooculography (EOG) signals, integrating signal processing, deep learning, and ensemble learning techniques to enhance accuracy and reliability. A complex technique is proposed in which several feature types are derived from EOG data. Spectral properties generated from power spectral density analysis augment basic statistical characteristics such as mean and standard deviation, revealing the frequency content of the signal. Skewness, kurtosis, and cross-channel correlations are also used to represent intricate nonlinear dynamics and inter-channel interactions. These characteristics are then reformatted into a two-dimensional array imitating picture data, enabling the use of the pre-trained ResNet50 model to extract deep and high-level characteristics. Using these deep features, an ensemble of bagging-trained decision trees classifies gaze directions, lowering model variance and increasing prediction accuracy. The results show that the ensemble deep learning model obtained outstanding performance metrics, with accuracy and sensitivity ratings exceeding 97% and F1-score of 98%. These results not only confirm the effectiveness of the proposed approach in managing challenging EOG signal classification tasks but also imply important consequences for the improvement of Human-Computer Interaction (HCI) systems, especially in assistive technologies where accurate gaze tracking is fundamental.