Stacking Ensemble Research Articles

Exploring forest fire susceptibility and management strategies in Western Himalaya: Integrating ensemble machine learning and explainable AI for accurate prediction and comprehensive analysis

Forest fires pose a significant threat to ecosystems and socio-economic activities, necessitating the development of accurate predictive models for effective management and mitigation. In this study, we present a novel machine learning approach combined with Explainable Artificial Intelligence (XAI) techniques to predict forest fire susceptibility in Nainital district. Our innovative methodology integrates several robust models — AdaBoost, Gradient Boosting Machine (GBM), XGBoost and Random Forest — with a Deep Neural Network (DNN) as a meta-model in a stacking framework. This approach not only utilises the individual strengths of these models, but also improves the overall prediction performance and reliability. By using XAI techniques, in particular SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations), we improve the interpretability of the models and provide insights into the decision-making processes. Our results show the effectiveness of the ensemble model in categorising forest fire susceptibility into different zones: very low, low, moderate, high and very high. In particular, the stacking ensemble and XGBoost models identified extensive areas of high and very high susceptibility, with precision, recall and F1 values underpinning their effectiveness. These models achieved ROC AUC values above 0.90, with XGBoost performing exceptionally well with an AUC of 0.94. The precision, recall and F1 values for these models are remarkably high. The inclusion of confidence intervals for the most important metrics in all models emphasises their robustness and reliability and supports their practical use in forest fire management. Through SHAP summary plots, we analyze the global variable importance, revealing annual rainfall and Evapotranspiration (ET) as key factors influencing susceptibility. Local analysis using LIME consistently highlights the importance of annual rainfall, ET, and distance from roads across all models. This study fills an important research gap by providing a comprehensive and interpretable modelling approach that improves our ability to predict and effectively manage forest fire risk and is consistent with environmental protection and sustainable development goals.

Sampling-based novel heterogeneous multi-layer stacking ensemble method for telecom customer churn prediction

In recent times, customer churn has become one of the most significant issues in business-oriented sectors with telecommunication being no exception. Maintaining current customers is particularly valuable due to the high degree of rivalry among telecommunication companies and the costs of acquiring new ones. The early prediction of churned customers may help telecommunication companies to identify the causes of churn and design industrial tactics to address or mitigate the churn problem. Controlling customer churn by developing efficient and reliable customer churn prediction (CCP) solutions is essential to achieving this objective. Findings from existing CCP studies have shown that numerous methods, such as rule-based and machine-learning (ML) mechanisms, have been devised to solve the CCP problem. Nonetheless, the problems of adaptability and the resilience of rule-based CCP solutions are its major weaknesses, and the skewed pattern of churn datasets (class imbalance) is detrimental to the prediction performances of conventional ML models in CCP. Hence, this research developed a robust heterogeneous multi-layer stacking ensemble method (HMSE) for effective CCP. Specifically, in the HMSE method, the prediction prowess of five ML classifiers (Random Forest (RF), Bayesian network (BN), Support Vector Machine (SVM), K-Nearest Neighbour (KNN), and Repeated Incremental Pruning to Produce Error Reduction (RIPPER)) with distinct computational characteristics are ensembled based on stacking and the resulting model is further enhanced using a forest penalizing attribute (FPA) model. The synthetic minority oversampling technique (SMOTE) is integrated with the proposed HMSE to balance the skewed class label present in the original experimental datasets. Extensive tests were carried out to determine the effectiveness of the proposed HMSE and S-HMSE on standard telecom CCP datasets. Observed findings from the experimental results showed that HMSE and S-HMSE can effectively predict churners even with the class imbalance (skewed datasets) problem. In addition, comparison studies demonstrated that the suggested S-HMSE offered improved prediction performance and optimum solutions for CCP in the telecom sector in comparison with baseline classifiers, homogeneous ensemble methods, and current CCP approaches.

SEnSCA: Identifying possible ligand-receptor interactions and its application in cell-cell communication inference.

Multicellular organisms have dense affinity with the coordination of cellular activities, which severely depend on communication across diverse cell types. Cell-cell communication (CCC) is often mediated via ligand-receptor interactions (LRIs). Existing CCC inference methods are limited to known LRIs. To address this problem, we developed a comprehensive CCC analysis tool SEnSCA by integrating single cell RNA sequencing and proteome data. SEnSCA mainly contains potential LRI acquisition and CCC strength evaluation. For acquiring potential LRIs, it first extracts LRI features and reduces the feature dimension, subsequently constructs negative LRI samples through K-means clustering, finally acquires potential LRIs based on Stacking ensemble comprising support vector machine, 1D-convolutional neural networks and multi-head attention mechanism. During CCC strength evaluation, SEnSCA conducts LRI filtering and then infers CCC by combining the three-point estimation approach and single cell RNA sequencing data. SEnSCA computed better precision, recall, accuracy, F1 score, AUC and AUPR under most of conditions when predicting possible LRIs. To better illustrate the inferred CCC network, SEnSCA provided three visualization options: heatmap, bubble diagram and network diagram. Its application on human melanoma tissue demonstrated its reliability in CCC detection. In summary, SEnSCA offers a useful CCC inference tool and is freely available at https://github.com/plhhnu/SEnSCA.

Ensemble Stacking for Grading Facial Paralysis Through Statistical Analysis of Facial Features

Ensemble Stacking for Grading Facial Paralysis Through Statistical Analysis of Facial Features

Concept Drift Mitigation in Low-Cost Air Quality Monitoring Networks.

Future air quality monitoring networks will integrate fleets of low-cost gas and particulate matter sensors that are calibrated using machine learning techniques. Unfortunately, it is well known that concept drift is one of the primary causes of data quality loss in machine learning application operational scenarios. The present study focuses on addressing the calibration model update of low-cost NO2 sensors once they are triggered by a concept drift detector. It also defines which data are the most appropriate to use in the model updating process to gain compliance with the relative expanded uncertainty (REU) limits established by the European Directive. As the examined methodologies, the general/global and the importance weighting calibration models were applied for concept drift effects mitigation. Overall, for all the devices under test, the experimental results show the inadequacy of both models when performed independently. On the other hand, the results from the application of both models through a stacking ensemble strategy were able to extend the temporal validity of the used calibration model by three weeks at least for all the sensor devices under test. Thus, the usefulness of the whole information content gathered throughout the original co-location process was maximized.

Enhancing Academic Success Prediction: An Ensemble Approach

The process of extracting meaning and knowledge from large volumes of data is termed as Data mining. It also refers to the way of inferring information from databases, which can be used in diverse fields including educational domain. Educational data mining plays a key role in finding ways to discover knowledge from data in the education sector. Educational Data Mining has evolved as a significant element in prediction of students’ academic performance. The most important goal of the paper is to analyze and evaluate the engineering students’ performance by applying stacking, an ensemble method in orange tool. Ensemble Stacking combines several base classifier models in order to create one optimal prediction model. Engineering students’ dataset was used to build predictive model using traditional classifiers SVM, Logistic Regression, Naïve Bayes and then stacking technique was implemented. The results showed that the proposed stacking technique obtains a high performance, which has a superior result compared to the other base classifiers techniques. Therefore, conclusion could be reached that the stacking performance is better than that of different algorithms.

An Efficient and Interpretable Stacked Model for Wind Speed Estimation Based on Ensemble Learning Algorithms

Wind energy has gained tremendous attention recently due to its abundance potential and to obviate the adverse impacts of fossil fuel consumption. However, the efficient development and operation of wind energy systems require precise and accurate wind speed prediction. To fulfill this aim, machine learning algorithms have shown its prowess in accurately predicting the wind speed and this study proposes an interpretable stacked ensemble with ridge regressor utilizing ML algorithms such as catboost (CB), gradient boost (GB), multilayer perceptron (MLP), and random forest (RF) as a base model. The base models’ estimations are integrated by applying a ridge regressor to enhance the accuracy of wind speed estimation. The dataset consists of 87 600 samples from which 80% samples are used for training. The proposed approach provides interpretability of the prediction results through Shapley additive explanations and feature importance analysis. The result of the proposed method is calculated by comparing the estimation outcomes with each model such as CB, GB, MLP, and RF in terms of error metrics such as root mean squared error (RMSE) and mean absolute error (MAE). The proposed model outperforms the others due to a reduction of error in RMSE by 19.89% and MAE by 24% respectively.

Requirement Dependency Extraction Based on Improved Stacking Ensemble Machine Learning

To address the cost and efficiency issues of manually analysing requirement dependency in requirements engineering, a requirement dependency extraction method based on part-of-speech features and an improved stacking ensemble learning model (P-Stacking) is proposed. Firstly, to overcome the problem of singularity in the feature extraction process, this paper integrates part-of-speech features, TF-IDF features, and Word2Vec features during the feature selection stage. The particle swarm optimization algorithm is used to allocate weights to part-of-speech tags, which enhances the significance of crucial information in requirement texts. Secondly, to overcome the performance limitations of standalone machine learning models, an improved stacking model is proposed. The Low Correlation Algorithm and Grid Search Algorithms are utilized in P-stacking to automatically select the optimal combination of the base models, which reduces manual intervention and improves prediction performance. The experimental results show that compared with the method based on TF-IDF features, the highest F1 scores of a standalone machine learning model in the three datasets were improved by 3.89%, 10.68%, and 21.4%, respectively, after integrating part-of-speech features and Word2Vec features. Compared with the method based on a standalone machine learning model, the improved stacking ensemble machine learning model improved F1 scores by 2.29%, 5.18%, and 7.47% in the testing and evaluation of three datasets, respectively.

An Analytical Predictive Model and Secure Wed Based Personalized Diabetes Monitoring System using Stacking Ensemble Classification

As the number of people diagnosed with diabetes continues to rise, this study takes a groundbreaking approach by developing a secure web-based personalized diabetes monitoring system that incorporates analytical prediction models. This groundbreaking technology was developed in response to the critical need for sophisticated monitoring solutions that address the unique demands of each patient. The suggested method aims to transform diabetes treatment by using predictive modeling to anticipate diabetic trends and possible consequences. The study demonstrates a strong commitment to security by creating a web-based platform that handles patient data with the highest level of care. By resolving important privacy problems and creating a trustworthy environment for users, this secure framework guarantees the protection of sensitive health information. This approach takes use of several models' characteristics to make diabetes trend estimates more accurate and reliable. In addition to improving the system's prediction skills, stacking ensemble classification helps it adapt to different patient profiles. Because of the importance of accessibility and usability in encouraging patient participation, the suggested solution revolves around the creation of a user-friendly online interface. The interface is a living, breathing platform that allows for frictionless communication between healthcare practitioners and their patients. With the help of tailored insights and trend predictions, patients are better able to manage their diabetes. At the same time, doctors and hospitals have access to all the patient information they need, which allows them to take better, more preventative measures.

Application of Stacking Ensemble Learning in Predicting Copper’s Flotation Concentrate Grade

Addressing issues such as a low operational condition recognition efficiency, strong subjectivity, and significant fluctuations in Outotec X fluorescence analysis data in copper flotation production, a copper concentrate grade classification model is constructed based on image processing technology and the Stacking ensemble learning algorithm. Firstly, a feature extraction model for copper concentration flotation foam images is established, extracting color, texture, and size statistical features to build a feature dataset. Secondly, to avoid redundancy in the feature data, which could reduce model accuracy, a combined correlation feature selection is employed for dimensionality reduction, with the filtered feature subset being used as the model input. Finally, to fully leverage the strengths of each model, a Stacking ensemble learning copper concentrate grade classification model is constructed with support vector machine (SVM), random forest (RF), and adaptive boosting (AdaBoost) as base models and logistic regression (LR) as the meta-model. The experimental results show that this ensemble model achieves good recognition for different grade categories, with a precision, recall, and F1 score of 90.01%, 89.85%, and 89.93%, respectively. The accuracy of this Stacking ensemble model, with a 7% improvement over Outotec X fluorescence analysis, demonstrates a potential to meet the daily production needs of beneficiation plants.

Power System Transient Stability Preventive Control via Aptenodytes Forsteri Optimization with an Improved Transient Stability Assessment Model

Transient stability preventive control (TSPC), a method to efficiently withstand the severe contingencies in a power system, is mathematically a transient stability constrained optimal power flow (TSC-OPF) issue, attempting to maintain the economical and secure dispatch of a power system via generation rescheduling. The traditional TSC-OPF issue incorporated with differential-algebraic equations (DAE) is time consumption and difficult to solve. Therefore, this paper proposes a new TSPC method driven by a naturally inspired optimization algorithm integrated with transient stability assessment. To avoid solving complex DAE, the stacking ensemble multilayer perceptron (SEMLP) is used in this research as a transient stability assessment (TSA) model and integrated into the optimization algorithm to replace transient stability constraints. Therefore, less time is spent on challenging calculations. Simultaneously, sensitivity analysis (SA) based on this TSA model determines the adjustment direction of the controllable generators set. The results of this SA can be utilized as prior knowledge for subsequent optimization algorithms, thus further reducing the time consumption process. In addition, a naturally inspired algorithm, Aptenodytes Forsteri Optimization (AFO), is introduced to find the best operating point with a near-optimal operational cost while ensuring power system stability. The accuracy and effectiveness of the method are verified on the IEEE 39-bus system and the IEEE 300-bus system. After the implementation of the proposed TSPC method, both systems can ensure transient stability under a given contingency. The test experiment using AFO driven by SEMLP and SA on the IEEE 39-bus system is completed in about 35 s, which is one-tenth of the time required by the time domain simulation method.

Presenting a three layer stacking ensemble classifier of deep learning and machine learning for skin cancer classification

Presenting a three layer stacking ensemble classifier of deep learning and machine learning for skin cancer classification

Daily reservoir inflow prediction using stacking ensemble of machine learning algorithms

ABSTRACT The present study aims to evaluate the potentiality of Bidirectional Long Short-Term Memory (Bi-LSTM), Convolutional Neural Networks (CNNs), eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LGBM), and Random Forest (RF) for predicting daily inflows to the Sri Ram Sagar Project (SRSP), Telangana, India. Inputs to the model are rainfall, evaporation, time lag inflows, and climate indices. Seven combinations (S1–S7) of inputs were made. Fifteen and a half years of data were considered, out of which 11 years were used for training. Hyperparameter tuning is performed with the Tree-Structured Parzen Estimator. The performance of the algorithms is assessed using Kling–Gupta efficiency (KGE). Results indicate that Bi-LSTM with combination S7 performed better than others, as evident from KGE values of 0.92 and 0.87 during the training and testing, respectively. Furthermore, the Stacking Ensemble Mechanism (SEM) has also been employed to ascertain its efficacy over other chosen algorithms, resulting in KGE values of 0.94 and 0.89 during training and testing. It has also been able to simulate peak inflow events satisfactorily. Thus, SEM is a better alternative for reservoir inflow predictions.

Explainable geospatial-artificial intelligence models for the estimation of PM2.5 concentration variation during commuting rush hours in Taiwan

PM2.5 concentrations are higher during rush hours at background stations compared to the average concentration across these stations. Few studies have investigated PM2.5 concentration and its spatial distribution during rush hours using machine learning models. This study employs a geospatial-artificial intelligence (Geo-AI) prediction model to estimate the spatial and temporal variations of PM2.5 concentrations during morning and dusk rush hours in Taiwan. Mean hourly PM2.5 measurements were collected from 2006 to 2020, and aggregated into morning (7 a.m.–9 a.m.) and dusk (4 p.m.–6 p.m.) rush-hour mean concentrations. The Geo-AI prediction model was generated by integrating kriging interpolation, land-use regression, machine learning, and a stacking ensemble approach. A forward stepwise variable selection method based on the SHapley Additive exPlanations (SHAP) index was used to identify the most influential variables. The performance of the Geo-AI models for morning and dusk rush hours had accuracy scores of 0.95 and 0.93, respectively and these results were validated, indicating robust model performance. Spatially, PM2.5 concentrations were higher in southwestern Taiwan for morning rush hours, and suburban areas for dusk rush hours. Key predictors included kriged PM2.5 values, SO2 concentrations, forest density, and the distance to incinerators for both morning and dusk rush hours. These PM2.5 estimates for morning and dusk rush hours can support the development of alternative commuting routes with lower concentrations.

PseUpred-ELPSO Is an Ensemble Learning Predictor with Particle Swarm Optimizer for Improving the Prediction of RNA Pseudouridine Sites.

RNA pseudouridine modification exists in different RNA types of many species, and it has a significant role in regulating the expression of biological processes. To understand the functional mechanisms for RNA pseudouridine sites, the accurate identification of pseudouridine sites in RNA sequences is essential. Although several fast and inexpensive computational methods have been proposed, the challenge of improving recognition accuracy and generalization still exists. This study proposed a novel ensemble predictor called PseUpred-ELPSO for improved RNA pseudouridine site prediction. After analyzing the nucleotide composition preferences between RNA pseudouridine site sequences, two feature representations were determined and fed into the stacking ensemble framework. Then, using five tree-based machine learning classifiers as base classifiers, 30-dimensional RNA profiles are constructed to represent RNA sequences, and using the PSO algorithm, the weights of the RNA profiles were searched to further enhance the representation. A logistic regression classifier was used as a meta-classifier to complete the final predictions. Compared to the most advanced predictors, the performance of PseUpred-ELPSO is superior in both cross-validation and the independent test. Based on the PseUpred-ELPSO predictor, a free and easy-to-operate web server has been established, which will be a powerful tool for pseudouridine site identification.

A Weighted Stacking Ensemble Model With Sampling for Fake Reviews Detection

Customers use reviews as a primary source of information to judge a product or service. Positive reviews help boost companies’ reputations, increasing their revenue by attracting new clients, and increasing the purchasing order size. On the other hand, negative reviews significantly reduce sales, which might be the case due to competitive advantage. Organizations can use fake (i.e., misleading or fraudulent) reviews to generate fast profits by deceiving customers into buying their products. Recently, various methods to assess the legitimacy of reviews have been introduced using advances in machine learning. However, existing methods fall short of achieving highly accurate detection results for unbalanced classes. We aimed to create a spam review identification model using ensemble-based learning while balancing classes using sampling techniques. This article proposes a weighted stacking ensemble model with sampling (WSEM-S) for efficient fake reviews detection. We used <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$</tex-math> </inline-formula> -gram models to effectively model language data for feature retrieval. The experimental results on three customer reviews datasets: YELPNYC, Deceptive Opinion Spam Corpus (DOSC) v1.4, and Deception datasets show that the proposed model outperforms the conventional machine learning techniques Naïve Bayes, logistic regression, K-nearest neighbor (KNN), random forest, extreme gradient boosting (XGBoost), and convolutional neural network (CNN) as well as the state-of-the-art ensemble models.

DEF: Deep Ensemble Neural Network Classifier for Android Malware Detection

Malware is one of the threats to security of computer networks and information systems. Since malware instances are available sufficiently, there is increased interest among researchers on usage of Artificial Intelligence (AI). Of late AI-enabled methods such as machine learning (ML) and deep learning paved way for solving many real-world problems. As it is a learning-based approach, accumulated training samples help in improving thequality of training and thus leveraging malware detection accuracy. Existing deep learning methods are focusing on learning-based malware detection systems. However, there is need for improving the state of the art through ensemble approach. Towards this end, in this paper we proposed a framework known as Deep Ensemble Framework (DEF) for automatic malware detection. The framework obtains features from training samples. From given malware instance a grayscale image is generated. There is another process to extract the opcode sequences. Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) techniques are used to obtain grayscale image and opcode sequence respectively. Afterwards, a stacking ensemble is employed in order to achieve efficient malware detection and classification. Malware samples collected from the Internet sources and Microsoft are used for the empirical study. An algorithm known as Ensemble Learning for Automatic Malware Detection (EL-AML) is proposed to realize our framework. Another algorithm named Pre-Process is proposed to assist the ELAML algorithm for obtaining intermediate features required by CNN and LSTM. Empirical study reveals that our framework outperforms many existing methods in terms of speed-up and accuracy.

A multimodal stacked ensemble model for cardiac output prediction utilizing cardiorespiratory interactions during general anesthesia

This study examined the possibility of estimating cardiac output (CO) using a multimodal stacking model that utilizes cardiopulmonary interactions during general anesthesia and outlined a retrospective application of machine learning regression model to a pre-collected dataset. The data of 469 adult patients (obtained from VitalDB) with normal pulmonary function tests who underwent general anesthesia were analyzed. The hemodynamic data in this study included non-invasive blood pressure, plethysmographic heart rate, and SpO2. CO was recorded using Vigileo and EV1000 (pulse contour technique devices). Respiratory data included mechanical ventilation parameters and end-tidal CO2 levels. A generalized linear regression model was used as the metalearner for the multimodal stacking ensemble method. Random forest, generalized linear regression, gradient boosting machine, and XGBoost were used as base learners. A Bland–Altman plot revealed that the multimodal stacked ensemble model for CO prediction from 327 patients had a bias of − 0.001 L/min and − 0.271% when calculating the percentage of difference using the EV1000 device. Agreement of model CO prediction and measured Vigileo CO in 142 patients reported a bias of − 0.01 and − 0.333%. Overall, this model predicts CO compared to data obtained by the pulse contour technique CO monitors with good agreement.

Loss of Life Transformer Prediction Based on Stacking Ensemble Improved by Genetic Algorithm By IJISRT

Prediction for loss of life transfomer is very important to ensure the reliability and efficiency of the power system. In this paper, an innovative model is proposed to improve the accuracy of lost of life transfomer prediction using stacking ensembles enhanced with genetic algorithm (GA). The aim is to develop a robust model to estimate the remaining life of a transformer in order to generally increase the reliability of the electrical energy distribution system. This approach involves integrating various machine learning models as a basic model, namely Support Vector Machines (SVM) and K-Nearest Neighbor (KNN). A stacking ensemble framework is then used to combine the predictions of these base models using a meta model namely Logistic Regression (LR). The results show a significant improvement in both transformers using stacking-GA, both TR-A and TR-B, with each prediction evaluation 99% and with a minimal error rate, namely approaching 0.the developed framework presents a promising solution for accurate and reliable transformer life prediction. By integrating a variety of basic models, applying improved stacking layouts using GA, these models offer valuable insights to improve maintenance strategies and system reliability in power grids.

Adaptive Stacking Ensemble Techniques for Early Severity Classification of COVID-19 Patients

During outbreaks of infectious diseases, such as COVID-19, it is critical to rapidly determine treatment priorities and identify patients requiring hospitalization based on clinical severity. Although various machine learning models have been developed to predict COVID-19 severity, most have limitations, such as small dataset sizes, the limited availability of clinical variables, or a constrained classification of severity levels by a single classifier. In this paper, we propose an adaptive stacking ensemble technique that identifies various COVID-19 patient severity levels and separates them into three formats: Type 1 (low or high severity), Type 2 (mild, severe, critical), and Type 3 (asymptomatic, mild, moderate, severe, fatal). To enhance the model’s generalizability, we utilized a nationwide dataset from the South Korean government, comprising data from 5644 patients across over 100 hospitals. To address the limited availability of clinical variables, our technique employs data-driven strategies and a proposed feature selection method. This ensures the availability of clinical variables across diverse hospital environments. To construct optimal stacking ensemble models, our technique adaptively selects candidate base classifiers by analyzing the correlation between their predicted outcomes and performance. It then automatically determines the optimal multi-layer combination of base and meta-classifiers using a greedy search algorithm. To further improve the performance, we applied various techniques, including imputation of missing values and oversampling. The experimental results demonstrate that our stacking ensemble models significantly outperform existing single classifiers and AutoML approaches, with improvements of 6.42% and 8.86% in F1 and AUC scores for Type 1, 9.59% and 6.68% for Type 2, and 11.94% and 9.24% for Type 3, respectively. Consequently, our approach improves the prediction of COVID-19 severity levels and potentially assists frontline healthcare providers in making informed decisions.