Stack Model Research Articles

Machine learning enabling prediction in mechanical performance of Ti6Al4V fabricated by large-scale laser powder bed fusion via a stacking model

Machine learning enabling prediction in mechanical performance of Ti6Al4V fabricated by large-scale laser powder bed fusion via a stacking model

Cropping intensity mapping in Sentinel-2 and Landsat-8/9 remote sensing data using temporal transfer of a stacked ensemble machine learning model within google earth engine

This article aimed to map Cropping Intensity Patterns (CIPs) in the southwest region of Iran using Google Earth Engine and monthly composites of Sentinel-2 and Landsat-8/9 data. To detect CIPs with high inter- and intra-class variability of crops, a heterogeneous Stack ensemble of machine learning model was developed. The model incorporated the Minimum Distance (MD) approach as a meta-classifier, combining multiple base models, including Support Vector Machines (SVM), Random Forest (RF), Classification and Regression Trees (CART), and Gradient Boosted Trees (GBT). In 2021, the Stack model was trained and evaluated using Ground Truth (GT) samples from the same year, achieving an Overall Accuracy (OA) of 94.24%. This performance surpassed the base models by about 4% in OA and was also reflected in the detection accuracies, including User’s Accuracy (UA), Producer’s Accuracy (PA), and F1-score, of the target classes. Subsequently, the trained stack model was temporally transferred to generate CIP maps for other years. The model achieved high OAs of 91.82% and 90.97% based on GT samples from 2020 and 2022, respectively. Finally, the time series of CIP maps (2019-2023) were utilized by the Cellular Automata-Markov model to forecast the map for 2024.

Validation and Application of Hysteresis Loss Model for HTS Stacks and Conductors for Fusion Applications

Validation and Application of Hysteresis Loss Model for HTS Stacks and Conductors for Fusion Applications

Prediction of Super-enhancers Based on Mean-shift Undersampling

Background: Super-enhancers are clusters of enhancers defined based on the binding occupancy of master transcription factors, chromatin regulators, or chromatin marks. It has been reported that super-enhancers are transcriptionally more active and cell-type-specific than regular enhancers. Therefore, it is necessary to identify super-enhancers from regular enhancers. A variety of computational methods have been proposed to identify super-enhancers as auxiliary tools. However, most methods use ChIP-seq data, and the lack of this part of the data will make the predictor unable to execute or fail to achieve satisfactory performance. Objective: The aim of this study is to propose a stacking computational model based on the fusion of multiple features to identify super-enhancers in both human and mouse species. Methods: This work adopted mean-shift to cluster majority class samples and selected four sets of balanced datasets for mouse and three sets of balanced datasets for human to train the stacking model. Five types of sequence information are used as input to the XGBoost classifier, and the average value of the probability outputs from each classifier is designed as the final classification result. Results: The results of 10-fold cross-validation and cross-cell-line validation prove that our method has superior performance compared to other existing methods. The source code and datasets are available at https://github.com/Cheng-Han-max/SE_voting. Conclusion: The analysis of feature importance indicates that Mismatch accounts for the highest proportion among the top 20 important features.

Stacking-Kcr: A Stacking Model for Predicting the Crotonylation Sites of Lysine by Fusing Serial and Automatic Encoder

Background: Protein lysine crotonylation (Kcr), a newly discovered important posttranslational modification (PTM), is typically localized at the transcription start site and regulates gene expression, which is associated with a variety of pathological conditions such as developmental defects and malignant transformation. Objective: Identifying Kcr sites is advantageous for the discovery of its biological mechanism and the development of new drugs for related diseases. However, traditional experimental methods for identifying Kcr sites are expensive and inefficient, necessitating the development of new computational techniques. Methods: In this work, to accurately identify Kcr sites, we propose a model for ensemble learning called Stacking-Kcr. Firstly, extract features from sequence information, physicochemical properties, and sequence fragment similarity. Then, the two characteristics of sequence information and physicochemical properties are fused using automatic encoder and serial, respectively. Finally, the fused two features and sequence fragment similarity features are then respectively input into the four base classifiers, a meta classifier is constructed using the first level prediction results, and the final forecasting results are obtained. Results: The five-fold cross-validation of this model has achieved an accuracy of 0.828 and an AUC of 0.910. This shows that the Stacking-Kcr method has obvious advantages over traditional machine learning methods. On independent test sets, Stacking-Kcr achieved an accuracy of 84.89% and an AUC of 92.21%, which was higher than 1.7% and 0.8% of other state-of-the-art tools. Additionally, we trained Stacking-Kcr on the phosphorylation site, and the result is superior to the current model. Conclusion: These outcomes are additional evidence that Stacking-Kcr has strong application potential and generalization performance.

Leveraging cfDNA fragmentomic features in a stacked ensemble model for early detection of esophageal squamous cell carcinoma

In this study, we develop a stacked ensemble model that utilizes cell-free DNA (cfDNA) fragmentomics for the early detection of esophageal squamous cell carcinoma (ESCC). This model incorporates four distinct fragmentomics features derived from whole-genome sequencing (WGS) and advanced machine learning algorithms for robust analysis. It is validated across both an independent validation cohort and an external cohort to ensure its generalizability and effectiveness. Notably, the model maintains its robustness in low-coverage sequencing environments, demonstrating its potentials in clinical settings with limited sequencing resources. With its remarkable sensitivity and specificity, this approach promises to significantly improve the early diagnosis and management of ESCC. This study represents a substantial step forward in the application of cfDNA fragmentomics in cancer diagnostics, emphasizing the need for further research to fully establish its clinical efficacy.

Crustal thicknesses in Uruguay from joint inversion of receiver functions and surface wave dispersion

The development of a seismic network in Uruguay in recent years has enabled studies of crustal structure in a region with few seismological studies of this type. In this work, we update the crustal thicknesses and Vp/Vs ratios calculated by H-k stack and present S-wave velocity models based on joint inversion of receiver functions and Rayleigh wave group velocity dispersion curves.Some interesting results are the presence of a lower crust with a high S-wave velocity of 4.1 km/s below one of the stations located on the Río de la Plata Craton and the existence of a transitional Moho in Uruguay's northernmost station on the Paraná Basin, perhaps suggesting the presence of localized underplated material. A relatively thick crust, 41.8 km, compared to surrounding stations, was found beneath the Sierra Ballena Shear Zone in the Dom Feliciano Belt. We confirm the decrease in crustal thickness when approaching the oceanic coast, reaching a Moho depth of 36.3 km in SE Uruguay. Finally, the calculated Poisson's ratio allows inferring a crust of felsic to intermediate composition beneath most of Uruguay.

A stacking machine learning model for predicting pullout capacity of small ground anchors

Small ground anchors are widely used to fix securing tents in disaster relief efforts. Given the urgent nature of rescue operations, it is crucial to obtain prompt and accurate estimations of their pullout capacity. In this study, a stacking machine learning (ML) model is developed for the rapid estimation of pullout capacity offered by small ground anchors used for temporary tents, leveraging cone penetration data. The proposed stacking model incorporates three ML algorithms as the base regression models: K-nearest neighbors (KNN), support vector regression (SVR), and extreme gradient boosting (XGBoost). A dataset comprising 119 in-situ anchor pullout tests, where the cone penetration data were measured, is utilized to train and assess the stacking model performance. Three metrics, i.e., coefficient of determination (R2), mean absolute error (MAE), and root mean square error (RMSE), are employed to evaluate the predictive accuracy of the proposed model and compare its performance against four popular ML models and an empirical formula to highlight the advantages of the proposed stacking approach. The results affirm that the proposed stacking model outperforms other ML models and the empirical approach as achieving higher R2 and lower MAE and RMSE and more predicted data points falling within 20% error line. Thus, the proposed stacking model holds promising potential as a solution for efficiently predicting the pullout capacity of small ground anchors.

Optimization-based stacked machine-learning method for seismic probability and risk assessment of reinforced concrete shear walls

Efficient seismic risk assessment aids decision-makers in formulating citywide risk mitigation plans, providing insights into building performance and retrofitting costs. The complexity of modeling, analysis, and post-processing of the results makes it hard to fast-track the seismic probabilities, and there is a need to optimize the computational time. This research addresses seismic probability and risk assessment of reinforced concrete shear walls (RCSWs) by introducing stacked machine learning (Stacked ML) models based on Bayesian optimization (BO), genetic algorithm (GA), particle swarm optimization (PSO), and gradient-based optimization (GBO) algorithms. The study investigates 4-, to 15-Story RCSWs assuming different bay lengths and soil types to build a comprehensive database based on the incremental dynamic analysis (IDA) subjected to 56 near-field pulse-like and no-pulse records. Having 227,200 and 63,384 data points for a median of IDA curve (MIDA) and seismic probability curve, respectively, the proposed Stacked ML models have shown good performance on curve fitting ability by accuracy of 99.1% and 99.4% for MIDA and seismic fragility curves, respectively. In addition, the proposed models can estimate the mean annual frequency, λ, which is a key parameter in seismic risk assessment of buildings. To provide the results of the study for general buildings, a user-friendly GUI is proposed that facilitates result utilization, offering insights into seismic performance levels, providing the estimated MIDA and seismic failure probability curves, and mean annual frequency calculations for specific performance levels and seismic hazard curves.

Predicting depression in patients with heart failure based on a stacking model.

There is a lack of literature discussing the utilization of the stacking ensemble algorithm for predicting depression in patients with heart failure (HF). To create a stacking model for predicting depression in patients with HF. This study analyzed data on 1084 HF patients from the National Health and Nutrition Examination Survey database spanning from 2005 to 2018. Through univariate analysis and the use of an artificial neural network algorithm, predictors significantly linked to depression were identified. These predictors were utilized to create a stacking model employing tree-based learners. The performances of both the individual models and the stacking model were assessed by using the test dataset. Furthermore, the SHapley additive exPlanations (SHAP) model was applied to interpret the stacking model. The models included five predictors. Among these models, the stacking model demonstrated the highest performance, achieving an area under the curve of 0.77 (95%CI: 0.71-0.84), a sensitivity of 0.71, and a specificity of 0.68. The calibration curve supported the reliability of the models, and decision curve analysis confirmed their clinical value. The SHAP plot demonstrated that age had the most significant impact on the stacking model's output. The stacking model demonstrated strong predictive performance. Clinicians can utilize this model to identify high-risk depression patients with HF, thus enabling early provision of psychological interventions.

SEDAT: A Stacked Ensemble Learning-Based Detection Model for Multiscale Network Attacks

SEDAT: A Stacked Ensemble Learning-Based Detection Model for Multiscale Network Attacks

Effects of wave breaking on moisture flux

Wave breaking is a ubiquitous phenomenon in ocean dynamics, serving as a critical conduit for the exchange of momentum, heat, and energy between the atmosphere and the ocean. Although its vital role in air-sea moisture exchange is widely acknowledged, quantifying its exact impact on moisture flux remained quite challenging due to data limitations and the inherently turbulent nature of the process. To overcome these challenges, we construct a comprehensive 10-year global dataset that incorporates multiple breaking wave variables, informed by statistical breaking theories, to capture the intricacies of the breaking process and its consequential effects. We then employ a stacked machine learning model to elucidate the complex relationships between wave breaking and moisture flux. The performance of our stacked model, which is enriched with breaking wave variables, is validated against the fifth-generation European Centre for Medium-Range Weather Forecasts reanalysis data (ERA5). The results offer excellent predictions and highlight the importance of breaking-wave-related variables in regulating moisture flux, thereby substantiating the integral role of wave breaking in modulating air-sea interactions and moisture transport.

M3S-ALG: Improved and robust prediction of allergenicity of chemical compounds by using a novel multi-step stacking strategy

A wide variety of chemicals cannot be introduced to the marketplace because of their high allergenicity. Therefore, it is fundamentally crucial to assess the allergenic potential of chemicals before introducing them into clinical therapeutics. However, assessing the allergenicity of chemical compounds experimentally is time-consuming and costly. To tackle this challenge, we propose M3S-ALG, a novel multi-step stacking strategy (M3S) for rapid and accurate identification of the allergenicity of chemical compounds by using only the SMILES notation. The proposed M3S method involves three steps, as follows. First, ten different balanced datasets were constructed using an under-sampling approach. Second, for each balanced dataset, 144 base-classifiers were trained and optimized to generate the prediction scores of allergenic chemical compounds considered as new probabilistic features. Third, we selected the important probabilistic features and employed them to construct the final stacked model (M3S-ALG). Experimental results show that M3S-ALG outperforms conventional ensemble strategies and its constituent base-classifiers on both the training and independent test datasets. This indicates the effectiveness and robustness of our proposed strategy in identifying the allergenicity of chemical compounds. In addition, M3S-ALG exhibited excellent prediction performance compared to existing methods on the independent test dataset, achieving a balanced accuracy of 0.877, MCC of 0.712, and AUC of 0.931. Finally, we developed a user-friendly online web server at https://pmlabqsar.pythonanywhere.com/M3SALG. This new approach is anticipated to facilitate the drug discovery and development community for the large-scale identification of chemical compounds with no allergenic properties.

Predicting viscosity for steelmaking slag: A stacking regression approach

The viscosity of steel slag plays a pivotal role in steelmaking, impacting various aspects of the steelmaking operation and product quality. This paper introduces a novel approach for predicting the viscosity of turndown slag from a basic oxygen furnace using a stacking regression model. The stacking model in this approach incorporates four base models: multiple linear regression (LR), random forest (RF), k-nearest neighbour (KNN) and AdaBoost (ADB), while random forest is selected as a meta-model. The selection of meta-models is done through a systematic framework provided for identifying suitable meta-models in a stacking model. Similarly, the framework also demonstrated the workflow for constructing and evaluating a series of stacking models with different configurations. On the evaluation of 20 stacking models (including two and three-layers), the optimal configuration is found to be LR + KNN with R2 value of 0.993 and the shortest training time. Additionally, a correlation analysis of slag viscosity with composition and temperature is done, which will help the operators of steel melting shops in optimize the steelmaking process based on the required slag viscosity. Ultimately, this study emphasizes the efficacy of optimal-configured stacking regression models in industrial applications, achieving both maximum accuracy and minimal computational time.

A two-tier feature selection method for predicting mortality risk in ICU patients with acute kidney injury

Acute kidney injury (AKI) is one of the most important lethal factors for patients admitted to intensive care units (ICUs), and timely high-risk prognostic assessment and intervention are essential to improving patient prognosis. In this study, a stacking model using the MIMIC-III dataset with a two-tier feature selection approach was developed to predict the risk of in-hospital mortality in ICU patients admitted for AKI. External validation was performed using separate MIMIC-IV and eICU-CRD. The area under the curve (AUC) was calculated using the stacking model, and features were selected using the Boruta and XGBoost feature selection methods. This study compares the performance of a stacking model using two-tier feature selection with a model using single-tier feature selection (XGBoost: 85; Boruta: 83; two-tier: 0.91). The predictive effectiveness of the stacking model was further validated by using different datasets (Validation 1: 0.83; Validation 2: 0.85) and comparing it with a simpler model and traditional clinical scores (SOFA: 0.65; APACH IV: 0.61). In addition, this study combined interpretable techniques and causal inference to analyze the causal relationship between features and predicted outcomes.

Predicting Grapevine Physiological Parameters Using Hyperspectral Remote Sensing Integrated with Hybrid Convolutional Neural Network and Ensemble Stacked Regression

Grapevine rootstocks are gaining importance in viticulture as a strategy to combat abiotic challenges, as well as enhance scion physiology. Direct leaf-level physiological parameters like net assimilation rate, stomatal conductance to water vapor, quantum yield of PSII, and transpiration can illuminate the rootstock effect on scion physiology. However, these measures are time-consuming and limited to leaf-level analysis. This study used different rootstocks to investigate the potential application of aerial hyperspectral imagery in the estimation of canopy level measurements. A statistical framework was developed as an ensemble stacked regression (REGST) that aggregated five different individual machine learning algorithms: Least absolute shrinkage and selection operator (Lasso), Partial least squares regression (PLSR), Ridge regression (RR), Elastic net (ENET), and Principal component regression (PCR) to optimize high-throughput assessment of vine physiology. In addition, a Convolutional Neural Network (CNN) algorithm was integrated into an existing REGST, forming a hybrid CNN-REGST model with the aim of capturing patterns from the hyperspectral signal. Based on the findings, the performance of individual base models exhibited variable prediction accuracies. In most cases, Ridge Regression (RR) demonstrated the lowest test Root Mean Squared Error (RMSE). The ensemble stacked regression model (REGST) outperformed the individual machine learning algorithms with an increase in R2 by (0.03 to 0.1). The performances of CNN-REGST and REGST were similar in estimating the four different traits. Overall, these models were able to explain approximately 55–67% of the variation in the actual ground-truth data. This study suggests that hyperspectral features integrated with powerful AI approaches show great potential in tracing functional traits in grapevines.

Enhancing genomic prediction with Stacking Ensemble Learning in Arabica Coffee.

Coffee Breeding programs have traditionally relied on observing plant characteristics over years, a slow and costly process. Genomic selection (GS) offers a DNA-based alternative for faster selection of superior cultivars. Stacking Ensemble Learning (SEL) combines multiple models for potentially even more accurate selection. This study explores SEL potential in coffee breeding, aiming to improve prediction accuracy for important traits [yield (YL), total number of the fruits (NF), leaf miner infestation (LM), and cercosporiosis incidence (Cer)] in Coffea Arabica. We analyzed data from 195 individuals genotyped for 21,211 single-nucleotide polymorphism (SNP) markers. To comprehensively assess model performance, we employed a cross-validation (CV) scheme. Genomic Best Linear Unbiased Prediction (GBLUP), multivariate adaptive regression splines (MARS), Quantile Random Forest (QRF), and Random Forest (RF) served as base learners. For the meta-learner within the SEL framework, various options were explored, including Ridge Regression, RF, GBLUP, and Single Average. The SEL method was able to predict the predictive ability (PA) of important traits in Coffea Arabica. SEL presented higher PA compared with those obtained for all base learner methods. The gains in PA in relation to GBLUP were 87.44% (the ratio between the PA obtained from best Stacking model and the GBLUP), 37.83%, 199.82%, and 14.59% for YL, NF, LM and Cer, respectively. Overall, SEL presents a promising approach for GS. By combining predictions from multiple models, SEL can potentially enhance the PA of GS for complex traits.

TriStack enables accurate identification of antimicrobial and anti-inflammatory peptides by combining machine learning and deep learning approaches

The identification of antimicrobial peptides (AMPs) and anti-inflammatory peptides (AIPs) is crucial for drug design and disease treatment. However, it remains a computational challenge to accurately identify these peptides due to insufficient information encoding the peptide sequences. In this study, we propose TriStack, a powerful and interpretable model for accurate identification of AMPs and AIPs by stacking a machine learning-based module using a multi-layer residual network. It first extracts three types of function-related features from peptide sequences to comprehensively characterize the composition, distribution, and physicochemical properties of residues. Furthermore, these features are fused and fed into a two-module stacked model. The first module provides the preliminary predictions based on three machine learning methods, while the second module refines these predictions further via a multi-layer residual network. After training and testing, TriStack outperforms all the compared leading methods for both AMPs and AIPs predictions. TriStack is expected to contribute to antimicrobial and anti-inflammatory drug based on peptide sequences.

Cardiovascular disease prediction based on Stacking integrated strategy

According to the report of the World Health Organization, cardiovascular disease causes at least 17.1 million deaths worldwide every year, ranking second among the top ten causes of death for many years, and is still a problem to be solved in China and even in the world. No less dangerous than cancer on the list. Early diagnosis is of great significance for patients with cardiovascular diseases, which can diagnose patients with cardiovascular diseases as early as possible to achieve the purpose of early treatment and reduce the cost and pain of patients. In order to achieve accurate early diagnosis of cardiovascular diseases, Logistic Regression, Decision Tree, Random Forest, Gradient Boosting (GBDT), Adaptive Boosting (AdaBoost), eXtreme Gradient Boosting (XGBoost), Deep Neural Network (DNN), and Stacking integrated models were used to predict cardiovascular diseases. The comparison results showed that the Stacking model is the optimal prediction model. The precision reached 86.80%, the recall reached 84.78%, and the f1 reached 85.76%. The proposed model can be used in cardiovascular disease prediction to reduce the incidence of cardiovascular disease.

Comprehensive analysis of pathological and biochemical data of prostate tumors based on multiple machine learning models

This study conducts a comprehensive analysis of global prostate tumor cases, employing various machine learning models for classification. The research focuses on data preprocessing, feature selection, and model training, utilizing effective techniques for handling biochemical and pathological data. In model training, a range of algorithms such as XGBoost, Random Forest, LightGBM, and CatBoost were explored, ultimately achieving an 89% test accuracy through a Stacking model. For pathological data, strategies included addressing missing values, text cleaning, and segmentation, combining CountVectorizer with multiple algorithms. LightGBM demonstrated optimal performance with an accuracy of 82%. The Bert model, post-segmentation and Focal loss optimization, achieved an accuracy of 80.39%. In summary, this research leverages machine learning to provide a convenient approach for diagnosing and treating prostate tumors, offering robust model support for predicting related diseases.