Prediction Bias Research Articles

A Large-Scale Inter-Comparison and Evaluation of Spatial Feature Engineering Strategies for Forest Aboveground Biomass Estimation Using Landsat Satellite Imagery

Aboveground biomass (AGB) estimates derived from Landsat’s spectral bands are limited by spectral saturation when AGB densities exceed 150–300 Mg ha−1. Statistical features that characterize image texture have been proposed as a means to alleviate spectral saturation. However, apart from Gray Level Co-occurrence Matrix (GLCM) statistics, many spatial feature engineering techniques (e.g., morphological operations or edge detectors) have not been evaluated in the context of forest AGB estimation. Moreover, many prior investigations have been constrained by limited geographic domains and sample sizes. We utilize 176 lidar-derived AGB maps covering ∼9.3 million ha of forests in the Pacific Northwest of the United States to construct an expansive AGB modeling dataset that spans numerous biophysical gradients and contains AGB densities exceeding 1000 Mg ha−1. We conduct a large-scale inter-comparison of multiple spatial feature engineering techniques, including GLCMs, edge detectors, morphological operations, spatial buffers, neighborhood vectorization, and neighborhood similarity features. Our numerical experiments indicate that statistical features derived from GLCMs and spatial buffers yield the greatest improvement in AGB model performance out of the spatial feature engineering strategies considered. Including spatial features in Random Forest AGB models reduces the root mean squared error (RMSE) by 9.97 Mg ha−1. We contextualize this improvement model performance by comparing to AGB models developed with multi-temporal features derived from the LandTrendr and Continuous Change Detection and Classification algorithms. The inclusion of temporal features reduces the model RMSE by 18.41 Mg ha−1. When spatial and temporal features are both included in the model’s feature set, the RMSE decreases by 21.71 Mg ha−1. We conclude that spatial feature engineering strategies can yield nominal gains in model performance. However, this improvement came at the cost of increased model prediction bias.

Comprehensive propagation of errors for the prediction of woody biomass

Abstract Managing vegetation to sequester carbon in biomass requires estimates to meet standards for accuracy, with methods that are transparent, verifiable and cost‐effective. Allometric models are commonly used to predict biomass from non‐destructive field inventory data. Although a number of studies have addressed biomass error propagation, none have provided a general set of methods for linking errors all the way from initial allometric model development through to the final site‐based biomass prediction, for both above‐ and below‐ground biomass. Error sources in total biomass (above‐ + below‐ground) were quantified using a combination of analytical and Monte Carlo methods, illustrated with four contrasting case studies using either site‐ and‐species‐specific, species‐specific or generalised allometric models. Sampling error was found to be the most important contributor to site‐level biomass uncertainty, arising from the interaction between spatial variability and the field sampling design. The contribution of allometric model covariance to total error was also quantified, with errors in the determination of moisture content during allometric model development identified as a potentially important yet often overlooked error source. Application of different allometric models to the same inventory data suggested the error from generalised models was no greater than that from site‐ or species‐specific models, with increases in the generalised model prediction error balanced by decreases in other error sources associated with the increased sample size on which generalised models are based. Recommendations for reducing errors in predicted biomass include increasing field survey sample size, adopting field survey designs that ensure spatial representativeness and improving moisture content measurement protocols and increasing the moisture content sample size during allometric model development. To reduce costs while maintaining acceptable accuracy, the use of generalised allometric models is recommended, with the caveat that additional biomass sampling for model validation may be required to limit the potential for biased predictions.

Development of a machine learning model in prediction of the rapid progression of interstitial lung disease in patients with idiopathic inflammatory myopathy.

Rapidly progressive interstitial lung disease (RP-ILD) significantly impacts the prognosis of patients with idiopathic inflammatory myopathies (IIM). High-resolution computed tomography (HRCT) is a crucial noninvasive technique for evaluating interstitial lung disease (ILD). Utilizing quantitative computed tomography (QCT) enables accurate quantification of disease severity and evaluation of prognosis, thereby serving as a crucial computer-aided diagnostic method. This study aimed to establish and validate a machine learning (ML) model to predict RP-ILD in patients with idiopathic inflammatory myopathy-related interstitial lung disease (IIM-ILD) based on QCT and clinical features. A total of 514 patients (367 females, median age 54 years) with IIM-ILD in the China-Japan Friendship Hospital were retrospectively included, out of which 249 cases (165 females, median age 55 years) were identified as having RP-ILD. To extract the quantitative features on HRCT, deep learning (DL) methods were employed, along with demographic factors, pulmonary function test results, and blood gas analysis results; these factors were integrated into a final prediction model. Logistic regression was chosen as the final model due to its superior area under the curve (AUC) and explainability compared to the other seven ML models. The validation dataset yielded an AUC of 0.882 [95% confidence interval (CI): 0.797-0.967], indicating that the combined QCT and clinical features model outperformed both the QCT-only model and the clinically-only model. In calibration and clinical decision curve analysis, the final model demonstrated minimal prediction bias (concordance index: 0.887, 95% CI: 0.800-0.974, P<0.001) and provided greater net benefit across most thresholds. The nomogram encompassed the incorporation of the following variables: subtype, gender, forced expiratory volume in one second (FEV1%), diffusing capacity for carbon monoxide (DLCO%), oxygenation index (OI), and quantitative ground-glass opacities (GGOs), consolidation, pulmonary vascular, and branches on HRCT. When utilizing ML techniques, the baseline QCT has the potential to predict rapid progression in patients with IIM-ILD. The prediction performance will be further improved by incorporating clinical data alongside HRCT features. Idiopathic inflammatory myopathy (IIM); rapidly progressive interstitial lung disease (RP-ILD); high-resolution computed tomography (HRCT); machine learning (ML); quantitative computed tomography (QCT).

Deep Learning for Karolinska Sleepiness Scale Classification Based On Eye Aspect Ratio with SMOTE-Enhanced Data Balancing

This paper addresses the challenge of accurately classifying sleepiness levels based on the Karolinska Sleepiness Scale (KSS) using Eye Aspect Ratio (EAR) data, especially when class imbalance leads to biased predictions. The research proposes a deep learning framework that integrates a Multi-Layer Perceptron (MLP) with the Synthetic Minority Over-sampling Technique (SMOTE) to balance the training data. EAR features, representing eye closure patterns, are extracted from video frames, and SMOTE is applied to generate synthetic data for underrepresented sleepiness classes. By training the MLP model on this balanced dataset, the system achieves a 97.6% classification accuracy in distinguishing four distinct sleepiness levels based on the KSS, demonstrating its effectiveness in reducing prediction bias and managing class imbalance, both crucial for real-time drowsiness detection systems

Optimizing Diabetic Neuropathy Severity Classification Using Electromyography Signals Through Synthetic Oversampling Techniques

Electromyography signals are electrical signals generated by muscle activity and are very useful for analyzing the health conditions of muscles and nerves. Data imbalance is a prevalent issue in EMG signal data, especially when addressing patients with varied health conditions and restricted data availability. A major difficulty for machine learning models is class imbalance in datasets, which frequently leads to biased predictions favoring the dominant class and neglecting the minority classes. The data augmentation method employs the Synthetic Minority Over Sampling Technique (SMOTE) and Random Over Sampling (ROS) to address data imbalances and enhance the performance of classification models for underrepresented classes. This study employs an oversampling technique to enhance the efficacy of the XG Boost model. SMOTE exhibits better efficacy relative to competing methods; the application of appropriate oversampling techniques allows models to integrate patterns from both majority and often neglected minority data.

The Prioritization of Prospection.

Humans frequently engage in mental time travel, reflecting on the past and anticipating the future. Although these processes may seem similar, research documents systematic differences between retrospection and prospection. We propose a conceptual framework to organize and explain these differences based on three axiomatic temporal asymmetries: The past occurs before the future; the past is more certain than the future; and the past is less controllable than the future. People's experience with these axiomatic differences is internalized and overgeneralized to shape mental representations of the past and future. Our review shows that people generally prioritize prospection over retrospection, attending more to the future than the past and reacting more intensely to future events than to past events. We consider potential moderators of and constraints on the generality of prioritizing prospection. We explore the implications of these temporal asymmetries, emphasizing their theoretical and practical significance. While daily life centers on the present, people often reflect on the past and anticipate the future. But which direction of mental time travel-backward or forward-has more influence? We identify three key differences that shape how people engage with the past and future: time flows from past to future, the future is more uncertain, and people have more control over the future. These differences affect the frequency, intensity, and nature of thoughts and feelings, leading to predictable biases in how we mentally represent and emotionally engage with events over time. Because focusing on the future often provides greater benefits, people tend to prioritize prospection over retrospection in everyday life.

Boreal Summer Extratropical Intraseasonal Oscillation Prediction in Current Subseasonal‐To‐Seasonal Operational Models Over Eurasia

AbstractExtratropical intraseasonal oscillations (EISO), which is akin to the tropical ISO, significantly influence mid‐high‐latitude intraseasonal variability. Previous studies have shown that biases in the dynamical prediction of EISOs reduce the accuracy of subseasonal forecasts for extreme events, particularly in extratropical regions. However, the current dynamical prediction skills of EISO and associated dependence remain unclarified. Based on the recently proposed EISO metrics, which represent three dominant zonally propagating EISO modes traveling along the westerly jets, this study conducts a comprehensive evaluation of the EISOs' prediction skill in 13 state‐of‐the‐art subseasonal‐to‐seasonal (S2S) prediction project models. Firstly, the S2S models display skill to predict the EISOs out to a range of 9–20 days, which demonstrates higher predictability compared to other conventional circulation modes in the extratropics. In the subseasonal prediction, EISOs' amplitudes are underestimated, and their propagating speeds feature large random errors in all S2S models. Secondly, the EISOs' prediction skill depends on their initial/target amplitude, particularly for strong target amplitudes, which showcase higher skill than weak‐amplitude EISOs. Thirdly, although the EISOs' prediction skill is sensitive to the EISOs' phase, it is not locked to any fixed phase, indicating that EISOs do not have an inherent prediction barrier like the tropical ISO. This study also discusses how the humidity initialization and ensemble sizes influence the EISOs' prediction skill, suggesting that increasing the ensemble members in the saturation range is an effective way of improving the prediction for extratropical subseasonal variation and related extreme events.

Near Infrared Reflectance Spectroscopy-Driven Chemometric Modeling for Predicting Key Quality Traits in Lablab Bean (Lablab purpureus L.) Germplasm

Lablab bean (Lablab purpureus L.) is a multipurpose crop, commonly used for food, feed, and fodder, and its potential as a plant-based meat alternative. Its nutritional diversity, including high protein, starch, and phenolic content, makes it a suitable candidate for nutritional profiling, which is essential for developing nutritionally enhanced varieties. Traditional methods for analyzing its nutritional parameters are labor-intensive, time-consuming, and expensive. This study employs Near-Infrared Reflectance Spectroscopy (NIRS) as a rapid, non-destructive alternative to evaluate 112 Lablab bean genotypes. We developed prediction models for starch, amylose, protein, fat, and phenols using a Modified Partial Least Squares (MPLS) approach, with spectral pre-processing using Standard Normal Variate (SNV) to remove scatter effects and Detrending (DT) to reduce baseline shifts and noise. The models were optimized for derivatives, gap selection, and smoothing, and evaluated using independent test data and key performance metrics including coefficient of determination (R²), bias, and Residual Prediction Deviation (RPD). The best-performing models were: starch (R² = 0.959, RPD = 4.57), amylose (R² = 0.737, RPD = 1.76), protein (R² = 0.911, RPD = 3.09), fat (R² = 0.894, RPD = 2.92), and phenols (R² = 0.816, RPD = 2.36). Statistical tests, including paired t-tests, correlation, and reliability analysis, confirmed the robustness of these models. This study presents a first report offering rapid, multi-trait assessment method for evaluating Lablab bean germplasm, demonstrating high predictive accuracy for pre-breeding practices. It has broad applications in developing nutritionally enhanced varieties, supporting plant-based protein alternatives, and optimizing food production processes to meet the growing demand for healthier, sustainable foods.

The EU legal framework for using artificial intelligence and imaging databases and imaging biobanks for research purposes: applying the notion of Fairness

Objective: To analyze the issue of justice and discrimination in artificial intelligence systems based on medical image databases. Methodology: Analysis of documents that constitute the regulatory framework of the European Union for the use of artificial intelligence, compared with the report FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging. Results: The study indicates that artificial intelligence trained with unbalanced data tends to generate biased predictions, which can exacerbate health inequalities and affect justice. Discrimination in artificial intelligence systems appears abstract, subtle, and difficult to detect compared to traditional forms of discrimination. Final Considerations: Robust regulation is necessary to ensure justice in artificial intelligence systems, considering the need for interdisciplinary collaboration to prepare this new generation of legal professionals with an enhanced perspective on the topic and its various dimensions. Submission: 10/01/24| Review: 10/04/24| Approval: 10/04/24

Addressing Class Imbalance in Bayesian Classification Through Posterior Probability Adjustment

ABSTRACTClass imbalance is a known issue in classification tasks that can lead to predictive bias toward dominant classes. This paper introduces a novel straightforward Bayesian framework that adjusts posterior probabilities to counteract the bias introduced by imbalanced data sets. Instead of relying on the mean posterior distribution of class probabilities, we propose a method that scales the posterior probability of each class according to their representation in the training data.

Genomic Prediction Using Imputed Whole-Genome Sequence Data in Australian Angus Cattle.

Whole-genome sequence (WGS) data was used to estimate genomic breeding values for growth and carcass traits in Australian Angus cattle. The study aimed to compare the accuracy and bias of genomic predictions with three marker densities, including 50K, high-density (HD) and WGS. The dataset used in this study consisted of animals born between 2013 and 2022. Body weight traits included birthweight, weight at 400 days and weight at 600 days of age. The carcass traits were carcass weight, carcass intramuscular fat and carcass marbling score. The accuracy and bias of prediction were assessed using the cross-validation. Further, for the growth traits, animals in the validation group were subdivided into two subgroups, which were moderately or highly related to the reference. Genomic best linear unbiased prediction (GBLUP) was used to compare genomic predictions with the three marker densities. The prediction accuracies were generally similar across the marker densities, ranging between 0.61 and 0.68 for the body weight traits and between 0.40 and 0.52 for the carcass traits. However, the accuracies marginally decreased as the marker density increased for all the traits studied. A similar lack of difference was found when considering the accuracy by the relatedness subgroups. The results indicated that no meaningful difference in prediction accuracy was estimated when comparing the three marker densities due to the population structure. In conclusion, there was no substantial improvement in genomic prediction when using the WGS in this study.

Integrating data imputation and augmentation with interpretable machine learning for efficient strength prediction of fly ash-based alkali-activated concretes

Fly ash-based alkali-activated concrete (AAC) is renowned for its superior mechanical performance and sustainability, presenting an attractive alternative to traditional Portland cement concrete. Despite these advantages, the broad compositional range of AACs presents challenges in precisely tailoring material properties. In this context, machine learning (ML) offers promising prospects to streamline and fast-track the development of advanced materials design strategies by predicting mechanical properties from compositional variations. Effective ML model development, however, hinges on the availability of a comprehensive, high-quality dataset. Previous studies often relied on literature-derived datasets, which typically include outliers, noise, and missing values, potentially leading to biased predictions. Moreover, limited dataset sizes could undermine the robustness of the models. Traditional ML methods applied to AACs also tend to lack interpretability. To address these issues, this paper utilizes several data imputation methods and Generative Adversarial Networks (GANs) for data augmentation, effectively doubling the dataset size. Following this, ML algorithms such as Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Neural Networks (NNs) are leveraged to predict compressive strength. The NN model, especially when enhanced by k-nearest neighbors (kNN) imputation (k = 5), demonstrated superior predictive accuracy compared to RF and XGBoost models. Further, SHAP (SHapley Additive exPlanations) analysis reveals key determinants of compressive strength, such as water content, SiO2, and curing conditions. Visualizations such as SHAP violin and river flow plots further elucidated feature contributions and property distributions. Overall, this study provides a robust framework for exploring composition-strength relationships in AACs, advancing the design of these environment-friendly materials.

A Convolutional Neural Network–Long Short-Term Memory–Attention Solar Photovoltaic Power Prediction–Correction Model Based on the Division of Twenty-Four Solar Terms

The prevalence of extreme weather events gives rise to a significant degree of prediction bias in the forecasting of photovoltaic (PV) power. In order to enhance the precision of forecasting outcomes, this study examines the interrelationships between China’s 24 conventional solar terms and extreme meteorological events. Additionally, it proposes a methodology for estimating the short-term generation of PV power based on the division of solar term time series. Firstly, given that the meteorological data from the same festival is more representative of the climate state at the current prediction moment, the sample data are grouped according to the 24 festival time nodes. Secondly, a convolutional neural network–long short-term memory (CNN-LSTM) PV power prediction model based on an Attention mechanism is proposed. This model extracts temporal change information from nonlinear sample data through LSTM, and a CNN link is added at the front end of LSTM to address the issue of LSTM being unable to obtain the spatial linkage of multiple features. Additionally, an Attention mechanism is incorporated at the back end of the CNN to obtain the feature information of crucial time steps, further reducing the multi-step prediction error. Concurrently, a PV power error prediction model is constructed to rectify the outcomes of the aforementioned prediction model. The examination of the measured data from PV power stations and the comparison and analysis with other prediction models demonstrate that the model presented in this paper can effectively enhance the accuracy of PV power predictions.

Predictors of Trauma-Related Diagnostic Overshadowing Bias

Diagnostic overshadowing occurs when healthcare professionals misattribute an individual’s presenting symptoms to other features of an individual’s clinical presentation. Mental health providers may incorrectly diagnose and treat trauma-exposed individuals due to trauma-related diagnostic overshadowing bias. No research has investigated provider factors associated with this bias. Mental health provider background and training characteristics were examined as predictors of trauma-related diagnostic overshadowing in a sample of mental health providers (N = 210). Hierarchical regression modeling was used to evaluate predictors related to trauma-related diagnostic overshadowing in diagnostic and treatment decision-making. Providers with diagnostic responsibilities in their professional role were significantly less likely to demonstrate bias in some diagnostic and treatment decisions. Older mental health providers and providers with more clients experiencing traumatic stress demonstrated greater bias in some diagnostic responses. Providers with doctoral degrees, compared to all other degree types, demonstrated less bias through some diagnostic and treatment responses. Bias in diagnostic decisions strongly predicted bias in treatment decision-making. Findings suggest that factors related to clinical experience may be important in predicting trauma-related diagnostic overshadowing. Future work should extend this research to understand which factors affect trauma-related diagnostic overshadowing in real-world settings.

Wearing myopia glasses on GANs: Mitigating bias for pre-trained Generative Adversarial Networks via online prior perturbation

Pre-trained Generative Adversarial Networks (GANs) can provide rich information and make various downstream tasks beneficial. However, the training process of GANs potentially learns pre-existing biases in the historical data, which are not only inherited, but also amplified to some extent when using the generated data for prediction. Most available methods for bias mitigation utilize pre-processing or in-processing strategies to retrain GANs, which needs to access the raw data and also increases the time and computational cost required to train a fully expressive GAN. Therefore, we propose a novel post-processing mechanism to achieve fairness outputs of GANs through online prior perturbation. Unlike traditional output-based offline processing, our method mitigates biases from model inputs by designing a prior perturbation network, called “prior perturber”, to form a combined network with the pre-trained GAN. Specifically, we introduce a bias prediction network, called “bias predictor”, for online adversarial training with the prior perturber, which can decouple the generated representations from the bias features and improve the fairness of the downstream prediction results. In addition, the training process is independent of specific target labels, and the generated fair representations have good transferability. Experimental evaluations on five real datasets validate the effectiveness of the proposed method and the best utility is obtained on the same fairness level as compared to other output-based baseline methods. The result on fairness-utility trade-offs significantly improves the fairness of original samples by 56.40%, while the utility is reduced by only 2.60%.

Jointly stochastic fully symmetric interpolatory rules and local approximation for scalable Gaussian process regression

When exploring the broad application prospects of large-scale Gaussian process regression (GPR), three core challenges significantly constrain its full effectiveness: firstly, the O(n3) time complexity of computing the inverse covariance matrix of n training points becomes an insurmountable performance bottleneck when processing large-scale datasets; Secondly, although traditional local approximation methods are widely used, they are often limited by the inconsistency of prediction results; The third issue is that many aggregation strategies lack discrimination when evaluating the importance of experts (i.e. local models), resulting in a loss of overall prediction accuracy. In response to the above challenges, this article innovatively proposes a comprehensive method that integrates third-degree stochastic fully symmetric interpolatory rules (TDSFSI), local approximation, and Tsallis mutual information (TDSFSIRLA), aiming to fundamentally break through existing limitations. Specifically, TDSFSIRLA first introduces an efficient third-degree stochastic fully symmetric interpolatory rules, which achieves accurate approximation of Gaussian kernel functions by generating adaptive dimensional feature maps. This innovation not only significantly reduces the number of required orthogonal nodes and effectively lowers computational costs, but also maintains extremely high approximation accuracy, providing a solid theoretical foundation for processing large-scale datasets. Furthermore, in order to overcome the inconsistency of local approximation methods, this paper adopts the Generalized Robust Bayesian Committee Machine (GRBCM) as the aggregation framework for local experts. GRBCM ensures the harmonious unity of the prediction results of each local model through its inherent consistency and robustness, significantly improving the stability and reliability of the overall prediction. More importantly, in response to the issue of uneven distribution of expert weights, this article creatively introduces Tsallis mutual information as a metric for weight allocation. Tsallis mutual information, with its sensitive ability to capture information complexity, assigns weights to different local experts that match their contribution, effectively solving the problem of prediction bias caused by uneven weight distribution and further improving prediction accuracy. In the experimental verification phase, this article conducted comprehensive testing on multiple synthetic datasets and seven representative real datasets. The results show that the TDSFSIRLA method not only achieves significant reduction in time complexity, but also demonstrates excellent performance in prediction accuracy, fully verifying its significant advantages and broad application prospects in the field of large-scale Gaussian process regression.

Reliable uncertainty estimation in emotion recognition in conversation using conformal prediction framework

Abstract Emotion recognition in conversation (ERC) faces two major challenges: biased predictions and poor calibration. Classifiers often disproportionately favor certain emotion categories, such as neutral, due to the structural complexity of classifiers, the subjective nature of emotions, and imbalances in training datasets. This bias results in poorly calibrated predictions where the model’s predicted probabilities do not align with the true likelihood of outcomes. To tackle these problems, we introduce the application of conformal prediction (CP) into ERC tasks. CP is a distribution-free method that generates set-valued predictions to ensure marginal coverage in classification, thus improving the calibration of models. However, inherent biases in emotion recognition models prevent baseline CP from achieving a uniform conditional coverage across all classes. We propose a novel CP variant, class spectrum conformation, which significantly reduces coverage bias in CP methods. The methodologies introduced in this study enhance the reliability of prediction calibration and mitigate bias in complex natural language processing tasks.

Drosophila Eye Gene Regulatory Network Inference Using BioGRNsemble: An Ensemble-of-Ensembles Machine Learning Approach

Background: Gene regulatory networks (GRNs) are complex gene interactions essential for organismal development and stability, and they are crucial for understanding gene-disease links in drug development. Advances in bioinformatics, driven by genomic data and machine learning, have significantly expanded GRN research, enabling deeper insights into these interactions. Methods: This study proposes and demonstrates the potential of BioGRNsemble, a modular and flexible approach for inferring gene regulatory networks from RNA-Seq data. Integrating the GENIE3 and GRNBoost2 algorithms, the BioGRNsemble methodology focuses on providing trimmed-down sub-regulatory networks consisting of transcription and target genes. Results: The methodology was successfully tested on a Drosophila melanogaster Eye gene expression dataset. Our validation analysis using the TFLink online database yielded 3703 verified predicted gene links, out of 534,843 predictions. Conclusion: Although the BioGRNsemble approach presents a promising method for inferring smaller, focused regulatory networks, it encounters challenges related to algorithm sensitivity, prediction bias, validation difficulties, and the potential exclusion of broader regulatory interactions. Improving accuracy and comprehensiveness will require addressing these issues through hyperparameter fine-tuning, the development of alternative scoring mechanisms, and the incorporation of additional validation methods.

Enhancing Machine Learning Models Through PCA, SMOTE-ENN, and Stochastic Weighted Averaging

Predicting survival outcomes in critical accidents has been a focal point in machine learning research. This study addresses several limitations of existing methods, including insufficient management of data imbalance, lack of emphasis on hyperparameter tuning, and proneness to overfitting. Many existing models struggle to generalize effectively on imbalanced datasets or depend on default hyperparameter settings, resulting in biased predictions. By integrating Principal Component Analysis (PCA), hyperparameter optimization, and resampling methods, as well as combining Edited Nearest Neighbors (ENN) with the Synthetic Minority Oversampling Technique (SMOTE), the model significantly improves predictive accuracy and model generalization. An ensemble model combining seven machine learning algorithms—Logistic Regression, Support Vector Machine, KNN, Random Forest, XGBoost, LightGBM, and CatBoost—was applied to predict survival outcomes. Stochastic Weighted Averaging (SWA) was applied to mitigate overfitting and enhance generalization. The accuracy increased from 91.97% to 94.89% after SWA was applied in this specific scenario. The combination of PCA-based dimensionality reduction, hyperparameter tuning, and resampling techniques (ENN + SMOTE) ensured the model handled data imbalance and optimized predictive accuracy. The final model demonstrated excellent performance, with Area Under the Curve (AUC) and Average Precision (AP) values both reaching 0.98, indicating high accuracy and precision. These improvements were validated using the Titanic dataset in a binary classification problem of predicting passenger survival. The results emphasize that ensemble learning, enhanced by SWA, offers a powerful framework for handling imbalanced and complex datasets, providing significant advancements in predictive modeling accuracy. This study provides insights into how machine learning techniques can be effectively combined to solve classification challenges in real-world scenarios.

Data driven cost-sensitive boosted tree for interpretable banking systemic risk prediction

Systemic risk (SR) in the banking sector poses a significant threat to both the financial system and the real economy. Its inherent characteristics of nonlinearity, non-equilibrium, and interconnectedness make it challenging to analyze using conventional statistical methods. In this paper, a cost-sensitive gradient boosting tree algorithm, FLXGBoost, is proposed for predicting SR. FLXGBoost considers the boosted tree, XGBoost as the base framework, boosting trees as the fundamental framework, guaranteeing the robustness of SR prediction. Additionally, to tackle the challenge of extreme data imbalance prevalent in SR prediction tasks, a cost-aware loss function, focal loss, is embedded into the boosted tree to enable FLXGBoost a risk-aware fashion. Moreover, a tree-derived interpretable algorithm SHAP is incorporated into this cost-sensitive solution, making FLXGBoost an accurate and interpretable risk-aware model. Experimental results on a financial risk prediction dataset pertaining to banking SR evince the capacity of FLXGBoost to significantly reduce the misclassification rate of risk banks, thereby mitigating substantial losses attributed to erroneous predictions of risky scenarios. Moreover, compared with classical imbalanced machine learning-based SR prediction approaches, the diverse evaluation metrics of FLXGBoost show that it is a competitive solution for accurate SR prediction. Besides, the explanatory analysis further demonstrates that FLXGBoost is a promising solution to address the issue of biased predictions in imbalanced banking SR in the interpretation perspective.