Bayesian Model Averaging Framework Research Articles

Integrating Hydrological and Machine Learning Models for Enhanced Streamflow Forecasting via Bayesian Model Averaging in a Hydro-Dominant Power System

Streamflow forecasting plays a crucial role in the operational planning of hydro-dominant power systems, providing valuable insights into future water inflows to reservoirs and hydropower plants. It relies on complex mathematical models, which, despite their sophistication, face various uncertainties affecting their performance. These uncertainties can significantly influence both short-term and long-term operational planning in hydropower systems. To mitigate these effects, this study introduces a novel Bayesian model averaging (BMA) framework to improve the accuracy of streamflow forecasts in real hydro-dominant power systems. Designed to serve as an operational tool, the proposed framework incorporates predictive uncertainty into the forecasting process, enhancing the robustness and reliability of predictions. BMA statistically combines multiple models based on their posterior probability distributions, producing forecasts from the weighted averages of predictions. This approach updates weights periodically using recent historical data of forecasted and measured streamflows. Tested on inflows to 139 reservoirs and hydropower plants in Brazil, the proposed BMA framework proved to be more skillful than individual models, showing improvements in forecasting accuracy, especially in the South and Southeast regions of Brazil. This method offers a more reliable tool for streamflow prediction, enhancing decision making in hydropower system operations.

Fusion of gauge-based, reanalysis, and satellite precipitation products using Bayesian model averaging approach: Determination of the influence of different input sources

Selection of the number and which of multisource precipitation datasets is crucially important for precipitation fusion. Considering the effects of different inputs, this study proposes a new framework based on the Bayesian model averaging (BMA) algorithm to integrate precipitation information from gauge-based analysis CPC, reanalysis-derived dataset ERA5, and satellite-retrieval products IMERG-E and GSMaP-RT. The BMA weights were optimized for the period 2001–2010 using daily measurements and then applied to the period 2011–2015 for model validation. Seven BMA-merged precipitation products (i.e., MCE, MCI, MCG, MCEI, MCEG, MCIG, and MCEIG) were thoroughly evaluated across mainland China and then compared against the state-of-the-art ensemble-based product, MSWEP. The results indicate that the BMA predictions performed substantially better than the reanalysis and satellite precipitation datasets in both daily statistics and seasonal analyses. MCE, MCI, and MCEG demonstrated better performances relative to CPC in terms of individual metrics. Moreover, MCI, MCG, and MCEI generally outperformed MSWEP over the entire study area, particularly in local regions, such as southwestern China and the eastern Tibetan Plateau. During Typhoon Rammasun in 2014, MCG and MCEG provided greater detail for heavy rainfall events than the four ensemble members and the MSWEP product. Thus, the performance of the BMA predictions exhibited evident differences because of various input sources. CPC was the major internal influencing factor with the highest weight score. Meanwhile, the increased-input CPC dataset into the BMA-based schemes exerted a significant influence on the precipitation estimates, which markedly facilitated the performance improvement of the fusion model, and its improved degree (greater than 14 %) was obtained using a ‘changed-initial’ comparison method. Our results demonstrate that the developed modifiable BMA framework is useful for analyzing the impacts of ensemble members on BMA predictions and suggests that it is considerate in the use of different input sources for generating ensemble-based precipitation products.

Bayesian Model Averaging to Account for Model Uncertainty in Estimates of a Vaccine's Effectiveness.

PurposeVaccine effectiveness (VE) studies are often conducted after the introduction of new vaccines to ensure they provide protection in real-world settings. Control of confounding is often needed during the analyses, which is most efficiently done through multivariable modeling. When many confounders are being considered, it can be challenging to know which variables need to be included in the final model. We propose an intuitive Bayesian model averaging (BMA) framework for this task.Patients and MethodsData were used from a matched case–control study that aimed to assess the effectiveness of the Lyme vaccine post-licensure. Cases were residents of Connecticut, 15–70 years of age with confirmed Lyme disease. Up to 2 healthy controls were matched to each case subject by age. All participants were interviewed, and medical records were reviewed to ascertain immunization history and evaluate potential confounders. BMA was used to systematically search for potential models and calculate the weighted average VE estimate from the top subset of models. The performance of BMA was compared to three traditional single-best-model-selection methods: two-stage selection, stepwise elimination, and the leaps and bounds algorithm.ResultsThe analysis included 358 cases and 554 matched controls. VE ranged between 56% and 73% and 95% confidence intervals crossed zero in <5% of all candidate models. Averaging across the top 15 models, the BMA VE was 69% (95% CI: 18–88%). The two-stage, stepwise, and leaps and bounds algorithm yielded VE of 71% (95% CI: 21–90%), 73% (95% CI: 26–90%), and 74% (95% CI: 27–91%), respectively.ConclusionThis paper highlights how the BMA framework can be used to generate transparent and robust estimates of VE. The BMA-derived VE and confidence intervals were similar to those estimated using traditional methods. However, by incorporating model uncertainty into the parameter estimation, BMA can lend additional rigor and credibility to a well-designed study.

Sub-daily soil moisture estimate using dynamic Bayesian model averaging

Accurate estimation of soil moisture (SM) from satellite products and model simulations at sub-daily timescale remains a challenge. This study proposes a general dynamic Bayesian model averaging (BMA) framework for merging sub-daily model products. Compared to the traditional BMA method, this study introduces adaptive weights (dynamically variant with time) for BMA members. Based on the previous evaluation work, a subset of model products is selected from eight model products as BMA members. The dynamic BMA experiment is performed for the surface SM (0–10 cm) model products at sub-daily (6-h) timescale in 2017 over the Yangtze-Huaihe river basin. The results are compared with the automatic SM observations (ASMOs) with unprecedented high spatial and temporal resolution (up to 7 stations within a 104 km2 pixel; hourly). Because weather pattern and model performance change over time, the determination of an optimal training period is critical to obtain adaptive BMA weights for rapid weather regime changes. The sensitivity of training length (days) is then examined, and the optimum data length used in the BMA training period proves to be about 80 days. With deterministic and probabilistic verification metrics, the dynamic BMA estimated SM is comprehensively evaluated against the ASMOs, eight global model products, and the CMA’s (China Meteorological Administration) regional Land Data Assimilation System (CLDAS) product. To better compare the probability distribution of different products, the cumulative distribution function (CDF) consistency histogram and a more objective metric consistency deviation (CD) are proposed to diagnose the consistency of two SM CDFs (e.g., the BMA estimated and the observed CDF). In terms of both the deterministic (the Kling-Gupta efficiency, correlation, system bias, and bias adjusted root-mean square error) and probabilistic verification methods (CD, QQ-plots, and reliability), the dynamic BMA estimated SM outperforms any BMA members and even the benchmark product CLDAS. This study demonstrates that the dynamic BMA framework provides a new solution for merging SM model products. The merged SM and the BMA combined probability distribution can be further used for drought monitoring and prediction.

Bayesian Model Averaging as an Alternative to Model Selection for Multilevel Models

We investigated the Bayesian model averaging (BMA) technique as an alternative method to the traditional model selection approaches for multilevel models (MLMs). BMA synthesizes the information derived from all possible models and comes up with a weighted estimate. A simulation study compared BMA with additional modeling techniques, including the single “best” model approach, Bayesian MLM using informative, diffuse, and inaccurate priors, and restricted maximum likelihood. A two-level random intercept and random slope model was examined with these modeling techniques. Generated data used two types of true models: a full MLM and a reduced MLM. Findings of the simulation study suggested that BMA was a trustworthy alternative to traditional model comparison and selection approaches through the Bayesian and the frequentist frameworks. We also include an empirical example highlighting the extension of MLMs into the BMA framework, as well as model interpretation.

Sorting through global corruption determinants: Institutions and education matter – Not culture

Abstract Identifying the robust determinants of corruption among cultural, economic, institutional, and geographical factors has proven difficult. From a policy perspective, it is important to know whether inherent, largely unchangeable attributes are responsible or if institutional and economic attributes are at work. Accounting for model uncertainty, we use Bayesian Model Averaging (BMA) to analyze a comprehensive list of 36 potential corruption determinants across 123 countries (covering 87 percent of the world population). The BMA methodology sorts through all 68,719,476,736 possible model combinations ( 2 36 ) in order to carve out the robust correlates. We then take a step toward alleviating endogeneity concerns in an Instrumental Variable BMA framework. Our results indicate that cultural factors are largely irrelevant, whereas particular economic and institutional characteristics matter. The rule of law emerges as the most persistent predictor with a posterior inclusion probability (PIP) in the true model of 1.00, whereas we find strong evidence for government effectiveness (PIP of 0.88), urbanization (0.85), and the share of women in parliament (0.80) as meaningful determinants of lower corruption levels. In developing countries, the extent of primary schooling enters as a powerful factor with a PIP of 1.00.

Towards Improving the Efficiency of Bayesian Model Averaging Analysis for Flow in Porous Media via the Probabilistic Collocation Method

The characterization of flow in subsurface porous media is associated with high uncertainty. To better quantify the uncertainty of groundwater systems, it is necessary to consider the model uncertainty. Multi-model uncertainty analysis can be performed in the Bayesian model averaging (BMA) framework. However, the BMA analysis via Monte Carlo method is time consuming because it requires many forward model evaluations. A computationally efficient BMA analysis framework is proposed by using the probabilistic collocation method to construct a response surface model, where the log hydraulic conductivity field and hydraulic head are expanded into polynomials through Karhunen–Loeve and polynomial chaos methods. A synthetic test is designed to validate the proposed response surface analysis method. The results show that the posterior model weight and the key statistics in BMA framework can be accurately estimated. The relative errors of mean and total variance in the BMA analysis results are just approximately 0.013% and 1.18%, but the proposed method can be 16 times more computationally efficient than the traditional BMA method.

A multiple crop model ensemble for improving broad-scale yield prediction using Bayesian model averaging

Process-based crop models are popular tools to evaluate the impact of climate change and agricultural management on crop growth. Accurate simulation of crop production over large geographic regions using an individual crop model remains challenging due to different sources of uncertainty. We present a Bayesian model averaging (BMA) method for a multiple crop-growth model ensemble to provide more reliable predictions of maize yields in Liaoning Province, northeastern China, which covers an area of 148,000km2 and has 2200,000ha of maize. We apply the photosynthesis-oriented WOFOST (WOrld FOod STudy) model, the water-oriented AquaCrop model and the nitrogen-oriented DNDC (DeNitrification and DeComposition) model to independently generate original predictions of county-level maize yields. The integrated prediction is achieved using a linear combination of the three ensemble members using BMA weights. This integrated approach results in more accurate and precise predictions than any individual model over the entire province. This is because the BMA framework effectively compensates for the uncertainty of individual model simulation and takes advantage of each competing model for reliable prediction. Furthermore, the interpretation of the BMA weight values is also strengthened by comparison with regional precipitation, fertilization and radiation data. We find these values adequately fit the regional limiting factors, e.g., the AquaCrop model generally has a high weight value in counties with frequent droughts, while WOFOST is the dominant member in areas with radiation deficit. Compared with the simple average method and median estimate, the results show that the BMA framework is powerful in computing the ensemble weights and interpreting the mechanism beyond the observed data.

Asset Returns Under Model Uncertainty: Evidence from the Euro Area, the US and the UK

We analyze predictability of risk premium in the context of model uncertainty. Using data for the euro area, the US and the UK, we show that there is a large amount of model uncertainty and one can improve the forecasts of stock returns with a Bayesian Model Averaging (BMA) approach. The empirical evidence for the euro area suggests that several macroeconomic, financial and macro-financial variables are consistently among the most prominent determinants of risk premium. As for the US, only a few number of predictors play an important role. In the case of the UK, future stock returns are better forecasted by financial variables. These results are corroborated for both the M-open and the M-closed perspectives, different model priors and in the context of “in-sample” and “out-of-sample” forecasting. Finally, we highlight that the predictive ability of the BMA framework is stronger at longer periods, and clearly outperforms the constant expected returns and the autoregressive benchmark models.

Model uncertainty in Panel Vector Autoregressive models

We develop methods for Bayesian model averaging (BMA) or selection (BMS) in Panel Vector Autoregressions (PVARs). Our approach allows us to select between or average over all possible combinations of restricted PVARs where the restrictions involve interdependencies between and heterogeneities across cross-sectional units. The resulting BMA framework can find a parsimonious PVAR specification, thus dealing with overparameterization concerns. We use these methods in an application involving the euro area sovereign debt crisis and show that our methods perform better than alternatives. Our findings contradict a simple view of the sovereign debt crisis which divides the euro zone into groups of core and peripheral countries and worries about financial contagion within the latter group.

Limited Information Bayesian Model Averaging for Dynamic Panels with Short Time Periods

Bayesian Model Averaging (BMA) provides a coherent mechanism to address the problem of model uncertainty. In this paper we extend the BMA framework to panel data models where the lagged dependent variable as well as endogenous variables appear as regressors. We propose a Limited Information Bayesian Model Averaging (LIBMA) methodology and then test it using simulated data. Simulation results suggest that asymptotically our methodology performs well both in Bayesian model selection and averaging. In particular, LIBMA recovers the data generating process very well, with high posterior inclusion probabilities for all the relevant regressors, and parameter estimates very close to the true values. These findings suggest that our methodology is well suited for inference in dynamic panel data models with short time periods in the presence of endogenous regressors under model uncertainty.

Limited Information Bayesian Model Averaging for Dynamic Panels with Short Time Periods

Bayesian Model Averaging (BMA) provides a coherent mechanism to address the problem of model uncertainty. In this paper we extend the BMA framework to panel data models where the lagged dependent variable as well as endogenous variables appear as regressors. We propose a Limited Information Bayesian Model Averaging (LIBMA) methodology and then test it using simulated data. Simulation results suggest that asymptotically our methodology performs well both in Bayesian model selection and averaging. In particular, LIBMA recovers the data generating process very well, with high posterior inclusion probabilities for all the relevant regressors, and parameter estimates very close to the true values. These findings suggest that our methodology is well suited for inference in dynamic panel data models with short time periods in the presence of endogenous regressors under model uncertainty.

Combining multiobjective optimization and Bayesian model averaging to calibrate forecast ensembles of soil hydraulic models

Most studies in vadose zone hydrology use a single conceptual model for predictive inference and analysis. Focusing on the outcome of a single model is prone to statistical bias and underestimation of uncertainty. In this study, we combine multiobjective optimization and Bayesian model averaging (BMA) to generate forecast ensembles of soil hydraulic models. To illustrate our method, we use observed tensiometric pressure head data at three different depths in a layered vadose zone of volcanic origin in New Zealand. A set of seven different soil hydraulic models is calibrated using a multiobjective formulation with three different objective functions that each measure the mismatch between observed and predicted soil water pressure head at one specific depth. The Pareto solution space corresponding to these three objectives is estimated with AMALGAM and used to generate four different model ensembles. These ensembles are postprocessed with BMA and used for predictive analysis and uncertainty estimation. Our most important conclusions for the vadose zone under consideration are (1) the mean BMA forecast exhibits similar predictive capabilities as the best individual performing soil hydraulic model, (2) the size of the BMA uncertainty ranges increase with increasing depth and dryness in the soil profile, (3) the best performing ensemble corresponds to the compromise (or balanced) solution of the three‐objective Pareto surface, and (4) the combined multiobjective optimization and BMA framework proposed in this paper is very useful to generate forecast ensembles of soil hydraulic models.