Bayesian Model Selection Criteria Research Articles

Characterizing the spatial correlation of coseismic slip distributions: A data driven Bayesian approach

Summary The spatial correlation of coseismic slip is a necessary input for generating stochastic seismic rupture models, which are commonly employed in seismic and tsunami hazard assessments. To date, the spatial correlation of individual earthquakes is characterized using finite fault models by finding the combination of parameters of a von Kármán autocorrelation function that best fits the observed autocorrelation function of the finite fault model. However, because a priori spatial correlation conditions (i.e., not in the data) are generally applied in finite fault model generation, the results obtained using this method may be biased. Additionally, robust uncertainty estimates for spatial correlations of coseismic slip are generally not performed. Considering these limitations in the classic method, here, a method is developed based on a Bayesian formulation of Finite Fault Inversion (FFI) with positivity constraints. This method allows for characterizing the spatial correlation of coseismic slip and its uncertainties for an earthquake by using samples of coseismic slip from a posterior probability density function (PDF). Furthermore, a Bayesian model selection criterion called Akaike Bayesian Information Criterion (ABIC) is applied to objectively choose between different prior spatial correlation schemes before computing the posterior, to reduce subjectivity due to this prior condition. The ABIC is calculated using an approximate analytical expression of Bayesian evidence. The method is applied to simulated P-waves, demonstrating that model selection allows for objectively estimating the most suitable prior spatial correlation scheme in FFI. Additionally, the target spatial correlation of coseismic slip is accurately recovered using samples from the posterior PDF, as well as their uncertainties. Moreover, in the simulated experiment, it is shown that a non-robust choice of the prior spatial correlation scheme can significantly bias the estimated spatial correlations of coseismic slip. We apply our method to observed P-waves from the 2015, Illapel earthquake (Mw = 8.3), finding that the spatial correlation of coseismic slip of this earthquake is better described by a von Kármán ACF, with mean correlation lengths of around 47 km and Hurst parameter of 0.58. We conclude that using our method reduces biases associated with prior spatial correlation conditions and allows for robust estimation of spatial correlations of coseismic slip and their uncertainties.

Bayesian identification and estimation of radon-related increased hazard rates of cancer death in the updated French cohort of uranium miners (1946-2014).

A recent update of the French cohort of uranium miners added seven years of follow-up data. We use these new data to look for new possible radon-related increased risks and refine the estimation of the potential association between cumulative radon exposure and four cancer sites: lung cancer, kidney cancer, brain and central nervous system (CNS) cancer and leukemia (excluding chronic lymphocytic leukemia, which is not radiation-induced). Several parametric survival models are proposed, fitted and compared under the Bayesian paradigm, to perform new and original exposure-risk analyses. In line with recent UNSCEAR recommendations, we consider time-related effect modifiers and exposure rate as potential effect modifying factors. We use Bayesian model selection criteria to identify radon-related increased hazard rates. Under the assumption of a linear exposure-risk relationship, we found a substantial evidence for a strictly positive effect of cumulative radon exposure on the hazard rate of death by lung cancer among French uranium miners. Given the current available data under the assumptions of a linear or log-linear exposure-risk relationship, it is not possible to conclude in favour of the absence or the existence of a strictly positive effect of chronic exposure to radon on the hazard rate of death by kidney cancer. Regarding death by brain and CNS cancer, there is a substantial evidence for the absence of radon-related effect. Finally, under the assumption of a log-linear exposure-risk relationship, a small positive radon-related effect appears when looking at the risk of death by leukemia (excluding CLL). This study investigates the existence of radon-related increased risk of death by lung cancer, kidney cancer, brain and CNS cancer and leukemia under a Bayesian framework and assumptions of linear and log-linear exposure-risk relationships. If there is no doubt in the interpretation of the results for lung cancer and brain and CNS cancer, the conclusion is less clear-cut in the case of kidney cancer and leukemia (excluding CLL). A future update of the French cohort, increasing the follow-up time for miners, may help to reach a clearer conclusion for these two cancer sites.

Measurement science meets the reproducibility challenge

Measurement science is particularly well equipped not only to meet reproducibility challenges arising within the field of metrology, but also to suggest strategies and best practices for how such challenges can be met in other fields. This contribution illustrates three such challenges, in three different fields, and proposes ways to address them that can supplement the only way in which reproducibility challenges in science can be resolved definitively: by validated scientific advances that point toward the truth. The first example concerns a large interlaboratory, international comparison of the measurement of the mass fraction of silica in a granite reference material, using classical methods of wet analytical chemistry, carried out in the 1940s. The results delivered a shock to analysts worldwide about the state of the art at the time. The challenge was magnified by the fact that none of the measured values was qualified with an evaluation of measurement uncertainty. We offer an approach developed by Andrew Rukhin from NIST, for how to compute a meaningful consensus value in such case, and explain how the associated uncertainty can be characterized. The second example is about the currently hot topic of the Hubble tension, which refers to the mutual inconsistency of the measurement results, obtained by different methods, for the Hubble-Lemaître constant, which expresses the rate of expansion of the Universe. We suggest that such tension can be quantified in terms of the dark uncertainty that figures as a parameter in a laboratory random effects model, thus providing an objective metric whereby progress toward resolving such tension can be gauged. The third example discusses two sources of lack of reproducibility: on the one hand, the fact that different laboratories produced strikingly discrepant values for the mass fraction of arsenic in kudzu; on the other hand, that different models can be fitted to these data, each producing its own set of results. Here we use a Bayesian model selection criterion to choose one from among four models that are natural candidates to address this double reproducibility challenge. This third example also affords us the opportunity to deflate two widespread myths: that one needs at least four observations to obtain a Bayesian evaluation of standard uncertainty, and that sample standard deviations of small samples are systematically too small.

Zero-augmented beta-prime model for multilevel semi-continuous data: a Bayesian inference

Semi-continuous data characterized by an excessive proportion of zeros and right-skewed continuous positive values appear frequently in medical research. One example would be the pharmaceutical expenditure (PE) data for which a substantial proportion of subjects investigated may report zero. Two-part mixed-effects models have been developed to analyse clustered measures of semi-continuous data from multilevel studies. In this study, we propose a new flexible two-part mixed-effects model with skew distributions for nested semi-continuous cost data under the framework of a Bayesian approach. The proposed model specification consists of two mixed-effects models linked by the correlated random effects: Part I) a model on the occurrence of positive values using a generalized logistic mixed model; and Part II) a model on the magnitude of positive values using a linear mixed model where the model errors follow skew distributions including beta-prime (BP). The proposed method is illustrated with pharmaceutical expenditure data from a multilevel observational study and the analytic results are reported by comparing potential models under different skew distributions. Simulation studies are conducted to assess the performance of the proposed model. The DIC3, LPML, WAIC, and LOO as the Bayesian model selection criteria and measures of divergence used to compare the models.

Bayesian Item Response Theory Models With Flexible Generalized Logit Links.

In educational and psychological research, the logit and probit links are often used to fit the binary item response data. The appropriateness and importance of the choice of links within the item response theory (IRT) framework has not been investigated yet. In this paper, we present a family of IRT models with generalized logit links, which include the traditional logistic and normal ogive models as special cases. This family of models are flexible enough not only to adjust the item characteristic curve tail probability by two shape parameters but also to allow us to fit the same link or different links to different items within the IRT model framework. In addition, the proposed models are implemented in the Stan software to sample from the posterior distributions. Using readily available Stan outputs, the four Bayesian model selection criteria are computed for guiding the choice of the links within the IRT model framework. Extensive simulation studies are conducted to examine the empirical performance of the proposed models and the model fittings in terms of "in-sample" and "out-of-sample" predictions based on the deviance. Finally, a detailed analysis of the real reading assessment data is carried out to illustrate the proposed methodology.

ORDINAL PROBIT FUNCTIONAL OUTCOME REGRESSION WITH APPLICATION TO COMPUTER-USE BEHAVIOR IN RHESUS MONKEYS.

Research in functional regression has made great strides in expanding to non-Gaussian functional outcomes, but exploration of ordinal functional outcomes remains limited. Motivated by a study of computer-use behavior in rhesus macaques (Macaca mulatta), we introduce the Ordinal Probit Functional Outcome Regression model (OPFOR). OPFOR models can be fit using one of several basis functions including penalized B-splines, wavelets, and O'Sullivan splines-the last of which typically performs best. Simulation using a variety of underlying covariance patterns shows that the model performs reasonably well in estimation under multiple basis functions with near nominal coverage for joint credible intervals. Finally, in application, we use Bayesian model selection criteria adapted to functional outcome regression to best characterize the relation between several demographic factors of interest and the monkeys' computer use over the course of a year. In comparison with a standard ordinal longitudinal analysis, OPFOR outperforms a cumulative-link mixed-effects model in simulation and provides additional and more nuanced information on the nature of the monkeys' computer-use behavior.

Estimating Country-Specific Incidence Rates of Rare Cancers: Comparative Performance Analysis of Modeling Approaches Using European Cancer Registry Data.

Estimating incidence of rare cancers is challenging for exceptionally rare entities and in small populations. In a previous study, investigators in the Information Network on Rare Cancers (RARECARENet) provided Bayesian estimates of expected numbers of rare cancers and 95% credible intervals for 27 European countries, using data collected by population-based cancer registries. In that study, slightly different results were found by implementing a Poisson model in integrated nested Laplace approximation/WinBUGS platforms. In this study, we assessed the performance of a Poisson modeling approach for estimating rare cancer incidence rates, oscillating around an overall European average and using small-count data in different scenarios/computational platforms. First, we compared the performance of frequentist, empirical Bayes, and Bayesian approaches for providing 95% confidence/credible intervals for the expected rates in each country. Second, we carried out an empirical study using 190 rare cancers to assess different lower/upper bounds of a uniform prior distribution for the standard deviation of the random effects. For obtaining a reliable measure of variability for country-specific incidence rates, our results suggest the suitability of using 1 as the lower bound for that prior distribution and selecting the random-effects model through an averaged indicator derived from 2 Bayesian model selection criteria: the deviance information criterion and the Watanabe-Akaike information criterion.

Generalized Topp-Leone-Weibull AFT Modelling: A Bayesian Analysis with MCMC Tools using R and Stan

The generalized Topp-Leone-Weibull (GTL-W) distribution is a generalization of Weibull distribution which is obtained by using generalized Topp-Leone (GTL) distribution as a generator and considering Weibull distribution as a baseline distribution. Weibull distribution is a widely used survival model that has monotone- increasing or decreasing hazard. But it cannot accommodate bathtub shaped and unimodal shaped hazards. As a survival model, GTL-W distribution is more flexible than the Weibull distribution to accommodate different types of hazards. The present study aims at fitting GTL-W model as an accelerated failure time (AFT) model to censored survival data under Bayesian setting using R and Stan languages. The GTL-W AFT model is compared with its sub-model and the baseline model. The Bayesian model selection criteria LOOIC and WAIC are applied to select the best model.

Testing gravity with the Milky Way: Yukawa potential

We test a Yukawa correction to the Newtonian potential, making use of our own Galaxy - the Milky Way - as a testbed. We include as free parameter the Yukawa strength and range and the dark matter profile parameters, and compare several morphologies for the bulge, gas, and disk components, also using Bayesian model selection criteria. We employ up-to-date datasets for both the visible (baryonic) component of the Milky Way, and for the tracers of the gravitational potential (the Rotation Curve). We find that the data are consistent with the Newtonian potential, and constrain the Yukawa coupling $\beta$ to be negative and $\lambda$ to range along the curve $\lambda = a|\beta|^{c}$ with $a = (0.77 \pm 0.06)$ kpc and $c = -0.503\substack{+0.016 \\ -0.019}$.

A Comparative Study of Shared Frailty Models for Kidney Infection Data with Generalized Exponential Baseline Distribution

Shared frailty models are often used to model heterogeneity in survival analysis. The most common shared frailty model is a model in which hazard function is a product of random factor (frailty) and baseline hazard function which is common to all individuals. There are certain as sumptions about the baseline distribution and distribution of frailty. Mostly assumption of gamma distribution is considered for frailty distribution. To compare the results with gamma frailty model, we introduce three shared frailty models with generalized exponential as baseline distribution. The other three shared frailty models are inverse Gaussian shared frailty model, compound Poisson shared frailty model and compound negative binomial shared frailty model. We fit these models to a real life bivariate survival data set of McGilchrist and Aisbett (1991) related to kidney infection using Markov Chain Monte Carlo (MCMC) technique. Model comparison is made using Bayesian model selection criteria and a better model is suggested for the data.

Exploring Multiple Strategic Problem Solving Behaviors in Educational Psychology Research by Using Mixture Cognitive Diagnosis Model.

A mixture cognitive diagnosis model (CDM), which is called mixture multiple strategy-Deterministic, Inputs, Noisy “and” Gate (MMS-DINA) model, is proposed to investigate individual differences in the selection of response categories in multiple-strategy items. The MMS-DINA model system is an effective psychometric and statistical approach consisting of multiple strategies for practical skills diagnostic testing, which not only allows for multiple strategies of problem solving, but also allows for different strategies to be associated with different levels of difficulty. A Markov chain Monte Carlo (MCMC) algorithm for parameter estimation is given to estimate model, and four simulation studies are presented to evaluate the performance of the MCMC algorithm. Based on the available MCMC outputs, two Bayesian model selection criteria are computed for guiding the choice of the single strategy DINA model and multiple strategy DINA models. An analysis of fraction subtraction data is provided as an illustration example.

The (in)stability of Bayesian model selection criteria in disease mapping

Several model comparison techniques exist to select the best fitting model from a set of candidate models. This study explores the performance of model comparison tools that are commonly used in Bayesian spatial disease mapping and that are available among several Bayesian software packages: the deviance information criterion (DIC), the Watanabe–Akaike information criterion (WAIC) and the log marginal predictive likelihood (LMPL). We compare R packages CARBayes and NIMBLE, and R interfaces to OpenBUGS (R2OpenBUGS) and Stan (RStan), by fitting Poisson models to disease incidence outcomes with intrinsic conditional autoregressive, convolution conditional autoregressive and log-normal error terms. From three data analyses that differ in the number of areal units and background incidence/prevalence of the outcome of interest, we learn that the estimates of model comparison statistics coming from different software packages can lead to disagreements regarding model preference. Furthermore, we show that the distributional convergence of parameter estimates does not necessarily imply numerical convergence of the model comparison tool. We warn users to be careful when doing model comparison when using different software packages, and to make use of one specific method for the calculation of the model selection criteria.

Fully and empirical Bayes approaches to estimating copula-based models for bivariate mixed outcomes using Hamiltonian Monte Carlo

We provide a fully Bayesian approach to conduct estimation and inference for a copula model to jointly analyze bivariate mixed outcomes. To obtain posterior samples, we use Hamiltonian Monte Carlo, which avoids the random walk behavior of Metropolis and Gibbs sampling algorithms. We also provide an empirical Bayes approach to estimate the copula parameter, which is useful when prior specification on that parameter is difficult. We further propose the use of Bayesian model selection criteria to select the most appropriate copula family. We conduct simulation studies to compare the two approaches and to examine copula selection performance and illustrate the application of the fully Bayesian approach on a burn injury data set.

Model selection for Bayesian linear mixed models with longitudinal data: Sensitivity to the choice of priors

We explore the performance of three popular Bayesian model-selection criteria when vague priors are used for the covariance parameters of the random effects in a linear mixed-effects model (LMM) using an extensive simulation study. In a previous paper, we have shown that the conditional selection criteria perform worse than their marginal counterparts. It is known that for some “vague” priors, their impact on the estimated model parameters can be non-negligible, e.g., for the priors of the covariance matrix of the random effects in a longitudinal LMM. We evaluate here the impact of vague priors for the covariance matrix of the random effects on selecting the correct LMM using classical Bayesian selection criteria. We consider marginal and conditional criteria. For the random intercept case, we assign different vague priors to the variance parameters. With two or more random effects, we considered five different specifications of inverse-Wishart (IW) prior, five different separation priors and a joint prior. The results show again the better performance of the marginal over the conditional criteria and the superiority of joint and separation priors over IW in all settings. We also illustrate the performance of the selection criteria on a practical dataset.

D-Measure: A Bayesian Model Selection Criterion for Survival Data

An authentic way for assessing the goodness of a model is to estimate its predictive capability. In this paper, we propose the D-measure, which measures the goodness of a model by comparing how close its predictions are from the observed data based on the survival function. The proposed D-measure can be used for all kinds of survival data in the presence of censoring. It can also be used to compare cure rate models, even in the presence of random effects or frailties. The advantages of the D-measure are verified via simulation, in which it is compared to the deviance information criterion, which is a widely used Bayesian model comparison criterion. The D-measure is illustrated in two real data sets.

Incorporating Realized Quarticity into a Realized Stochastic Volatility Model

This study proposes an extension of the realized stochastic volatility model by incorporating realized quarticity RQ into the volatility process. We employ an efficient Riemann Manifold Hamiltonian Monte Carlo method in a Markov Chain Monte Carlo algorithm to estimate parameters that could not be sampled directly. We investigate the empirical performance of the proposed model using data for six equity indices and 24 individual stocks listed on the Tokyo Stock Exchange. Parameter estimates and two Bayesian model selection criteria reveal evidence supporting RQ-based models that are driven by the maximum value of RQ data. That model consistently outperforms benchmark realized stochastic volatility models in capturing spikes in volatility caused by large RQ values. Including such stylised facts as the asymmetric effect of returns-volatility and heavy tailed returns, our results reveal that the proposed models exhibit a weaker correlation between stock returns and volatility and heavier tails in equity returns.

Threshold Stochastic Volatility Models with Heavy Tails: A Bayesian Approach

This paper extends the threshold stochastic volatility (THSV) model specification proposed in So et al. (2002) and Chen et al. (2008) by incorporating thick-tails in the mean equation innovation using the scale mixture of normal distributions (SMN). A Bayesian Markov Chain Monte Carlo algorithm is developed to estimate all the parameters and latent variables. Value-at-Risk (VaR) and Expected Shortfall (ES) forecasting via a computational Bayesian framework are considered. The MCMC-based method exploits a mixture representation of the SMN distributions. The proposed methodology is applied to daily returns of indexes from BM&F BOVESPA (BOVESPA), Buenos Aires Stock Exchange (MERVAL), Mexican Stock Exchange (MXX) and the Standar & Poors 500 (SP500). Bayesian model selection criteria reveals that there is a significant improvement in model fit for the returns of the data considered here, by using the THSV model with slash distribution over the usual normal and Student-t models. Empirical results show that the skewness can improve VaR and ES forecasting in comparison with the normal and Student-t models.

Shared frailty models with baseline generalized Pareto distribution

In this article, we have considered three different shared frailty models under the assumption of generalized Pareto Distribution as baseline distribution. Frailty models have been used in the survival analysis to account for the unobserved heterogeneity in an individual risks to disease and death. These three frailty models are with gamma frailty, inverse Gaussian frailty and positive stable frailty. Then we introduce the Bayesian estimation procedure using Markov chain Monte Carlo (MCMC) technique to estimate the parameters. We applied these three models to a kidney infection data and find the best fitted model for kidney infection data. We present a simulation study to compare true value of the parameters with the estimated values. Model comparison is made using Bayesian model selection criterion and a well-fitted model is suggested for the kidney infection data.

Comparison of Bayesian Model Selection Criteria in Fay-Herriot Model

베이지안 모형의 적합도를 비교하기 위한 여러 가지 베이지안 모형 선택기준이 제시되고 있다. 이러한 모형 선택기준들이 동일한 모형을 최적의 모형으로 선택하기도 하지만 때로는 서로 다른 모형을 최적 모형으로 선택하기도 한다. 이에 본 연구에서는 모의실험을 통해 기능적 및 구조적 측정오차 하에서 준모수적 Fay-Herriot 모형의 가장 적절한 베이지안 모형 선택기준을 확인하고자 한다. 본 연구에서 고려하고자 하는 베이지안 모형 선택기준에는 LCPO(logarithmic conditional predictive ordinate), DIC(deviance information criterion), PEPD(posterior expected predictive deviance)이며, 모의실험에서 모수 추정 및 모형 적합을 위해서 계층적 구조를 기반으로 하는 MCMC(Markov chain Monte Carlo) 방법을 사용하고자 한다. 모의실험 연구에서 최적의 모형 선정은 RMSE(root mean square error)를 이용하였으며 모의실험 결과 PEPD가 최적의 모형을 선택하는 비율이 가장 높게 나타나지만 직접추정치의 분산이 커질수록 최적의 모형을 선택하는 비율이 낮아지는 경향을 보였으며, LCPO와 DIC는 직접추정치의 분산에 영향을 크게 받지 않는 것으로 나타났다.There are various model selection criteria for Bayesian model. Sometimes model selection criteria may determine the different models as the best model. In this study we confirm the best Bayesian model selection criteria in semi-parametric Fay-Herriot model under measurement error covariate based on the simulation studies. We consider the best true model based on the root mean square errors and compare the correct proportions among the mean logarithmic conditional predictive ordinate (LCPO), the deviance information criterion (DIC) and the posterior expected predictive deviance (PEPD) for simulation data. For fitting the model and estimating parameters, we consider hierarchical Bayesian approach based on the Markov chain Monte Carlo (MCMC) method. As a result PEPD has the highest correct proportion of selecting the best true model. But the correct proportion tends to decrease when the variance increases, whereas the DIC and LCPO were not significantly affected by variance.

Non-Markovian Regime-Switching Models

This paper revisits the non-Markovian regime switching model considered by Chib and Dueker (2004), who employ an autoregressive continuous latent variable in order to specify the dynamics of the latent regime-indicator variable. We show that, in spite of the non-Markovian nature of the regime indicator variable, the Markovian property of this continuous latent variable allows us to easily estimate the model within the Bayesian framework without any approximations. In particular, we show that the conventional Gibbs sampling is enough in generating the regime indicator variable as well as the continuous latent variable conditional on all the parameters of the model and data. For an application to business cycle modeling of postwar US real GDP, a modified version of Hamilton’s (1989) Markovian switching model is slightly preferred to a non-Markovian switching model by the Bayesian model selection criterion. For an application to volatility modeling of the weekly stock return, a non-Markovian switching model with endogenous switching or the leverage effect is strongly preferred to Markovian switching models.