Bayesian Model Evaluation Research Articles

Evaluation of Bayesian models for focused clustering in health data

AbstractThis paper examines the ability of Bayesian hierarchical models to recover evidence of disease risk excess around a fixed location. This location can be a putative source of health hazard, such as an incinerator, mobile phone mast or dump site. While Bayesian models are convenient to use for modeling, it is useful to consider how well these models perform in the true risk scenarios. In what follows, we evaluate the ability of these models to recover the true risk under simulation. It is surprising that the resulting posterior parameters estimates are heavily biased. Using the credible intervals for distance decline parameter to assess ‘coverage or power’ of detecting distance effect, the ‘power’ decreases with increasing correlation in the background population effect. The inclusion of correlated heterogeneity in models does affect the ability of the models to detect the stronger distance decline scenarios. The uncorrelated heterogeneity seems little affect this ability however. Copyright © 2007 John Wiley & Sons, Ltd.

Information criteria for the predictive evaluation of bayesian models

As a natural successor of the information criteria AIC and ABIC, information criteria for the Bayes models were developed by evaluating the bias of the log likelihood of the predictive distribution as an estimate of its expected log-likelihood. Considering two specific situations for the true distribution, two information criteria, PIC1 and PIC2 are derived. Linear Gaussian cases are considered in details and the evaluation of the maximum a posteriori estimator is also considered. By a simple example of estimating the signal to noise ratio, it was shown that the PIC2 is a good approximation to the expected log-likelihood in the entire region of the signal to noise ratio. On the other hand, PIC1 performs good only for the smaller values of the variance ratio. For illustration, the problems of trend estimation and seasonal adjustment are considered. Examples show that the hyper-parameters estimated by the new criteria are usually closer to the best ones than those by the ABIC.