Posterior Bayes Factor Research Articles

Empirical Bayes factors for common hypothesis tests.

Bayes factors for composite hypotheses have difficulty in encoding vague prior knowledge, as improper priors cannot be used and objective priors may be subjectively unreasonable. To address these issues I revisit the posterior Bayes factor, in which the posterior distribution from the data at hand is re-used in the Bayes factor for the same data. I argue that this is biased when calibrated against proper Bayes factors, but propose adjustments to allow interpretation on the same scale. In the important case of a regular normal model, the bias in log scale is half the number of parameters. The resulting empirical Bayes factor is closely related to the widely applicable information criterion. I develop test-based empirical Bayes factors for several standard tests and propose an extension to multiple testing closely related to the optimal discovery procedure. When only a P-value is available, an approximate empirical Bayes factor is 10p. I propose interpreting the strength of Bayes factors on a logarithmic scale with base 3.73, reflecting the sharpest distinction between weaker and stronger belief. This provides an objective framework for interpreting statistical evidence, and realises a Bayesian/frequentist compromise.

A Bayesian Partially Observable Online Change Detection Approach with Thompson Sampling

This article proposes a Bayesian learning framework for online change detection of high-dimensional data streams where only a subset of variables can be observed at each time point due to limited sensing capacities. On the one hand, we need to build a change detection scheme based on partial observations. On the other, the scheme should be able to adaptively and actively select the most critical sensing variables to observe to maximize the detection power. To address these two points, in this article, first, a novel Bayesian Spike-Slab Composite Decomposition (BSSCD) is proposed to decompose the high-dimensional signals onto normal and abnormal bases, where the projection coefficients are efficiently estimated via variational Bayesian inference. Built upon it, the posterior Bayes factor is constructed as the detection statistic. Second, by further formulating the detection statistic as the reward function of combinatorial multi-armed bandit (CMAB), a Thompson sampling strategy is proposed for selecting the potential changed variables with the balance of exploration and exploitation. The efficacy and applicability of our method are demonstrated in practice with numerical studies and a real case study.

A Two-Sample Test of High Dimensional Means Based on Posterior Bayes Factor

In classical statistics, the primary test statistic is the likelihood ratio. However, for high dimensional data, the likelihood ratio test is no longer effective and sometimes does not work altogether. By replacing the maximum likelihood with the integral of the likelihood, the Bayes factor is obtained. The posterior Bayes factor is the ratio of the integrals of the likelihood function with respect to the posterior. In this paper, we investigate the performance of the posterior Bayes factor in high dimensional hypothesis testing through the problem of testing the equality of two multivariate normal mean vectors. The asymptotic normality of the linear function of the logarithm of the posterior Bayes factor is established. Then we construct a test with an asymptotically nominal significance level. The asymptotic power of the test is also derived. Simulation results and an application example are presented, which show good performance of the test. Hence, taking the posterior Bayes factor as a statistic in high dimensional hypothesis testing is a reasonable methodology.

Testing high dimensional covariance matrices via posterior Bayes factor

With the advent of the era of big data, high dimensional covariance matrices are increasingly encountered and testing covariance structure has become an active area in contemporary statistical inference. Conventional testing methods fail when addressing high dimensional data due to the singularity of the sample covariance matrices. In this paper, we propose a novel test for the prominent identity test and sphericity test based on posterior Bayes factor. For general population model with finite fourth order moment, the limiting null distribution of the test statistic is obtained. Furthermore, we derive the asymptotic power function when the sample size and dimension are proportional against spiked alternatives. When the dimension is much larger than the sample size, under general alternatives, the limiting alternative distribution together with the consistency of the new test is also obtained. Monte Carlo simulation results show that the limiting approximation is quite accurate under the null for finite sample, and the proposed test outperforms some well-known tests in the literature in terms of Type I error rate and the empirical power.

Bayesian Analysis of ODEs: Solver Optimal Accuracy and Bayes Factors

In most cases in the Bayesian analysis of ODE inverse problems, a numerical solver needs to be used. Therefore, we cannot work with the exact theoretical posterior distribution but only with an approximate posterior derived from the error in the numerical solver. To compare an approximate posterior distribution with the theoretical one, we propose using Bayes factors (BFs), considering both of them as models for the data at hand. From a theoretical point of view, we prove that the theoretical vs. numerical posterior BF tends to 1, in the same order as the numerical solver used. In practice, we illustrate the fact that for higher order solvers (e.g., Runge--Kutta) the BF is already nearly 1 for step sizes that would take far less computational effort. Considerable CPU time may be saved by using coarser solvers that nevertheless produce practically error-free posteriors. Two examples are presented where nearly 90% CPU time is saved, with all inference results being identical to those obtained using a solver with a much finer time step.

Semiparametric Bayesian hierarchical models for heterogeneous population in nonlinear mixed effect model: application to gastric emptying studies

Gastric emptying studies are frequently used in medical research, both human and animal, when evaluating the effectiveness and determining the unintended side-effects of new and existing medications, diets, and procedures or interventions. It is essential that gastric emptying data be appropriately summarized before making comparisons between study groups of interest and to allow study the comparisons. Since gastric emptying data have a nonlinear emptying curve and are longitudinal data, nonlinear mixed effect (NLME) models can accommodate both the variation among measurements within individuals and the individual-to-individual variation. However, the NLME model requires strong assumptions that are often not satisfied in real applications that involve a relatively small number of subjects, have heterogeneous measurement errors, or have large variation among subjects. Therefore, we propose three semiparametric Bayesian NLMEs constructed with Dirichlet process priors, which automatically cluster sub-populations and estimate heterogeneous measurement errors. To compare three semiparametric models with the parametric model we propose a penalized posterior Bayes factor. We compare the performance of our semiparametric hierarchical Bayesian approaches with that of the parametric Bayesian hierarchical approach. Simulation results suggest that our semiparametric approaches are more robust and flexible. Our gastric emptying studies from equine medicine are used to demonstrate the advantage of our approaches.

Alternative Procedures to Discriminate Non Nested Multivariate Linear Regression Models

ABSTRACT This article builds classical and Bayesian testing procedures for choosing between non nested multivariate regression models. Although there are several classical tests for discriminating univariate regressions, only the Cox test is able to consistently handle the multivariate case. We then derive the limiting distribution of the Cox statistic in such a context, correcting an earlier derivation in the literature. Further, we show how to build alternative Bayes factors for the testing of nonnested multivariate linear regression models. In particular, we compute expressions for the posterior Bayes factor, the fractional Bayes factor, and the intrinsic Bayes factor.

Confronting different models of community structure to species‐abundance data: a Bayesian model comparison

Species abundances are undoubtedly the most widely available macroecological data, but can we use them to distinguish among several models of community structure? Here we present a Bayesian analysis of species-abundance data that yields a full joint probability distribution of each model's parameters plus a relatively parameter-independent criterion, the posterior Bayes factor, to compare these models. We illustrate our approach by comparing three classical distributions: the zero-sum multinomial (ZSM) distribution, based on Hubbell's neutral model, the multivariate Poisson lognormal distribution (MPLN), based on niche arguments, and the discrete broken stick (DBS) distribution, based on MacArthur's broken stick model. We give explicit formulas for the probability of observing a particular species-abundance data set in each model, and argue that conditioning on both sample size and species count is needed to allow comparisons between the two distributions. We apply our approach to two neotropical communities (trees, fish). We find that DBS is largely inferior to ZSM and MPLN for both communities. The tree data do not allow discrimination between ZSM and MPLN, but for the fish data ZSM (neutral model) overwhelmingly outperforms MPLN (niche model), suggesting that dispersal plays a previously underestimated role in structuring tropical freshwater fish communities. We advocate this approach for identifying the relative importance of dispersal and niche-partitioning in determining diversity of different ecological groups of species under different environmental conditions.

Bayesian model selection of contemporary groups for BLUP genetic evaluation in Latxa dairy sheep

Milk yield data from the Latxa dairy sheep breed were used for model comparison. Three possible models for genetic evaluation, differing in the definition of the contemporary groups, were compared using a Bayesian approach. Contemporary groups were: flock–year+flock–month of lambing–number of parity and age, flock–year–month, and flock–year in models 1, 2, and 3, respectively. All available data from the two strains of the breed (Black-Faced and Blond-Faced) were used. The comparison criteria were: Bayes factor, posterior Bayes factor, pseudo-Bayes factor, two checking functions based on univariate predictive distributions, and the deviance information criteria. The two checking functions consisted in the measure of accuracy ( d 1) and systematic bias ( d 2) in the prediction of observations. All criteria except d 2 indicated the superiority of model 2 in both strains. Based on these results, it was concluded that model 2 reaches the best equilibrium between fit and predictive ability.

On the calibration of Bayesian model choice criteria

Model choice is a fundamental problem in data analysis. With interest in hierarchical models which typically arise as Bayesian specifications, we confine ourselves to Bayesian model choice criteria. If Y denotes the observed data and T( Y) is the criterion, our goal is to calibrate T( Y) in order to assess how large or small it is under a given model. In particular, if we have the distribution of T( Y) , then we can compute any probabilities or determine any quantities of interest. Apart from very special cases, analytic development of such distributions is intractable. Standard analytic approximations may be inapplicable if usual random effects are introduced at the various modeling levels. Indeed, calculation of T( Y) itself is often difficult enough. We suggest a generic simulation-intensive approach for obtaining the distribution of T( Y) to arbitrary accuracy. We focus on various Bayes factors, e.g., the usual Bayes factor, the posterior Bayes factor and the pseudo-Bayes factor. We illustrate with a binomial regression example.

Effects of models of rate evolution on estimation of divergence dates with special reference to the metazoan 18S ribosomal RNA phylogeny.

The molecular clock, i.e., constancy of the rate of evolution over time, is commonly assumed in estimating divergence dates. However, this assumption is often violated and has drastic effects on date estimation. Recently, a number of attempts have been made to relax the clock assumption. One approach is to use maximum likelihood, which assigns rates to branches and allows the estimation of both rates and times. An alternative is the Bayes approach, which models the change of the rate over time. A number of models of rate change have been proposed. We have extended and evaluated models of rate evolution, i.e., the lognormal and its recent variant, along with the gamma, the exponential, and the Ornstein-Uhlenbeck processes. These models were first applied to a small hominoid data set, where an empirical Bayes approach was used to estimate the hyperparameters that measure the amount of rate variation. Estimation of divergence times was sensitive to these hyperparameters, especially when the assumed model is close to the clock assumption. The rate and date estimates varied little from model to model, although the posterior Bayes factor indicated the Ornstein-Uhlenbeck process outperformed the other models. To demonstrate the importance of allowing for rate change across lineages, this general approach was used to analyze a larger data set consisting of the 18S ribosomal RNA gene of 39 metazoan species. We obtained date estimates consistent with paleontological records, the deepest split within the group being about 560 million years ago. Estimates of the rates were in accordance with the Cambrian explosion hypothesis and suggested some more recent lineage-specific bursts of evolution.

Bayesian estimation of cognitive decline in patients with alzheimer's disease

AbstractRecently, there has been great interest in estimating the decline in cognitive ability in patients with Alzheimer's disease. Measuring decline is not straightforward, since one must consider the choice of scale to measure cognitive ability, possible floor and ceiling effects, between‐patient variability, and the unobserved age of onset. The authors demonstrate how to account for the above features by modeling decline in scores on the Mini‐Mental State Exam in two different data sets. To this end, they use hierarchical Bayesian models with change points, for which posterior distributions are calculated using the Gibbs sampler. They make comparisons between several such models using both prior and posterior Bayes factors, and compare the results from the models suggested by these two model selection criteria.

Simpson’s Paradox and the Bayes Factor

Summary A counter-example presented by Lindley to Aitkin’s posterior Bayes factor is examined. The paradoxical feature of the counter-example is found to be a simple case of Simpson’s paradox.

Alternative Bayes factors for model selection

AbstractSeveral alternative Bayes factors have been recently proposed in order to solve the problem of the extreme sensitivity of the Bayes factor to the priors of models under comparison. Specifically, the impossibility of using the Bayes factor with standard noninformative priors for model comparison has led to the introduction of new automatic criteria, such as the posterior Bayes factor (Aitkin 1991), the intrinsic Bayes factors (Berger and Pericchi 1996b) and the fractional Bayes factor (O'Hagan 1995). We derive some interesting properties of the fractional Bayes factor that provide justifications for its use additional to the ones given by O'Hagan. We further argue that the use of the fractional Bayes factor, originally introduced to cope with improper priors, is also useful in a robust analysis. Finally, using usual classes of priors, we compare several alternative Bayes factors for the problem of testing the point null hypothesis in the univariate normal model.

Bayes inference for S-shaped software-reliability growth models

Bayes inference for a nonhomogeneous Poisson process with an S-shaped mean value function is studied. In particular, the authors consider the model of Ohba et al. (1983), and its generalization to a class of gamma distribution growth curves. Two Gibbs sampling approaches are proposed to compute the Bayes estimates of the mean number of errors remaining and the current system reliability. One algorithm is a Metropolis within Gibbs algorithm, The other is a stochastic substitution algorithm with data augmentation. Model selection based on the posterior Bayes factor is studied. A numerical example with simulated data is given.

The calibration of P-values, posterior Bayes factors and the AIC from the posterior distribution of the likelihood

The posterior distribution of the likelihood is used to interpret the evidential meaning of P-values, posterior Bayes factors and Akaike‘s information criterion when comparing point null hypotheses with composite alternatives. Asymptotic arguments lead to simple re-calibrations of these criteria in terms of posterior tail probabilities of the likelihood ratio. (’Prior‘) Bayes factors cannot be calibrated in this way as they are model-specific.

A new test for the presence of a normal mixture distribution based on the posterior Bayes factor

We present a new test for the presence of a normal mixture distribution, based on the posterior Bayes factor of Aitkin (1991). The new test has slightly lower power than the likelihood ratio test. It does not require the computation of the MLEs of the parameters or a search for multiple maxima, but requires computations based on ‘classification likelihood’ assignments of observations to mixture components.

Probability model choice in single samples from exponential families using Poisson log-linear modelling, and model comparison using Bayes and posterior Bayes factors

This paper describes a method due to Lindsey (1974a) for fitting different exponential family distributions for a single population to the same data, using Poisson log-linear modelling of the density or mass function. The method is extended to Efron's (1986) double exponential family, giving exact ML estimation of the two parameters not easily achievable directly. The problem of comparing the fit of the non-nested models is addressed by both Bayes and posterior Bayes factors (Aitkin, 1991). The latter allow direct comparisons of deviances from the fitted distributions.

Posterior Bayes factor analysis for an exponential regression model

In the exponential regression model, Bayesian inference concerning the non-linear regression parameter σ has proved extremely difficult. In particular, standard improper diffuse priors for the usual parameters lead to an improper posterior for the non-linear regression parameter. In a recent paper Ye and Berger (1991) applied the reference prior approach of Bernardo (1979) and Berger and Bernardo (1989) yielding a proper informative prior for σ. This prior depends on the values of the explanatory variable, goes to 0 as σ goes to 1, and depends on the specification of a hierarchical ordering of importance of the parameters.

Comparing several accelerated life models

This paper presents two approaches for comparing several exponential accelerated life models under the usual stress levels. The approaches are based on on the likelihood ratio statistics and on the posterior Bayes factor ( Aitkin, 1991). These procedures can be useful in many practical situations. An exact distribution and a table of critical values of the likelihood ratio statistics are presented. A simulation study is also performed.