Proper Posterior Distributions Research Articles

On Posterior Properties of the Two Parameter Gamma Family of Distributions.

The gamma distribution has been extensively used in many areas of applications. In this paper, considering a Bayesian analysis we provide necessary and sufficient conditions to check whether or not improper priors lead to proper posterior distributions. Further, we also discuss sufficient conditions to verify if the obtained posterior moments are finite. An interesting aspect of our findings are that one can check if the posterior is proper or improper and also if its posterior moments are finite by looking directly in the behavior of the proposed improper prior. To illustrate our proposed methodology these results are applied in different objective priors.

Bayesian inference using Hamiltonian Monte-Carlo algorithm for nonlinear joint modeling in the context of cancer immunotherapy.

Treatment evaluation in advanced cancer mainly relies on overall survival and tumor size dynamics. Both markers and their association can be simultaneously analyzed by using joint models, and these approaches are supported by many softwares or packages. However, these approaches are essentially limited to linear models for the longitudinal part, which limit their biological interpretation. More biological models of tumor dynamics can be obtained by using nonlinear models, but they are limited by the fact that parameter identifiability require rich dataset. In that context Bayesian approaches are particularly suited to incorporate the biological knowledge and increase the information available, but they are limited by the high computing cost of Monte-Carlo by Markov Chains algorithms. Here, we aimed to assess the performances of the Hamiltonian Monte-Carlo (HMC) algorithm implemented in Stan for inference in a nonlinear joint model. The method was validated on simulated data where HMC provided proper posterior distributions and credibility intervals in a reasonable computational time. Then the association between tumor size dynamics and survival was assessed in patients with advanced or metastatic bladder cancer treated with atezolizumab, an immunotherapy agent. HMC confirmed limited sensitivity to prior distributions. A cross-validation approach was developed and identified the current slope of tumor size dynamics as the most relevant driver of survival. In summary, HMC is an efficient approach to perform nonlinear joint models in a Bayesian framework, and opens the way for the use of nonlinear models to characterize both the rapid dynamics and the intersubject variability observed during cancer immunotherapy treatment.

Objective Bayesian Analysis of a Cokriging Model for Hierarchical Multifidelity Codes

Autoregressive cokriging models have been widely used to emulate multiple computer models with different levels of fidelity. The dependence structures are modeled via Gaussian processes at each level of fidelity, where covariance structures are often parameterized up to a few parameters. The predictive distributions typically require intensive Monte Carlo approximations in previous works. This article derives new closed-form formulas to compute the means and variances of predictive distributions in autoregressive cokriging models that only depend on correlation parameters. For parameter estimation, we consider objective Bayesian analysis of such autoregressive cokriging models. We show that common choices of prior distributions, such as the constant prior and inverse correlation prior, typically lead to improper posteriors. We also develop several objective priors such as the independent reference prior and the independent Jeffreys prior that are shown to yield proper posterior distributions. This development is illustrated with a borehole function in an eight-dimensional input space and applied to an engineering application in a six-dimensional input space.

Cauchy and other shrinkage priors for logistic regression in the presence of separation

AbstractIn recent years, the choice of prior distributions for Bayesian logistic regression has received considerable interest. It is widely acknowledged that noninformative, improper priors have to be used with caution because posterior propriety may not always hold. As an alternative, heavy‐tailed priors such as Cauchy prior distributions have been proposed by Gelman et al. (2008). The motivation for using Cauchy prior distributions is that they are proper, and thus, unlike noninformative priors, they are guaranteed to yield proper posterior distributions. The heavy tails of the Cauchy distribution allow the posterior distribution to adapt to the data. Thus Gelman et al. (2008) suggested the use of these prior distributions as a default weakly informative choice, in the absence of prior knowledge about the regression coefficients. While these prior distributions are guaranteed to have proper posterior distributions, Ghosh, Li, and Mitra (2018), showed that the posterior means may not exist, or can be unreasonably large, when they exist, for datasets with separation. In this paper, we provide a short review of the concept of separation, and we discuss how common prior distributions like the Cauchy and normal prior perform in data with separation. Theoretical and empirical results suggest that lighter tailed prior distributions such as normal prior distributions can be a good default choice when there is separation.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical Models

Reference Bayesian analysis of inverse Gaussian degradation process

• A new class of reference priors for IG process degradation model is proposed. • The modified reference priors are proved to have proper posterior distributions and probability matching properties . • Gibbs sampling algorithms for modified reference priors are proposed. • Simulation and real data studies show that the proposed method behaves better than the MLE and subjective Bayesian method. In this paper, objective Bayesian method is applied to analyze degradation model based on the inverse Gaussian process. Noninformative priors (Jefferys prior and two reference priors) for model parameters are obtained and their properties are discussed. Moreover, we propose a class of modified reference priors to remedy weaknesses of the usual reference priors and show that the modified reference priors not only have proper posterior distributions but also have probability matching properties for model parameters. Gibbs sampling algorithms for Bayesian inference based on the Jefferys prior and the modified reference priors are studied. Simulations are conducted to compare the objective Bayesian estimates with the maximum likelihood estimates and subjective Bayesian estimates and shows better performance of the objective method than the other two estimates especially for the case of small sample size. Finally, two real data examples are analyzed for illustration.

Objective Bayesian analysis for competing risks model with Wiener degradation phenomena and catastrophic failures

• Noninformative priors (Jefferys prior and two reference priors) for competing degradation failure model are proposed. • The noninformative priors are shown to have proper posterior distributions and probability matching properties. • Gibbs sampling algorithms for the Jeffreys prior and reference priors are proposed. • The advantages of our model have been illustrated by simulation and real data examples. In this paper, the objective Bayesian method is applied to investigate the competing risks model involving both catastrophic and degradation failures. By modeling soft failure as the Wiener degradation process , and hard failures as a Weibull distribution , we obtain the noninformative priors (Jefferys prior and two reference priors) for the parameters. Moreover, we show that their posterior distributions have good properties and we propose Gibbs sampling algorithms for the Bayesian inference based on the Jefferys prior and two reference priors. Some simulation studies are conducted to illustrate the superiority of objective Bayesian method. Finally, we apply our methods to two real data examples and compare the objective Bayesian estimates with the other estimates.

Parameter Analysis and Estimates for the MODIS Evapotranspiration Algorithm and Multiscale Verification

AbstractAccurate estimation of terrestrial evapotranspiration (E) is critical to understand the world's energy and water cycles. MOD16 is the core algorithm of the widely used global E data set (the Moderate Resolution Imaging Spectroradiometer [MODIS] E product). However, it exhibits considerable uncertainties in some regions. Based on the data from 175 flux towers, we identified the key parameters of the MOD16 algorithm using the Sobol’ sensitivity analysis method across biomes. The output of the MOD16 algorithm was sensitive to eight parameters. Among them, β, which is treated as a constant (0.2 kPa) across biomes in the original MOD16 algorithm, was identified as the parameter to which the algorithm was most sensitive. We used the differential‐evolution Markov chain method to obtain the proper posterior distributions for each key parameter across a range of biomes. The values of the key parameters for the different biomes were accurately estimated by differential‐evolution Markov chain in comparison with data from the flux towers. We then evaluated the performances of the original MOD16 and the optimized MOD16 and compared them at multiple spatial scales (i.e., site, catchment, and global). We obtained relatively consistent and more reliable E simulations using the optimized MOD16 at all three scales. In the future, more attention should be paid to uncertainties in the algorithm's structure and its parameterizations of soil moisture constraint, canopy resistance, and energy partitioning.

Objective Bayesian hypothesis testing and estimation for the intraclass model

ABSTRACTThe intraclass correlation coefficient (ICC) plays an important role in various fields of study as a coefficient of reliability. In this paper, we consider objective Bayesian analysis for the ICC in the context of normal linear regression model. We first derive two objective priors for the unknown parameters and show that both result in proper posterior distributions. Within a Bayesian decision-theoretic framework, we then propose an objective Bayesian solution to the problems of hypothesis testing and point estimation of the ICC based on a combined use of the intrinsic discrepancy loss function and objective priors. The proposed solution has an appealing invariance property under one-to-one reparametrisation of the quantity of interest. Simulation studies are conducted to investigate the performance the proposed solution. Finally, a real data application is provided for illustrative purposes.

Bayesian estimation of the generalized lognormal distribution using objective priors

ABSTRACTThe generalized lognormal distribution plays an important role in analysing data from different life testing experiments. In this paper, we consider Bayesian analysis of this distribution using various objective priors for the model parameters. Specifically, we derive expressions for the Jeffreys-type priors, the reference priors with different group orderings of the parameters, and the first-order matching priors. We also study the properties of the posterior distributions of the parameters under these improper priors. It is shown that only two of them result in proper posterior distributions. Numerical simulation studies are conducted to compare the performances of the Bayesian estimators under the considered priors and the maximum likelihood estimates. Finally, a real-data application is also provided for illustrative purposes.

Posterior propriety in Bayesian extreme value analyses using reference priors

The Generalized Pareto (GP) and Generalized extreme value (GEV) distributions play an important role in extreme value analyses, as models for threshold excesses and block maxima respectively. For each of these distributions we consider Bayesian inference using "reference" prior distributions (in the general sense of priors constructed using formal rules) for the model parameters, specifically a Jeffreys prior, the maximal data information (MDI) prior and independent uniform priors on separate model parameters. We investigate the important issue of whether these improper priors lead to proper posterior distributions. We show that, in the GP and GEV cases, the MDI prior, unless modified, never yields a proper posterior and that in the GEV case this also applies to the Jeffreys prior. We also show that a sample size of three (four) is sufficient for independent uniform priors to yield a proper posterior distribution in the GP (GEV) case.

Bayesian analysis of two-piece location–scale models under reference priors with partial information

Bayesian estimators are developed and compared with the maximum likelihood estimators for the two-piece location–scale models, which contain several well-known distributions such as the asymmetric Laplace distribution, the two-piece normal distribution, and the two-piece Student-t distribution. For the validity of Bayesian analysis, it is essential to use priors that could lead to proper posterior distributions. Specifically, reference priors with partial information have been considered. A sufficient and necessary condition is established to guarantee the propriety of the posterior distribution under a general class of priors. The performance of the proposed approach is illustrated through extensive simulation studies and real data analysis.

Bayesian reference analysis for exponential power regression models

We develop Bayesian reference analyses for linear regression models when the errors follow an exponential power distribution. Specifically, we obtain explicit expressions for reference priors for all the six possible orderings of the model parameters and show that, associated with these six parameters orderings, there are only two reference priors. Further, we show that both of these reference priors lead to proper posterior distributions. Furthermore, we show that the proposed reference Bayesian analyses compare favorably to an analysis based on a competing noninformative prior. Finally, we illustrate these Bayesian reference analyses for exponential power regression models with applications to two datasets. The first application analyzes per capita spending in public schools in the United States. The second application studies the relationship between sold home videos versus profits at the box office.MSC62F15; 62F35; 62J05

Objective Bayesian analysis of SAR models

The simultaneously autoregressive model (abbreviated as SAR) has been extensively applied for lattice (regional summary) data. A Bayesian approach has been studied by De Oliveira and Song (2008), but they only considered two versions of Jeffreys priors, Jeffreys-rule and independence Jeffreys priors. They recommended the independence Jeffreys prior for a default prior. This prior is known to have the potential problem of posterior impropriety. In this paper, we consider the reference priors including the commonly used reference and “exact” reference priors for the SAR model. We show that common reference priors typically result in improper posterior distributions. Next, two “exact” reference priors are developed and are shown to yield proper posterior distributions. Frequentist properties of inferences based on two “exact” reference and Jeffreys-rule priors are studied by means of simulation. For illustrative purposes, we apply the method to SAT verbal scores across the US.

The use of Jeffreys priors for the Student-t distribution

The Jeffreys-rule prior and the marginal independence Jeffreys prior are recently proposed in Fonseca et al. [Objective Bayesian analysis for the Student-t regression model, Biometrika 95 (2008), pp. 325–333] as objective priors for the Student-t regression model. The authors showed that the priors provide proper posterior distributions and perform favourably in parameter estimation. Motivated by a practical financial risk management application, we compare the performance of the two Jeffreys priors with other priors proposed in the literature in a problem of estimating high quantiles for the Student-t model with unknown degrees of freedom. Through an asymptotic analysis and a simulation study, we show that both Jeffreys priors perform better in using a specific quantile of the Bayesian predictive distribution to approximate the true quantile.

Improper Priors Are Not Improper

It is well known that improper priors in Bayesian statistics may lead to proper posterior distributions and useful inference procedures. This motivates us to give an elementary introduction to a theoretical frame for statistics that includes improper priors. Axioms that allow improper priors are given by a relaxed version of Kolmogorov’s formulation of probability theory. The theory of conditional probability spaces formulated by Renyi is closely related, but the initial axioms and the motivation differ. One consequence of the axioms is a general Bayes theorem which gives proper posterior distributions, and furthermore, the theory also gives a convenient frame for formulation of non-Bayesian statistical models. The results are in particular relevant for the current usage of improper priors in Markov chain Monte Carlo methods, and for methods for simulation from conditional distributions given sufficient statistics. This theory gives an alternative to ad hoc arguments without an underlying theory, and removes apparent paradoxes. Readers who acknowledge the need for a theoretical basis for statistical inference including improper priors are urged to consider the theory of conditional probability spaces as presented here.

Evaluation of formal posterior distributions via Markov chain arguments

We consider evaluation of proper posterior distributions obtained from improper prior distributions. Our context is estimating a bounded function Φ of a parameter when the loss is quadratic. If the posterior mean of 0 is admissible for all bounded Φ, the posterior is strongly admissible. We give sufficient conditions for strong admissibility. These conditions involve the recurrence of a Markov chain associated with the estimation problem. We develop general sufficient conditions for recurrence of general state space Markov chains that are also of independent interest. Our main example concerns the p-dimensional multivariate normal distribution with mean vector 0 when the prior distribution has the form g(∥6∥ 2 ) dθ on the parameter space R p . Conditions on g for strong admissibility of the posterior are provided.

Variable smoothing in Bayesian intrinsic autoregressions

AbstractWe introduce an adapted form of the Markov random field (MRF) for Bayesian spatial smoothing with small‐area data. This new scheme allows the amount of smoothing to vary in different parts of a map by employing area‐specific smoothing parameters, related to the variance of the MRF. We take an empirical Bayes approach, using variance information from a standard MRF analysis to provide prior information for the smoothing parameters of the adapted MRF. The scheme is shown to produce proper posterior distributions for a broad class of models. We test our method on both simulated and real data sets, and for the simulated data sets, the new scheme is found to improve modelling of both slowly‐varying levels of smoothness and discontinuities in the response surface. Copyright © 2007 John Wiley & Sons, Ltd.

Bayesian analysis of a Tobit quantile regression model

This paper develops a Bayesian framework for Tobit quantile regression. Our approach is organized around a likelihood function that is based on the asymmetric Laplace distribution, a choice that turns out to be natural in this context. We discuss families of prior distributions on the quantile regression vector that lead to proper posterior distributions with finite moments. We show how the posterior distribution can be sampled and summarized by Markov chain Monte Carlo methods. A method for comparing alternative quantile regression models is also developed and illustrated. The techniques are illustrated with both simulated and real data. In particular, in an empirical comparison, our approach out-performed two other common classical estimators.

Computational Discovery of Gene Regulatory Binding Motifs: A Bayesian Perspective

The Bayesian approach together with Markov chain Monte Carlo techniques has provided an attractive solution to many important bioinformatics problems such as multiple sequence alignment, microarray analysis and the discovery of gene regulatory binding motifs. The employment of such methods and, more broadly, explicit statistical modeling, has revolutionized the field of computational biology. After reviewing several heuristics-based computational methods, this article presents a systematic account of Bayesian formulations and solutions to the motif discovery problem. Generalizations are made to further enhance the Bayesian approach. Motivated by the need of a speedy algorithm, we also provide a perspective of the problem from the viewpoint of optimizing a scoring function. We observe that scoring functions resulting from proper posterior distributions, or approximations to such distributions, showed the best performance and can be used to improve upon existing motif-finding programs. Simulation analyses and a real-data example are used to support our observation.

A prior for the variance in hierarchical models

AbstractThe choice of prior distributions for the variances can be important and quite difficult in Bayesian hierarchical and variance component models. For situations where little prior information is available, a ‘nonin‐formative’ type prior is usually chosen. ‘Noninformative’ priors have been discussed by many authors and used in many contexts. However, care must be taken using these prior distributions as many are improper and thus, can lead to improper posterior distributions. Additionally, in small samples, these priors can be ‘informative’. In this paper, we investigate a proper ‘vague’ prior, the uniform shrinkage prior (Strawder‐man 1971; Christiansen & Morris 1997). We discuss its properties and show how posterior distributions for common hierarchical models using this prior lead to proper posterior distributions. We also illustrate the attractive frequentist properties of this prior for a normal hierarchical model including testing and estimation. To conclude, we generalize this prior to the multivariate situation of a covariance matrix.