Improper Prior Distributions Research Articles

A NOTE ON BAYESIAN ANALYSIS OF DECAPITATED GENERALIZED POISSON DISTRIBUTION UNDER VARIOUS LOSS FUNCTIONS

It is recognized that, for Bayes estimators, the performance depends on the form of the prior distribution and the assumed loss function. This paper resolves the problem of estimation of one parameter decapitated generalized Poisson distribution; using class of improper prior distributions under symmetric and asymmetric loss functions. The statistical performances of the Bayes estimates with respect to different priors and loss functions are compared using mean square error based on simulation study.

Bayes Estimator as a Function of Some Classical Estimator

Maximum likelihood estimation method, uniformly minimum variance unbiased estimation method and minimum mean square error estimation, as classical estimation procedures, are frequently used for parameter estimation in statistics, which assuming the parameter is constant , while Bayes method assuming the parameter is random variable and hence the Bayes estimator is an estimator which minimize the Bayes risk for each value the random observable and for square error lose function the Bayes estimator is the posterior mean. It is well known that the Bayesian estimation is hardly used as a parameter estimation technique due to some difficulties to finding a prior distribution.
 The interest of this paper is that whether above classical estimators of the parameter for a particular probability distribution can be obtained from Bayes estimator is determined. In this analysis one-parameter Pareto distribution is used to examine the relationship between Bayesian and classical estimators. Considering improper prior distribution for shape parameter of the Pareto distribution of the first kind with known scale parameter which equals one, we have tried to show how the classical estimators can be obtain from Bayes estimator for various choices of hyper parameters of the prior function.
 

Bayesian inference for Poisson and multinomial log-linear models

Categorical data frequently arise in applications in the Social Sciences. In such applications, the class of log-linear models, based on either a Poisson or (product) multinomial response distribution, is a flexible model class for inference and prediction. In this paper we consider the Bayesian analysis of both Poisson and multinomial log-linear models. It is often convenient to model multinomial or product multinomial data as observations of independent Poisson variables. For multinomial data, Lindley (1964) [20] showed that this approach leads to valid Bayesian posterior inferences when the prior density for the Poisson cell means factorises in a particular way. We develop this result to provide a general framework for the analysis of multinomial or product multinomial data using a Poisson log-linear model. Valid finite population inferences are also available, which can be particularly important in modelling social data. We then focus particular attention on multivariate normal prior distributions for the log-linear model parameters. Here, an improper prior distribution for certain Poisson model parameters is required for valid multinomial analysis, and we derive conditions under which the resulting posterior distribution is proper. We also consider the construction of prior distributions across models, and for model parameters, when uncertainty exists about the appropriate form of the model. We present classes of Poisson and multinomial models, invariant under certain natural groups of permutations of the cells. We demonstrate that, if prior belief concerning the model parameters is also invariant, as is the case in a ‘reference’ analysis, then the choice of prior distribution is considerably restricted. The analysis of multivariate categorical data in the form of a contingency table is considered in detail. We illustrate the methods with two examples.

El análisis bayesiano y la precisión de los valores de la heredabilidad en especies perennes

Los objetivos del presente trabajo fueron evaluar la precisión del valor estimado de la heredabilidad determinada por medio de la desviación estándar, considerando un enfoque Bayesiano, y comparar tal estimativa con el procedimiento clásico. Se utilizaron datos de un ensayo de progenie con 39 familias de Eucalyptus cladocalyx. La variable dependiente usada fue el diámetro basal del fuste medido a los seis años de edad. El método Bayesiano fue implementado por medio del algoritmo de Cadenas Independientes, con informaciones a priori informativas, el cual entregó bajos valores de desviaciones estándar de la heredabilidad, si comparado con la estimación clásica de Robertson y distribución a priori de Jeffreys (la cual es una clase de distribución a priori no informativa y a menudo impropia). El método de análisis Bayesiano es una herramienta de inferencia valiosa para la evaluación genética de especies perennes, ya que considera la variabilidad de los parámetros por medio de las distribuciones a posteriori.

Online data processing: Comparison of Bayesian regularized particle filters

The aim of this paper is to compare three regularized particle filters in an online data processing context. We carry out the comparison in terms of hidden states filtering and parameters estimation, considering a Bayesian paradigm and a univariate stochastic volatility model. We discuss the use of an improper prior distribution in the initialization of the filtering procedure and show that the Regularized Auxiliary Particle Filter (R-APF) outperforms the Regularized Sequential Importance Sampling (R-SIS) and the Regularized Sampling Importance Resampling (R-SIR).

Evaluation of formal posterior distributions via Markov chain arguments

We consider evaluation of proper posterior distributions obtained from improper prior distributions. Our context is estimating a bounded function Φ of a parameter when the loss is quadratic. If the posterior mean of 0 is admissible for all bounded Φ, the posterior is strongly admissible. We give sufficient conditions for strong admissibility. These conditions involve the recurrence of a Markov chain associated with the estimation problem. We develop general sufficient conditions for recurrence of general state space Markov chains that are also of independent interest. Our main example concerns the p-dimensional multivariate normal distribution with mean vector 0 when the prior distribution has the form g(∥6∥ 2 ) dθ on the parameter space R p . Conditions on g for strong admissibility of the posterior are provided.

Classical and Bayesian interpretation of the Birge test of consistency and its generalized version for correlated results from interlaboratory evaluations

A well-known test of consistency in the results from an interlaboratory evaluation is the Birge test, named after its developer Raymond T Birge, a physicist. We show that the Birge test of consistency may be interpreted as a classical test of the null hypothesis that the variances of the results are less than or equal to their stated values against the alternative hypothesis that the variances of the results are greater than their stated values. A modern protocol for hypothesis testing is to calculate the classical p-value of the test statistic. The p-value is the maximum probability under the null hypothesis of realizing in conceptual replications a value of the test statistic equal to or larger than the realized (observed) value of the test statistic. The null hypothesis is rejected when the p-value is too small. We show that, interestingly, the classical p-value of the Birge test statistic is equal to the Bayesian posterior probability of the null hypothesis based on suitably chosen non-informative improper prior distributions for the unknown statistical parameters. Thus the Birge test may be interpreted also as a Bayesian test of the null hypothesis. The Birge test of consistency was developed for those interlaboratory evaluations where the results are uncorrelated. We present a general test of consistency for both correlated and uncorrelated results. Then we show that the classical p-value of the general test statistic is equal to the Bayesian posterior probability of the null hypothesis based on non-informative prior distributions. The general test makes it possible to check the consistency of correlated results from interlaboratory evaluations. The Birge test is a special case of the general test.

Propriety of posterior in Bayesian space varying parameter models with normal data

In Bayesian spatially varying parameter models the covariates’ coefficients in a regression model are allowed to change smoothly in space. A Markov random field is adopted as an improper prior distribution for the area-specific spatial effects. We demonstrate that the posterior distribution is a proper probability distribution.

Objective Bayesian analysis for the Student-t regression model

We develop a Bayesian analysis based on two different Jeffreys priors for the Student-t regression model with unknown degrees of freedom. It is typically difficult to estimate the number of degrees of freedom: improper prior distributions may lead to improper posterior distributions, whereas proper prior distributions may dominate the analysis. We show that Bayesian analysis with either of the two considered Jeffreys priors provides a proper posterior distribution. Finally, we show that Bayesian estimators based on Jeffreys analysis compare favourably to other Bayesian estimators based on priors previously proposed in the literature.

Comment: Bayesian Checking of the Second Level of Hierarchical Models: Cross-Validated Posterior Predictive Checks Using Discrepancy Measures

We compliment Bayarri and Castellanos (BC) on producing an interesting and insightful paper on model checking applied to the second level of hierarchical models. Distributions of test statistics (functions of the observed data not involving parameters) for judg ing appropriateness of hierarchical models typically in volve nuisance (i.e., unknown) parameters. BC (2007) focus on ways to remove the dependency on nui sance parameters so that test statistics can be used to assess models, either through p-values or Berger's relative predictive surprise (RPS). They demonstrate shortcomings in terms of very low power of posterior predictive checks and a posterior empirical Bayesian method. They also demonstrate better performance of their partial posterior predictive (ppp) method over a prior empirical Bayesian method. Methods of Dey et al. (1998), O'Hagan (2003) and Marshall and Spiegel halter (2003) also are compared. Methods are contrasted in terms of whether they re quire proper prior distributions, how many measures of surprise (one per group or one total) are produced, and the degree to which data are used twice in estimation and testing. Their preferred method (ppp) can use im proper prior distributions, which are referred to as ob jective, produces a single measure of surprise for each test statistic, and avoids double use of the data. For the models and statistics considered, in comparison to the alternatives presented, ppp has a more uniform null dis tribution of p -values and more power versus alterna tives.

When is Eaton’s Markov chain irreducible?

Consider a parametric statistical model P(dx\0) and an improper prior distribution v(d0) that together yield a (proper) formal posterior distribution Q(d6\x). The prior is called strongly admissible if the generalized Bayes estimator of every bounded function of 0 is admissible under squared error loss. Eaton [Ann. Sta tist. 20 (1992) 1147-1179] has shown that a sufficient condition for strong admissibility of v is the local recurrence of the Markov chain whose transition function is R(0, dij) = f Q(dr)\x)P(dx\6). Applications of this result and its extensions are often greatly simplified when the Markov chain associated with R is ir reducible. However, establishing irreducibility can be difficult. In this paper, we provide a characterization of irreducibility for general state space Markov chains and use this characterization to develop an easily checked, necessary and sufficient condition for irreducibility of Eaton's Markov chain. All that is required to check this condition is a simple examination of P and v. Application of the main result is illustrated using two examples.

On perturbations of strongly admissible prior distributions

Consider a parametric statistical model, P ( d x | θ ) , and an improper prior distribution, ν ( d θ ) , that together yield a (proper) formal posterior distribution, Q ( d θ | x ) . The prior is called strongly admissible if the generalized Bayes estimator of every bounded function of θ is admissible under squared error loss. Eaton [M.L. Eaton, A statistical diptych: Admissible inferences-recurrence of symmetric Markov chains, Annals of Statistics 20 (1992) 1147–1179] used the Blyth–Stein Lemma to develop a sufficient condition, call it C , for strong admissibility of ν. Our main result says that, under mild regularity conditions, if ν satisfies C and g ( θ ) is a bounded, non-negative function, then the perturbed prior distribution g ( θ ) ν ( d θ ) also satisfies C and is therefore strongly admissible. Our proof has three basic components: (i) Eaton's [M.L. Eaton, A statistical diptych: Admissible inferences-recurrence of symmetric Markov chains, Annals of Statistics 20 (1992) 1147–1179] result that the condition C is equivalent to the local recurrence of the Markov chain whose transition function is R ( d θ | η ) = ∫ Q ( d θ | x ) P ( d x | η ) ; (ii) a new result for general state space Markov chains giving conditions under which local recurrence is equivalent to recurrence; and (iii) a new generalization of Hobert and Robert's [J.P. Hobert, C.P. Robert, Eaton's Markov chain, its conjugate partner and P -admissibility, Annals of Statistics 27 (1999) 361–373] result that says Eaton's Markov chain is recurrent if and only if the chain with transition function R ˜ ( d x | y ) = ∫ P ( d x | θ ) Q ( d θ | y ) is recurrent. One important application of our results involves the construction of strongly admissible prior distributions for estimation problems with restricted parameter spaces.

A Bayesian estimator of process capability index

Process capability and process performance index studies are to assess a process relative to specification criteria. Quantification of this measurement is often reported using various indices. One such index namely Cp has been considered in this paper. The equations for process capability indices are basically very simple; however, they are very sensitive to the input value for standard deviation, as it is not known. Unfortunately there can be difference of opinion on how to determine standard deviation for a given situation. Thus the basic problem is to estimate the standard deviation and thereby estimating the process capability index. The paper provides a Bayes estimator for process capability index Cp when a priori or guessed interval of standard deviation is available. To express the belief of the experimenter, an improper prior distribution is considered. The squared error loss function has been used to assess goodness of the suggested estimator. The Bayes estimator thus obtained has been compared theoretically and empirically with the minimum mean squared error estimator.

Simultaneous prediction of independent Poisson observables

Simultaneous predictive distributions for independent Poisson observables are investigated. A class of improper prior distributions for Poisson means is introduced. The Bayesian predictive distributions based on priors from the introduced class are shown to be admissible under the Kullback-Leibler loss. A Bayesian predictive distribution based on a prior in this class dominates the Bayesian predictive distribution based on the Jeffreys prior.

Bayes estimation of intraclass correlation coefficient

Bayes estimation of the intraciass correlation coefficient, ρ, based on improper prior distribution of the eigen values of the covariance matrix is studied. Asymptotic expansion of the Bayes estimator of in relation to the maximum likelihood estimator of is given. Exact expression of and asymptotic distribution of are also dervied. Bayesian percentile estimator form the poterior density of ρ and Bayesian interval estimator of ρ are also provided.

Existence of Bayesian Estimates for the Polychotomous Quantal Response Models

This paper investigates the existence of Bayesian estimates for polychotomous quantal response models using a uniform improper prior distribution on the regression parameters. Necessary and sufficient conditions for the propriety of the posterior distribution with a general link function are established. In addition, the sufficient conditions for the existence of the posterior moments and the posterior moment generating function are obtained. It is also found that the propriety guarantees the existence of the maximum likelihood estimate.

Consistency and Strong Inconsistency of Group-Invariant Predictive Inferences

Consider a statistical model which is invariant under a group of transformations that acts transitively on the parameter space. In this situation, the problem of constructing invariant predictive distributions is considered. It is shown, under certain assumptions, that Fisherian pivoting and the use of right Haar measure as an improper prior distribution both yield the same invariant predictive distribution. Furthermore, it is shown that any other invariant predictive distribution is strongly inconsistent in the sense of Stone.

Bayes Estimators for the Continuous Uniform Distribution

Classical estimators for the parameter of a uniform distribution on the interval (0, θ) are often discussed in mathematical statistics courses, but students are frequently left wondering how to distinguish which among the variety of classical estimators are better than the others. We show how classical estimators can be derived as Bayes estimators from a family of improper prior distributions. We believe that linking the estimation criteria in a Bayesian framework is of value to students in a mathematical statistics course, and we believe that the students benefit from the exposure to Bayesian methods. In addition, we compare classical and Bayesian interval estimators for the parameter θ and illustrate the Bayesian analysis with an example.

Improper and proper posteriors with improper priors in a hierarchical model with a beta-binomial likelihood

In a multistage hierarchical model statisticians often use improper prior distributions in the last stage of the model. The posterior distribution of the parameters, however, is not always proper when the prior is improper. The purpose of the paper is to derive necessary and sufficient conditions for the posterior distribution of the parameters to be proper in a hierarchical model with a beta-binomial likelihood if the prior belongs to a large family of improper distributions. Conditions are given also for the posterior cross-moments of the parameters to be finite.

A Statistical Diptych: Admissible Inferences--Recurrence of Symmetric Markov Chains

Given a parametric model and an improper prior distribution, the formal posterior distribution induces decision rules in any decision problem. The results here provide conditions under which this formal Bayes method produces admissible decision rules for all quadratically regular decision problems. The conditions derived are shown to be equivalent to the recurrence of a natural symmetric Markov chain (on the parameter space) generated by the model and the improper prior. The results are also used to give conditions under which formal predictive distributions are admissible decision rules in certain prediction problems.