Posterior Expectation Research Articles

Fully Exponential Laplace Approximations to Expectations and Variances of Nonpositive Functions

Abstract Tierney and Kadane (1986) presented a simple second-order approximation for posterior expectations of positive functions. They used Laplace's method for asymptotic evaluation of integrals, in which the integrand is written as f(θ)exp(-nh(θ)) and the function h is approximated by a quadratic. The form in which they applied Laplace's method, however, was fully exponential: The integrand was written instead as exp[− nh(θ) + log f(θ)]; this allowed first-order approximations to be used in the numerator and denominator of a ratio of integrals to produce a second-order expansion for the ratio. Other second-order expansions (Hartigan 1965; Johnson 1970; Lindley 1961, 1980; Mosteller and Wallace 1964) require computation of more derivatives of the log-likelihood function. In this article we extend the fully exponential method to apply to expectations and variances of nonpositive functions. To obtain a second-order approximation to an expectation E(g(θ)), we use the fully exponential method to approximate...

Approximate methods for assessing influence and sensitivity in Bayesian analysis

We use asymptotic approximations to posterior expectations in deriving diagnostics for case influence and prior sensitivity. We obtain an easily-interpreted expression for the change in posterior expectation following perturbation of the problem, which also produces a useful interpretation of the likelihood displacement discussed by Cook & Weisberg. In addition, we propose two simple diagnostics of posterior nonnormality. The leukaemia data analysed by Feigl & Zelen are used to illustrate the diagnostic measures we discuss

On robust empirical bayes analysis of means and variances from stratified samples

Ghosh and Lahiri (1987a,b) considered simultaneous estimation of several strata means and variances where each stratum contains a finite number of elements, under the assumption that the posterior expectation of any stratum mean is a linear function of the sample observations - the so called“posterior linearity” property. In this paper we extend their result by retaining the “posterior linearity“ property of each stratum mean but allowing the superpopulation model whose mean as well as the variance-covariance structure changes from stratum to stratum. The performance of the proposed empirical Bayes estimators are found to be satisfactory both in terms of “asymptotic optimality” (Robbins (1955)) and “relative savings loss” (Efron and Morris (1973)).

Asymptotic expansions of posterior expectations, distributions and densities for stochastic processes

Asymptotic expansions are derived for Bayesian posterior expectations, distribution functions and density functions. The observations constitute a general stochastic process in discrete or continuous time.

Characterizing priors by posterior expectations in multiparameter exponential families

Exploiting the notion of identifiability of mixtures of exponential families with respect to a vector parameter θ, it is shown that the posterior expectation of θ characterizes the prior distribution of θ. The result is applied to normal and negative multinomial distributions.

Robust Empirical Bayes Estimation of Means From Stratified Samples

Abstract This article considers simultaneous estimation of means from several strata, where each stratum contains a finite number of elements. For example, quite often estimates of annual incomes, unemployment rates, and so forth must be made simultaneously for many areas. Our results find application in small area estimation where very few samples are available from an individual area, and an estimate of a certain area mean or simultaneous estimates of several area means can be improved by incorporating information from similar neighboring areas. Ghosh and Meeden (1986) considered empirical Bayes estimation of the finite population mean assuming a normal superpopulation model. Their estimators generalize automatically when one estimates a vector of stratum means. This article relaxes the normality assumption and assumes instead that the posterior expectation of any stratum mean is a linear function of the sample observations. This is the so-called “posterior linearity” property as described in Ericson (1969b), Hartigan (1969), and Goldstein (1975), among others. The empirical Bayes estimators of Ghosh and Meeden (1986), however, enjoy a certain robustness property in the sense that they can be motivated with the basic assumption of posterior linearity rather than normality of the superpopulation. For a small number of strata, Monte Carlo simulations undertaken in this article clearly indicate that the Bayes risks of these empirical Bayes estimators are usually much smaller than the Bayes risk vector of sample means not only when the underlying distribution is normal, but also when they are binomial and Poisson. In addition, in the binomial case, the empirical Bayes estimators of Ghosh and Meeden perform better than some rival empirical Bayes estimators that can be developed along the lines of Morris (1983). This is quite interesting, especially since the latter uses the additional information about the form of the variance of the binomial distribution as a function of the mean. For the Poisson case, our estimators perform very closely to some of the rival estimators that can be proposed along the lines of Morris (1983) using the additional information that the variance of a Poisson distribution equals its mean. We have also found that several asymptotic (as the number of strata is large) optimality properties of the empirical Bayes estimators of Ghosh and Meeden established under the normality of the superpopulation continue to hold under the modest assumption of existence of certain superpopulation moments. This type of asymptotics is particularly suited for small area estimation where the number of small areas is very large but the sample size within each small area is typically very small. It is interesting to note that when samples of equal size are drawn from different strata, the empirical Bayes estimators of Ghosh and Meeden (1986) include as special cases the James—Stein estimators, the positive-part James—Stein estimators, and the estimators of Lindley and Smith (1972). A new feature of the present article is that the Bayes risks of such estimators, calculated without the normality assumption, indicate a clear robustness of procedures motivated originally under normality.

Robust Empirical Bayes Estimation of Means from Stratified Samples

Abstract This article considers simultaneous estimation of means from several strata, where each stratum contains a finite number of elements. For example, quite often estimates of annual incomes, unemployment rates, and so forth must be made simultaneously for many areas. Our results find application in small area estimation where very few samples are available from an individual area, and an estimate of a certain area mean or simultaneous estimates of several area means can be improved by incorporating information from similar neighboring areas. Ghosh and Meeden (1986) considered empirical Bayes estimation of the finite population mean assuming a normal superpopulation model. Their estimators generalize automatically when one estimates a vector of stratum means. This article relaxes the normality assumption and assumes instead that the posterior expectation of any stratum mean is a linear function of the sample observations. This is the so-called “posterior linearity” property as described in Ericson (1...

Empirical Bayes Confidence Intervals Based on Bootstrap Samples

Abstract Consider the model with data generated by the following two-stage process. First, a parameter θ is sampled from a prior distribution G, and then an observation is sampled from the conditional distribution f(y | θ). If the prior distribution is known, then the Bayes estimate under squared error loss is the posterior expectation of θ conditional on the data y. For example, if G is Gaussian with mean μ and variance τ2 and f(y | θ) is Gaussian with mean θ and variance σ2, then the posterior distribution is Gaussian with mean B μ + (1 – B)y and variance σ2(1 – B), where B = σ2/(σ2 + τ2). Inferences about θ are based on this distribution. We study the application of the bootstrap to situations where the prior must be estimated from the data (empirical Bayes methods). For this model, we observe data Y T = [Y 1, …, Y K]T, each independent Y K following the compound model described previously. As first shown by James and Stein (1961), setting each θk equal to its estimated posterior mean, where , and , pr...

A Bayesian Approach to Calculating Sample Sizes for Multinomial Sampling

Bayesian rules are developed for the determination of sample sizes for sampling from multinomial distributions. We develop rules based on the concept of a tolerance region R, such that a vector of parameters of interest, 0 say, is contained in R with a specified probability 1 -a which holds on average over all possible samples. We consider two classes of region (i) ellipsoids which are centred at the posterior expectation of 0 conditional on a sample matrix X and whose shape and orientation are determined by a given symmetric matrix A, and (ii) hyper-cubic regions consisting of intervals which are symmetric about the individual posterior means 6i the elements of 0. These procedures are applied to the sizing of market research surveys, for which the multinomial distribution is an appropriate model.

A note on the posterior expectation

A result by Ericson (1969) states that if are any jointly distributed random variables conditional on an unknown parameter θ which have common means μ(θ) and finite variances and the posterior expectation of μ(θ) is a linear function of , than the posterior expectation takes the form of a weighted average of [Xbar] and the prior expecta-tion of μ(θ), the weights being given in terms of the prior variance of u(9) and the prior expectation of the sampling variance of [Xbar] In this note we present a generalization of this result.

Power series distributions and their conjgates in stochastic modeling and baysian inference

Exponential families and their discrete versions such as the power series distributions (PDS's) have generated renewed interest in applied probaility and mathematical statistics. Under suitable definitions of conjugate pairs and their interrelations, certain PSD's are characterized as having unique conjugate pairs. Within the Baysian framework, it is shown that the linearity of the posterior expectation of the mean parameter of the PSD's characterizes the prior distribution uniquely. Characterizations through other forms of the posterior estiamtes are obtained, and examples are given

Decomposition and Bayesian analysis of invariant normal linear models

Using the theory of linear group representations, we analyse the normal linear model with known sampling covariance structure invariant under a symmetry group, and sampling mean structure equivariant under the same group. In particular, assuming an invariant normal prior distribution on the parameter space, the problem of Bayesian inference is shown to decompose naturally into several independent subproblems. Within any such subproblem, if additional irreducibility conditions hold, it is shown that the posterior expectation of any parameter is a fixed scalar multiple of its unique unbiased estimator, and similarly the posterior covariance of any two parameters is a fixed scalar multiple of the prior covariance. The theoretical framework is illustrated with reference to experimental designs.

Common knowledge, communication, and convergence of beliefs

Abstract A conceptual framework is established for the analysis of common knowledge and communication when information structures are represented by sigma-algebras instead of partitions. It involves construction of Boolean sigma-algebras of ‘generalized events’, events which are identified only up to addition or subtraction of null sets. If all agents' posterior expectations of a random variable are common knowledge, then they are almost surely identical. If the posterior expectations are communicated back and forth, then they converge almost surely to a consensus expectation. If the posterior distributions of a random variable are communicated back and forth, then they will almost surely converge weakly to a consensus distribution.

Estimating a Population of Parameter Values Using Bayes and Empirical Bayes Methods

Abstract In standard Bayes and empirical Bayes component decision problems, estimating inidividual parameters is the primary goal. In multiple comparison problems and in comparisons of histograms of estimates, however, the primary goal is to produce parameter estimates that can be considered as an ensemble. For example, the histogram of estimates should be a good estimate of the histogram of parameters. Standard methods of estimating by the posterior expectation do minimize symmetric, componentwise losses such as squared error, but they produce ensembles of estimates with a sample variance smaller than the posterior expected sample variance for parameters. In this article we propose new Bayes and empirical Bayes estimates that minimize a distance function between the empirical cdf of the estimates and the true parameters. These estimators are weighted averages of the prior mean and the data, with weight on the data being approximately the square root of that for the posterior expectation. We give theoreti...

W. E. Johnson's "Sufficientness" Postulate

How do Bayesians justify using conjugate priors on grounds other than mathematical convenience? In the 1920's the Cambridge philosopher William Ernest Johnson in effect characterized symmetric Dirichlet priors for multinomial sampling in terms of a natural and easily assessed subjective condition. Johnson's proof can be generalized to include asymmetric Dirichlet priors and those finitely exchangeable sequences with linear posterior expectation of success. Some interesting open problems that Johnson's result raises, and its historical and philosophical background, are also discussed.

Bayesian Inference Using Intervals of Measures

Partial prior knowledge is quantified by an interval $I(L, U)$ of $\sigma$-finite prior measures $Q$ satisfying $L(A) \leq Q(A) \leq U(A)$ for all measurable sets $A$, and is interpreted as acceptance of a family of bets. The concept of conditional probability distributions is generalized to that of conditional measures, and Bayes theorem is extended to accommodate unbounded priors. According to Bayes theorem, the interval $I(L, U)$ of prior measures is transformed upon observing $X$ into a similar interval $I(L_x, U_x)$ of posterior measures. Upper and lower expectations and variances induced by such intervals of measures are obtained. Under weak regularity conditions, as the amount of data increases, these upper and lower posterior expectations are strongly consistent estimators. The range of posterior expectations of an arbitrary function $b$ on the parameter space is asymptotically $b_N \pm \alpha\sigma_N + o(\sigma_N)$ where $b_N$ and $\sigma^2_N$ are the posterior mean and variance of $b$ induced by the upper prior measure $U$, and where $\alpha$ is a constant determined by the density of $L$ with respect to $U$ reflecting the uncertainty about the prior.

Only Normal Distributions Have Linear Posterior Expectations in Linear Regression

Abstract We consider the general linear model Y = Aβ + ε in the Bayesian framework and examine the implications of the statement that the posterior expectation of β, given Y, is a linear function of Y. Under various conditions on the model, it is shown that this linear posterior expectation implies that both β and ε are normally distributed. For most of the practical situations in which linear models are used, only normal distributions have linear posterior expectations.

Only Normal Distributions Have Linear Posterior Expectations in Linear Regression

Abstract We consider the general linear model Y = Aβ + e in the Bayesian framework and examine the implications of the statement that the posterior expectation of β, given Y, is a linear function of Y. Under various conditions on the model, it is shown that this linear posterior expectation implies that both β and e are normally distributed. For most of the practical situations in which linear models are used, only normal distributions have linear posterior expectations.

A Note on the Relationship of Minimum Expected Loss (MELO) and Other Structural Coefficient Estimates

In previous work (Zellner, 1978) structural coefficient estimates that minimize the posterior expectation of a generalized quadratic loss function were derived. On computing these minimum expected loss (MELO) estimates for coefficients of Klein's Model I and the Girschick-Haavelmo supply and demand model for food, it was found that the MELO coefficient estimates have values between corresponding direct least squares (DLS) and two stage least squares (2SLS) estimates (see Zellner and Park, 1979). This note explores the relation of MELO, DLS and 2SLS estimates. Below it is shown,that the MELO estimate of a vector of coefficients of endogenous variables in an equation of a linear, interdependent econometric model can be expressed as a matrix weighted average of the DLS and 2SLS estimates. In the scalar case, the MELO estimate can be expressed as a simple weighted average of the DLS and 2SLS estimates. Some properties of these matrix weighted averages are discussed using the results in Chamberlain and Leamer (1974).

Least squares approximation in Bayesian analysis

The paper presents in a simple and unified framework the Least-Squares approximation of posterior expectations. Particular structures of the sampling process and of the prior distribution are used to organize and to generalize previous results. The two basic structures are obtained by considering unbiased estimators and exchangeable processes. These ideas are applied to the estimation of the mean. Sufficient reduction of the data is analysed when only the Least-Squares approximation is involved. © 1980 Springer.