Estimation Of Normal Mean Research Articles

Erratum: Improved multivariate normal mean estimation with unknown covariance when p is greater than n

Erratum: Improved multivariate normal mean estimation with unknown covariance when p is greater than n

High-dimensional Bernstein–von Mises theorem for the Diaconis–Ylvisaker prior

We study the asymptotic normality of the posterior distribution of canonical parameter in the exponential family under the Diaconis–Ylvisaker prior which is a conjugate prior when the dimension of parameter space increases with the sample size. We prove under mild conditions on the true parameter value θ0 and hyperparameters of priors, the difference between the posterior distribution and a normal distribution centered at the maximum likelihood estimator, and variance equal to the inverse of the Fisher information matrix goes to 0 in the expected total variation distance. The proof assumes dimension of parameter space d grows linearly with sample size n only requiring d=o(n). En route, we derive a concentration inequality of the quadratic form of the maximum likelihood estimator without any specific assumption such as sub-Gaussianity. A specific illustration is provided for the Multinomial-Dirichlet model with an extension to the density estimation and Normal mean estimation problems.

Global-Local Shrinkage Priors for Asymptotic Point and Interval Estimation of Normal Means under Sparsity

Global-Local Shrinkage Priors for Asymptotic Point and Interval Estimation of Normal Means under Sparsity

Theory of new second-order expansions for the moments of $${100\rho \%}$$ accelerated sequential stopping times in normal mean estimation problems when $${0<\rho <1}$$ is arbitrary

Theory of new second-order expansions for the moments of $${100\rho \%}$$ accelerated sequential stopping times in normal mean estimation problems when $${0<\rho <1}$$ is arbitrary

Simultaneous estimation of normal means with side information

The integrative analysis of multiple datasets is an important strategy in data analysis. It is increasingly popular in genomics, which enjoys a wealth of publicly available datasets that can be compared, contrasted, and combined in order to extract novel scientific insights. This paper studies a stylized example of data integration for a classical statistical problem: leveraging side information to estimate a vector of normal means. This task is formulated as a compound decision problem, an oracle integrative decision rule is derived, and a data-driven estimate of this rule based on minimizing an unbiased estimate of its risk is proposed. The data-driven rule is shown to asymptotically achieve the minimum possible risk among all separable decision rules, and it can outperform existing methods in numerical properties. The proposed procedure leads naturally to an integrative high-dimensional classification procedure, which is illustrated by combining data from two independent gene expression profiling studies.

Estimation of normal means in the tree order model by the weighting methods

Consider k + 1 independent normal populations with the tree order restriction on the mean parameters. For the tree order model, the restricted estimator of control group parameter is dominated by the unrestricted estimator when the number of treatment groups is large. We discuss two techniques for reducing of mean squared error via to the two weighting methods which are dissimilarity and conditional Bayesian criteria. Based on the bias and mean squared error criteria, the performance of the proposed estimators is compared with the alternative estimators in order to search for a better estimator. Although the superior estimator that uniformly dominates the others does not exist in general, but the proposed estimators dominate the corresponding unrestricted estimator and compete very well with the other alternative estimators introduced by the authors.

Minimax Estimation of Discrete Distributions Under <inline-formula> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> Loss

We consider the problem of discrete distribution estimation under $\ell _{1}$ loss. We provide tight upper and lower bounds on the maximum risk of the empirical distribution (the maximum likelihood estimator), and the minimax risk in regimes where the support size $S$ may grow with the number of observations $n$ . We show that among distributions with bounded entropy $H$ , the asymptotic maximum risk for the empirical distribution is $2H/\ln n$ , while the asymptotic minimax risk is $H/\ln n$ . Moreover, we show that a hard-thresholding estimator oblivious to the unknown upper bound $H$ , is essentially minimax. However, if we constrain the estimates to lie in the simplex of probability distributions, then the asymptotic minimax risk is again $2H/\ln n$ . We draw connections between our work and the literature on density estimation, entropy estimation, total variation distance ( $\ell _{1}$ divergence) estimation, joint distribution estimation in stochastic processes, normal mean estimation, and adaptive estimation.

Improved multivariate normal mean estimation with unknown covariance when $p$ is greater than $n$

We consider the problem of estimating the mean vector of a $p$-variate normal $(\theta,\Sigma)$ distribution under invariant quadratic loss, $(\delta-\theta)'\Sigma^{-1}(\delta-\theta)$, when the covariance is unknown. We propose a new class of estimators that dominate the usual estimator $\delta^{0}(X)=X$. The proposed estimators of $\theta$ depend upon $X$ and an independent Wishart matrix $S$ with $n$ degrees of freedom, however, $S$ is singular almost surely when $p>n$. The proof of domination involves the development of some new unbiased estimators of risk for the $p>n$ setting. We also find some relationships between the amount of domination and the magnitudes of $n$ and $p$.

On regularized general empirical Bayes estimation of normal means

In this paper we study a monotone regularized kernel general empirical Bayes method for the estimation of a vector of normal means. This estimator is used to improve upon the kernel methods of Zhang (1997) [12] and Brown and Greenshtein (2009) [5]. We prove an oracle inequality for the regret of the proposed estimator compared with the optimal Bayes risk. The oracle inequality leads to the property that the ratio of the proposed estimator to that of the Bayes procedure approaches one, under mild conditions. We demonstrate the performance of the estimator in simulation experiments with sparse and normal setups. It turns out that the proposed procedure indeed improves over its kernel version.

Model selection and sharp asymptotic minimaxity

We obtain sharp minimax results for estimation of an n-dimensional normal mean under quadratic loss. The estimators are chosen by penalized least squares with a penalty that grows like ck log(n/k), for k equal to the number of nonzero elements in the estimating vector. For a wide range of sparse parameter spaces, we show that the penalized estimator achieves the exact minimax rate with the correct multiplication constant if and only if c equals 2. Our results unify the theory obtained by many other authors for penalized estimation of normal means. In particular we establish that a conjecture by Abramovich et al. (Ann Stat 34:584–653, 2006) is true.

General maximum likelihood empirical Bayes estimation of normal means

We propose a general maximum likelihood empirical Bayes (GMLEB) method for the estimation of a mean vector based on observations with i.i.d. normal errors. We prove that under mild moment conditions on the unknown means, the average mean squared error (MSE) of the GMLEB is within an infinitesimal fraction of the minimum average MSE among all separable estimators which use a single deterministic estimating function on individual observations, provided that the risk is of greater order than (log n)5/n. We also prove that the GMLEB is uniformly approximately minimax in regular and weak ℓp balls when the order of the length-normalized norm of the unknown means is between (log n)κ1/n1/(p∧2) and n/(log n)κ2. Simulation experiments demonstrate that the GMLEB outperforms the James–Stein and several state-of-the-art threshold estimators in a wide range of settings without much down side.

On Stein Type Two-Stage Shrinkage Testimator

The present article deals with an improved estimator of normal mean which is obtained by considering a two-stage procedure with appropriate shrinkage weight function. Indeed, two-stage shrunken testimators of Stein-type are proposed for the mean µ of normal distribution when a prior estimate µ0 of the mean µ is available. The goodness of this procedure as a means of reducing the time, using minimum cost of experimentation and maximizing the relative efficiency is explained in this paper. Computer-intensive calculations for bias, mean-squared error, relative efficiency, expected sample size, probability of avoiding the second-stage sample, and the percentage of the overall sample-saved expressions show that for different values of the first- and second-stage samples, for different values of levels of significance, and for varying the constants involved in the proposed estimator, the proposed testimator fares better than classical estimator and existing two stage testimation procedures.

Minimax estimation of normal precisions via expansion estimators

In this paper, the simultaneous estimation of the precision parameters of k normal distributions is considered under the squared loss function in a decision-theoretic framework. Several classes of minimax estimators are derived by using the chi-square identity, and the generalized Bayes minimax estimators are developed out of the classes. It is also shown that the improvement on the unbiased estimators is characterized by the superharmonic function. This corresponds to Stein's [1981. Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9, 1135–1151] result in simultaneous estimation of normal means.

On minimaxity and admissibility of hierarchical Bayes estimators

This paper obtains conditions for minimaxity of hierarchical Bayes estimators in the estimation of a mean vector of a multivariate normal distribution. Hierarchical prior distributions with three types of second stage priors are treated. Conditions for admissibility and inadmissibility of the hierarchical Bayes estimators are also derived using the arguments in Berger and Strawderman [Choice of hierarchical priors: admissibility in estimation of normal means, Ann. Statist. 24 (1996) 931–951]. Combining these results yields admissible and minimax hierarchical Bayes estimators.

NS Condition of Admissibility for the Linear Estimator of Normal Mean with Unknown Variance

Suppose Y ∼ N(β, σ2In), where β ∈ Rn and σ2 > 0 are unknown. We study the admissibility of linear estimators of mean vector under a quadratic loss function. A necessary and suffcient condition of the admissible linear estimator is given.

On an admissibility problem involving the moment generating function of the lognormal distribution

A new location invariant loss function is considered and the best invariant estimator of normal mean is obtained. This estimator is a function of the moment generating function of the lognormal distribution. The admissibility is studied of a class of linear estimators of the form cX + d, where X ∼ N(θ, σ2), with θ unknown and σ2 known. This yields the admissibility of the best invariant estimator of θ.

An analysis of the influence of some prior specifications in the identification of change points via product partition model

In this paper, we consider the product partition model for the estimation of normal means and variances of a sequence of observations that experiences changes in these parameters at unknown times. The estimates of the parameters by using product partition model are the weighted average of the estimates based in blocks (groups) of observations by the posterior relevance of these blocks which depends on the prior cohesions. We implement the Barry and Hartigan's method to this change point problem and propose an easy-to-implement modification to their method. We use Yao's prior cohesions and investigate the influence of different prior distributions to the parameter that indexes these cohesions in the product estimates. A comparison between the estimates obtained by using both these methods and those provided by using Yao's method is done considering different settings for its application. We apply the three methods presented in this paper to stock market data. The results seem to indicate that the method proposed is competitive and also that the prior specifications influence in the product estimates.

Estimation of bias and standard error of an improved estimator of normal mean

This paper considers an improved estimator of normal mean which is obtained by considering a feasible version of minimum mean squared error estimator. The exact expression for the bias and the mean squared error are fairly complicated and do not provide any guidelines as how to estimate the standard error of improved estimator. As is well known that any estimator without a formula for standard error has little practical utility. We therefore derive unbiased estimators for the bias and mean squared error of the improved estimator. Incidently, they turn out to be minimum variance unbiased estimators. Further, this exercise yields a simple formula for estimating the standard error. Based on the criterion of estimated standard error, the efficiency of the improved estimator with respect to the traditional unbiased estimator (i.e., sample mean) is examined numerically. The relationship with asymptotic standard error is also studied.

Estimation of clustered parameters

Abstract We consider probability models for the estimation of normal means that allow for equality among arbitrary subsets of the means. These models, which are called product partition models, assign probabilities to random partitions of sets of objects. Here, the objects correspond to the means. We look at two interesting cases – the first is when all the means are equal and the second is when there are several sets of equal means that are far apart. We show that the posterior distribution of the number of sets in the partition is asymptotically Poisson-like. This will help us calibrate the choice of one of our prior parameters. Finally, we look at simulations to see how well the above results hold for moderate sample sizes.

Sequential estimation of normal mean under asymmetric loss function with a shrinkage stopping rule

The problem of estimating a normal mean with unknown variance is considered under an asymmetric loss function such that the associated risk is bounded from above by a known quantity. In the absence of a fixed sample size rule, a sequential stopping rule and two sequential estimators of the mean are proposed and second-order asymptotic expansions of their risk functions are derived. It is demonstrated that the sample mean becomes asymptotically inadmissible, being dominated by a shrinkage-type estimator. Also a shrinkage factor is incorporated in the stopping rule and similar inadmissibility results are established.