Continuous Exponential Family Research Articles

Numerical solution of Kiefer-Weiss problems when sampling from continuous exponential families

In this article, we deal with problems of testing hypotheses in the framework of sequential statistical analysis. The main concern is the optimal design and performance evaluation of sampling plans in Kiefer-Weiss problems. The main goal of the Kiefer-Weiss problem is designing hypothesis tests that minimize the maximum average sample number, over all parameter values, as opposed to both the sequential probability tests (SPRTs) minimizing the average sample number only at two hypothesis points and the classical fixed-sample-size test. For observations that follow a distribution from an exponential family of the continuous type, we provide algorithms for optimal design in the modified Kiefer-Weiss problem and obtain formulas for evaluating the performance of sequential tests by calculating the operating characteristic function, the average sample number, and some related characteristics. These formulas cover, as a particular case, the SPRTs and their truncated versions, as well as optimal finite-horizon sequential tests. In the setting of the original Kiefer-Weiss problem we apply the method of our recent work (Sequential Analysis 2022, 41(2), 198–219) for numerical construction of the optimal tests. For the particular case of sampling from a normal distribution with a known variance, we make numerical comparisons of the Kiefer-Weiss solution with the SPRT and the fixed-sample-size test provided that the three tests have the same levels of the error probabilities. All of the algorithms are implemented in the form of computer code written in the R programming language and are available at the GitHub public repository (https://github.com/tosinabase/Kiefer-Weiss). Guidelines on the adaptation of the program code to other exponential family distributions are provided.

Chentsov’s theorem for exponential families

Chentsov’s theorem characterizes the Fisher information metric on statistical models as the only Riemannian metric (up to rescaling) that is invariant under sufficient statistics. This implies that each statistical model is equipped with a natural geometry, so Chentsov’s theorem explains why many statistical properties can be described in geometric terms. However, despite being one of the foundational theorems of statistics, Chentsov’s theorem has only been proved previously in very restricted settings or under relatively strong invariance assumptions. We therefore prove a version of this theorem for the important case of exponential families. In particular, we characterise the Fisher information metric as the only Riemannian metric (up to rescaling) on an exponential family and its derived families that is invariant under independent and identically distributed extensions and canonical sufficient statistics. We then extend this result to curved exponential families. Our approach is based on the central limit theorem, so it gives a unified proof for discrete and continuous exponential families, and it is less technical than previous approaches.

Maximum likelihood estimator of the parameter for a continuous one-parameter exponential family under the optimal ranked set sampling

This paper studies a maximum likelihood estimator (MLE) of the parameter for a continuous one-parameter exponential family under ranked set sampling (RSS). The authors first find the optimal RSS according to the character of the family, viz, arrange the RSS based on quasi complete and sufficient statistic of independent and identically distributed (iid) samples. Then under this RSS, some sufficient conditions for the existence and uniqueness of the MLE, which are easily used in practice, are obtained. Using these conditions, the existence and uniqueness of the MLEs of the parameters for some usual distributions in this family are proved. Numerical simulations for these distributions fully support the result from the above two step optimizations of the sampling and the estimation method.

Asymptotic adjustments of Pearson residuals in exponential family nonlinear models

ABSTRACTIn this work we define a set of corrected Pearson residuals for continuous exponential family nonlinear models that have the same distribution as the true Pearson residuals up to order , where n is the sample size. Furthermore, we also introduce a new modification of the Pearson residuals, which we call PCA Pearson residuals, that are approximately uncorrelated. These PCA residuals are new even for the generalized linear models. The numerical results show that the PCA residuals are approximately normally distributed, thus improving previous results by Simas and Cordeiro [Adjusted Pearson residuals in exponential family nonlinear models. J Stat Comput Simul. 2009;79:411–425]. These numerical results also show that the corrected Pearson residuals approximately follow the same distribution as the true residuals, which is a considerable improvement with respect to the Pearson residuals and also extends the previous work by Cordeiro and Simas [The distribution of Pearson residuals in generalized linear models. Comput Stat Data Anal. 2009;53:3397–3411].

Empirical Bayes estimation in continuous one-parameter exponential families under associated samples

ABSTRACTIn this paper, we study the empirical Bayes (EB) estimation in continuous one-parameter exponential families under negatively associated (NA) samples and positively associated (PA) samples. Under certain regularity conditions, it is shown that the convergence rates of proposed EB estimators under NA or PA samples are the same as those of EB estimators under independent observations, which significantly improve the existing results in EB estimation under associated samples.

Empirical Bayes Test Problem in Continuous One-parameter Exponential Families under Dependent Samples

In this paper, we study the empirical Byes (EB) test problem in the continuous one-parameter exponential family under associated samples and strong mixing samples. Under mild regularity conditions, it is shown that the convergence rates of proposed EB test rules under associated or strong mixing samples are the same as that of EB test rules under independent observations.

Fiducial and Confidence Distributions for Real Exponential Families

AbstractWe develop an easy and direct way to define and compute the fiducial distribution of a real parameter for both continuous and discrete exponential families. Furthermore, such a distribution satisfies the requirements to be considered a confidence distribution. Many examples are provided for models, which, although very simple, are widely used in applications. A characterization of the families for which the fiducial distribution coincides with a Bayesian posterior is given, and the strict connection with Jeffreys prior is shown. Asymptotic expansions of fiducial distributions are obtained without any further assumptions, and again, the relationship with the objective Bayesian analysis is pointed out. Finally, using the Edgeworth expansions, we compare the coverage of the fiducial intervals with that of other common intervals, proving the good behaviour of the former.

Two-sided Empirical Bayes test for the exponential family with contaminated data

In this study, the two-sided Empirical Bayes test (EBT) rules for the parameter of continuous one-parameter exponential family with contaminated data (errors in variables) are constructed by a deconvolution kernel method. The asymptotically optimal uniformly over a class of prior distributions and uniform rates of convergence, which depends on two types of the error distributions for the proposed EBT rules, are obtained under suitable conditions. Finally, an example about the main results of this paper is given.

Asymptotic normality of unbiased estimators in the case of an absolutely continuous multiparameter exponential family

Sufficient conditions are obtained for the asymptotic normality of unbiased estimators for the distribution density of a random vector with independent identically distributed components and for the linear integral functional of this density in the case of an exponential family of distributions. To analyze the limit behavior of unbiased estimators, multidimensional local limit theorems for density are used. Under certain regularity conditions, the limit law variance is demonstrated to coincide with the Cramer–Rao lower bound.

USE OF CONTINUOUS EXPONENTIAL FAMILIES TO LINK FORMS VIA ANCHOR TESTS

ABSTRACTContinuous exponential families are applied to linking test forms via an internal anchor. This application combines work on continuous exponential families for single‐group designs and work on continuous exponential families for equivalent‐group designs. Results are compared to those for kernel and equipercentile equating in the case of chained equating. The conversions produced by all methods are quite similar.

Robust empirical Bayes tests for continuous distributions

In this paper, we study the empirical Bayes two-action problem under linear loss function. Upper bounds on the regret of empirical Bayes testing rules are investigated. Previous results on this problem construct empirical Bayes tests using kernel type estimators of nonparametric functionals. Further, they have assumed specific forms, such as the continuous one-parameter exponential family for { F θ : θ ∈ Ω } , for the family of distributions of the observations. In this paper, we present a new general approach of establishing upper bounds (in terms of rate of convergence) of empirical Bayes tests for this problem. Our results are given for any family of continuous distributions and apply to empirical Bayes tests based on any type of nonparametric method of functional estimation. We show that our bounds are very sharp in the sense that they reduce to existing optimal or nearly optimal rates of convergence when applied to specific families of distributions.

LINKING WITH CONTINUOUS EXPONENTIAL FAMILIES: SINGLE‐GROUP DESIGNS

ABSTRACTContinuous exponential families are applied to linking forms via a single‐group design. In this application, a distribution from the continuous bivariate exponential family is used that has selected moments that match those of the bivariate distribution of scores on the forms to be linked. The selected continuous bivariate distribution then yields continuous univariate marginal distributions for the two forms. These marginal distributions then provide distribution functions and quantile functions that may be employed in equating. Normal approximations are obtained for the sample distributions of the conversion functions.

Testing for departures from nominal dispersion in generalized nonlinear models with varying dispersion and/or additive random effects

This paper discusses the tests for departures from nominal dispersion in the framework of generalized nonlinear models with varying dispersion and/or additive random effects. We consider two classes of exponential family distributions. The first is discrete exponential family distributions, such as Poisson, binomial, and negative binomial distributions. The second is continuous exponential family distributions, such as normal, gamma, and inverse Gaussian distributions. Correspondingly, we develop a unifying approach and propose several tests for testing for departures from nominal dispersion in two classes of generalized nonlinear models. The score test statistics are constructed and expressed in simple, easy to use, matrix formulas, so that the tests can easily be implemented using existing statistical software. The properties of test statistics are investigated through Monte Carlo simulations.

Simulation-assisted saddlepoint approximation

A general saddlepoint/Monte Carlo method to approximate (conditional) multivariate probabilities is presented. This method requires a tractable joint moment generating function (m.g.f.), but does not require a tractable distribution or density. The method is easy to program and has a third-order accuracy with respect to increasing sample size in contrast to standard asymptotic approximations which are typically only accurate to the first order. The method is most easily described in the context of a continuous regular exponential family. Here, inferences can be formulated as probabilities with respect to the joint density of the sufficient statistics or the conditional density of some sufficient statistics given the others. Analytical expressions for these densities are not generally available, and it is often not possible to simulate exactly from the conditional distributions to obtain a direct Monte Carlo approximation of the required integral. A solution to the first of these problems is to replace the intractable density by a highly accurate saddlepoint approximation. The second problem can be addressed via importance sampling, that is, an indirect Monte Carlo approximation involving simulation from a crude approximation to the true density. Asymptotic normality of the sufficient statistics suggests an obvious candidate for an importance distribution. The more general problem considers the computation of a joint probability for a subvector of random T, given its complementary subvector, when its distribution is intractable, but its joint m.g.f. is computable. For such settings, the distribution may be tilted, maintaining T as the sufficient statistic. Within this tilted family, the computation of such multivariate probabilities proceeds as described for the exponential family setting.

HEISENBERG–WEYL LIE ALGEBRA AND NATURAL EXPONENTIAL FAMILIES

We present in this work a specific construction of raising and lowering operators for 2-orthogonal quasi-monomial polynomials associated with continuous and discrete natural exponential families. We use these operators in order to characterize the real class of cubic natural exponential families.

A Unified Approach to Prior and Loss Robustness

ABSTRACT We introduce a new statistical framework in order to study Bayesian loss robustness under classes of priors distributions, thus unifying both concepts of robustness. We propose measures that capture variation with respect to both prior selection and selection of loss function and explore general properties of these measures. We illustrate the approach for the continuous exponential family. Robustness in this context is studied first with respect to prior selection where we consider several classes of priors for the parameter of interest, including unimodal and symmetric and unimodal with positive support. After prior variation has been measured we investigate robustness to loss function, using Hellinger and Linex (Linear Exponential) classes of loss functions. The methods are applied to standard examples.

Optimal rate of convergence of monotone empirical Bayes tests for normal means

This paper studies monotone empirical Bayes tests (MEBTs) for N ( θ , 1 ) under a linear loss. The purpose is to give a complete answer to an open problem raised by Karunamuni [1996. Optimal rates of convergence of empirical Bayes tests for the continuous one-parameter exponential family. Ann. Statist. 24, 212–231] and Liang [2000. On an empirical Bayes test for a normal mean. Ann. Statist. 28, 648–655] on the optimal rate of MEBTs. Through a novel construction of “hardest 2-point subproblems”, a lower bound rate O ( n - 1 ( ln n ) 1.5 ) is derived. This lower bound rate is shown to be achievable and therefore it is optimal.

Counting and locating the solutions of polynomial systems of maximum likelihood equations, I

In statistical inference, mixture models consisting of several component subpopulations are used widely to model data drawn from heterogeneous sources. In this paper, we consider maximum likelihood estimation for mixture models in which the only unknown parameters are the component proportions. By applying the theory of multivariable polynomial equations, we derive bounds for the number of isolated roots of the corresponding system of likelihood equations. If the component densities belong to certain familiar continuous exponential families, including the multivariate normal or gamma distributions, then our upper bound is, almost surely, the exact number of solutions.

On Measuring Loss Robustness Using Maximum A Posteriori Estimate

This paper investigates robustness of the loss function in a Bayesian framework. We incorporate maximum a posteriori (MAP) estimate, posterior expected loss (PEL) and introduce MAP range, in order to measure loss robustness under certain classes of loss functions, including Hellinger and LINEX loss functions. These classes of loss functions are considered for the estimation of the natural parameters of the continuous exponential family. The methods introduced are illustrated with some standard examples, including Normal and Gamma distributions. Finally, a comparison between MAP and PEL based loss robustness measures is presented to illustrate the importance of MAP estimation in measuring loss robustness.

Likelihood-Based Inference in Some Continuous Exponential Families With Unknown Threshold Parameters

We study likelihood-based inference in some continuous exponential families with unknown threshold parameters. The introduction of threshold parameters necessitates modification of the standard asymptotic arguments, and some possibly unexpected limiting distributions result.