Discrete Exponential Family Research Articles

Estimating Mixing Densities in Exponential Family Models for Discrete Variables

This paper is concerned with estimating a mixing density gusing a random sample from the mixture distribution f(x)=∫f x | θ)g(θ)dθ where f(· | θ)is a known discrete exponen tial family of density functions. Recently two techniques for estimatingghave been proposed. The first uses Fourier analysis and the method of kernels and the second uses orthogonal polynomials. It is known that the first technique is capable of yielding estimators that achieve (or almost achieve) the minimax convergence rate. We show that this is true for the technique based on orthogonal polynomials as well. The practical implementation of these estimators is also addressed. Computer experiments indicate that the kernel estimators give somewhat disappoint ing finite sample results. However, the orthogonal polynomial estimators appear to do much better. To improve on the finite sample performance of the orthogonal polynomial estimators, a way of estimating the optimal truncation parameter is proposed. The resultant estimators retain the convergence rates of the previous estimators and a Monte Carlo finite sample study reveals that they perform well relative to the ones based on the optimal truncation parameter.

A characterization of certain discrete exponential families

For independent random variables X and Y, define % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaiaabofaruWrL9MCNLwyaGqbciaa-bcacqGHHjIUcaWFGaGaa8hw% aiaa-TcacaWFzbaaaa!4551!\[{\rm{S}} \equiv X + Y\]. When the conditional expectations % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaiaadweacaGGBbqefCuzVj3zPfgaiuGajaaqcaWFNbGccaGGOaGa% amiwaiaacMcacaGG8bGaam4uaiaac2facqGHHjIUcaWGHbGaaiikai% aadofacaGGPaaaaa!4BC4!\[E[g(X)|S] \equiv a(S)\]and % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaiaadweacaGGBbGaamiAaiaacIcacaWGybGaaiykaiaacYhacaWG% tbGaaiyxaiabggMi6kaadkgacaGGOaGaam4uaiaacMcaaaa!4894!\[E[h(X)|S] \equiv b(S)\]are given, then under certain assumptions, the density function of X has the form of u(x)k(α)eax, where u(x) is uniquely determined by the functions a(·) and b(·).

A condition for the nonexistence of ancillary statistics

This note gives a simple condition for the nonexistence of nontrivial ancillary statistics. The result shows that in sampling from a discrete exponential family with a regression model, usually there is no nontrivial ancillary statistic.

On Estimating Mixing Densities in Discrete Exponential Family Models

This paper concerns estimating a mixing density function $g$ and its derivatives based on iid observations from $f(x) = \int f(x \mid \theta)g(\theta)d\theta$, where $f(x \mid \theta)$ is a known exponential family of density functions with respect to the counting measure on the set of nonnegative integers. Fourier methods are used to derive kernel estimators, upper bounds for their rate of convergence and lower bounds for the optimal rate of convergence. If $f(x \mid \theta_0) \geq \varepsilon^{x + 1} \forall x$, for some positive numbers $\theta_0$ and $\varepsilon$, then our estimators achieve the optimal rate of convergence $(\log n)^{-\alpha + m}$ for estimating the $m$th derivative of $g$ under a Lipschitz condition of order $\alpha > m$. The optimal rate of convergence is almost achieved when $(x!)^\beta f(x \mid \theta_0) \geq \varepsilon^{x + 1}$. Estimation of the mixing distribution function is also considered.

Lower Bounds for Contamination Bias: Globally Minimax Versus Locally Linear Estimation

We study how robust estimators can be in parametric families, obtaining a lower bound on the contamination bias of an estimator that holds for a wide class of parametric families. This lower bound includes as a special case the bound used to establish that the median is bias minimax among location equivariant estimators, and it is tight or nearly tight in a variety of other settings such as scale estimation, discrete exponential families and multiple linear regression. The minimum variation distance estimator has contamination bias within a dimension-free factor of this bound. A second lower bound applies to locally linear estimates and implies that such estimates cannot be bias minimax among all Fisher-consistent estimates in higher dimensions. In linear regression this class of estimates includes the familiar M-estimates, GM-estimates and S-estimates. In discrete exponential families, yet another lower bound implies that the "proportion of zeros" estimates has minimax bias if the median of the distribution is zero, a common situation in some fields. This bound also implies that the information-standardized sensitivity of every Fisher consistent estimate of the Poisson mean and of the Binomial proportion is unbounded.

Statistical inference for multiple choice tests

Finite sample inference procedures are considered for analyzing the observed scores on a multiple choice test with several items, where, for example, the items are dissimilar, or the item responses are correlated. A discrete p-parameter exponential family model leads to a generalized linear model framework and, in a special case, a convenient regression of true score upon observed score. Techniques based upon the likelihood function, Akaike's information criteria (AIC), an approximate Bayesian marginalization procedure based on conditional maximization (BCM), and simulations for exact posterior densities (importance sampling) are used to facilitate finite sample investigations of the average true score, individual true scores, and various probabilities of interest. A simulation study suggests that, when the examinees come from two different populations, the exponential family can adequately generalize Duncan's beta-binomial model. Extensions to regression models, the classical test theory model, and empirical Bayes estimation problems are mentioned. The Duncan, Keats, and Matsumura data sets are used to illustrate potential advantages and flexibility of the exponential family model, and the BCM technique.

Bayesian Marginal Inference

Abstract A method is proposed for approximating the marginal posterior density of a continuous function of several unknown parameters, thus permitting inferences about any parameter of interest for nonlinear models when the sample size is finite. Possibly tedious numerical integrations are replaced by conditional maximizations, which are shown to be quite accurate in a number of special cases. There are similarities with the profile likelihood ideas originated by Kalbfleisch and Sprott (1970), and the method is contrasted with a Laplacian approximation recommended by Kass, Tierney, and Kadane (1988, in press), referred to here as the “KTK procedure.” The methods are used to approximate the marginal posterior densities of the log-linear interaction effects and an overall measure of association in a two-way contingency table. Snee's (1974) hair/eye color data are reanalyzed, and adjustments are proposed to Goodman's (1964) analysis for the full-rank interaction model. Another application concerns marginalization problems for a discrete p-parameter exponential family distribution, and inferences are considered for the probability of a zero count.

Bayes empirical Bayes estimation for discrete exponential families

Bayes-empiric Bayes estimation of the parameter of certain one parameter discrete exponential families based on orthogonal poly- nomials on an interval (a, b) is introduced. The resulting estimator is shown to be asymptotically optimal. The application of this method to three special distributions, the binomial, Poisson and negative binomial, is discussed.

Minimum variance unbiased estimation for the lagrange power series distributions

It is shown that the modified power series distribution (MPSD) [GUPTA, 1974] the Lagrange probability distribution (LPD) [CONSUL SHENTON, 1972, the LAGBANGE power series distribution (LPSD) [JAIN, 1975] and the discrete exponential family (DEE) distribution [JANARDAN, 1982) represent the same probability model. A theorem on the MVU estimation of a function (θ) of the parameter θ in the LPSD is stated with the necessary conditions. The theorem is then applied to derive the MVU estimators for the mean, square of the mean and the variance of the generalized POISSON .distribution, generalized negative binomial distribution and some other related distributions

On the Convergence Rates of Empirical Bayes Rules for Two-Action Problems: Discrete Case

The purpose of this paper is to investigate the convergence rates of a sequence of empirical Bayes decision rules for the two-action decision problems where the distributions of the observations belong to a discrete exponential family. It is found that the sequence of the empirical Bayes decision rules under study is asymptotically optimal, and the order of associated convergence rates is $O(\exp(-cn))$, for some positive constant $c$, where $n$ is the number of accumulated past experience (observations) at hand. Two examples are provided to illustrate the performance of the proposed empirical Bayes decision rules. A comparison is also made between the proposed empirical Bayes rules and some earlier existing empirical Bayes rules.

Admissibility of goodness of fit tests for discrete exponential families

Consider testing the composite hypothesis that a population is a particular discrete exponential family. For example, a Poisson distribution with unknown parameter. We find a sufficient condition for the admissibility of goodness of fit tests. Included in the class of admissible tests is the usual chi — square test of goodness of fit.

Discussion of Professor Copas's Paper

Discussion of Professor Copas's Paper

Construction of Improved Estimators in Multiparameter Estimation for Discrete Exponential Families

This paper extends and unifies the theory of simultaneous estimation for the discrete exponential family. We discuss construction of estimators which theoretically dominate the uniformly minimum variance unbiased estimator (UMVUE) under a weighted squared error loss function, and show by means of computer simulation results that new simultaneous Poisson means estimators perform more favorably than those previously proposed. Our improved estimators shift the UMVUE towards a possibly nonzero point or a data-based point.

Multivariate discrete exponential family of distributions and their properties

The paper generalizes the univariate discrete exponential family of distributions to the multivariate situation, and this generalization includes the multivariate power series distributions, the multivariate Lagrangian distributions, and the modified multivariate power-series distributions. This provides a unified approach for the study of these three classes of distributions. We obtain recurrence relations for moments and cumulants, and the maximum likelihood estimation for the discrete exponential family. These results are applied to some multivariate discrete distributions like the Lagrangian Poisson, Lagrangian (negative) multinomial, logarithmic series distributions and multivariate Lagrangian negative binomial distribution.

Some Applications of the Family of Discrete Exponential Distributions in Biology and Toxicology

In this paper the relationship between the number of males and females in a community, between the number of boys and girls in a family, between the number of animals trapped and untrapped in an animal experiment, between the number of infected and uninfected in an epidemiological study, between the number of mutagens and non‐mutagens in a given sample of toxicological experiments, and others, is studied. The general expression for the correlation coefficient, ϱ, is obtained when N, the community size N, has a discrete exponential family distribution recently studied by Janardan (1980). This is then specialized to some well known discrete distributions.

On truncation of multiparameter estimator in discrete exponential families

Let X1,…Xpbe independent random variables, where each of the p distributions of the random variables belongs to the family of one parameter discrete exponential distributions. The problem is to estimate the unknown parameters simultaneously in the pres¬ence of extreme observations. Hudson (1978) and Tsui (1979) show that the minimum variance unbiased estimator (MVUE) of the param¬eters is inadmissible if p⩾3 under squared error loss, and esti¬mators better than MVUE are proposed. In this paper, a truncated version of the shrinkage estimator is proposed, in the case of discrete exponential family. This estimator is based on ordered observations which improves the MVUE.

Semi Tail Upper Bounds on the Class of Admissible Estimators in Discrete Exponential Families with Applications to Poisson and Negative Binomial Distributions

Admissibility problems involving simultaneous estimation in discrete exponential families are studied by solving difference inequalities. It is shown that if an estimator is admissible under the loss function $L_m(\mathbf{\theta, a)} = \sum^p_{i = 1} \theta^{m_i}_i (\theta_i - a_i)^2$, then in the tail (i.e., for large values of the observations), this estimator has to be less than certain bounds. Specific bounds, called Semi Tail Upper Bounds (STUB), are given here. These STUBs are not only of theoretical interest, but also are sharp enough that they establish many new results. Two of the most interesting ones are: (i) the establishment of Brown's conjecture concerning inadmissibility of some of the estimators proposed by Clevenson and Zidek (1975), and (ii) the establishment of inadmissibility of Hudson's (1978) estimator which improves upon the uniformly minimum variance unbiased estimator in Negative Binomial families.

Improving Upon Standard Estimators in Discrete Exponential Families with Applications to Poisson and Negative Binomial Cases

Assume that $X_1, \cdots, X_p$ are independent random observations having discrete exponential densities $\rho_i(\theta_i)t_i(x_i)\theta^{xi}_i, i = 1, \cdots, p$ respectively. A general technique of improving upon the uniform minimum variance unbiased estimator (UMVUE) of $(\theta_1, \cdots, \theta_p)$ is developed under possibly weighted squared error loss functions. It is shown that improved estimators can be constructed by solving a difference inequality. Typical difference inequalities of a fairly general type are presented and solved. When specialized to Poisson and Negative binomial cases, broad classes of estimators are given that dominate the UMVUE. These results unify many known results in this rapidly diverging field, and some of them are new (especially those related to Negative Binomial distributions). Improved estimators are also obtained for the problems in which some of the observations are from Poisson families and some from Negative Binomial families. For sum of squared errors loss, estimators which dominate the UMVUE in the discrete exponential families are also given explicitly.

A New Discrete Exponential Family of Distributions: Properties and Application to Power Series Distributions

SYNOPTIC ABSTRACTThis paper introduces a discrete exponential family distribution which includes the important classes of series distributions - generalized power series distributions, modified power series distributions, and power series distributions associated with the Lagrange expansion. This provides a unified approach for the study of these three classes of power series distributions. After discussing moments, recurrsion relations, and maximum likelihood estimation for the discrete exponential family, the results are applied to certain well-known discrete distributions like the generalized Poisson, negative binomial and logarithmic series distributions, and the lost games distribution. A new discrete distribution termed linear function binomial distribution is introduced and its properties studied.

A modified power series distribution

A modification of the usual power series distribution is proposed and a statistical treatment of data generated by this probability model is indicated. It is shown that this modified power series distribution belongs to a multiparameter discrete exponential family and standard theory is then applied to obtain complete sufficient statistics of the parameters, sampling distribution of these statistics, maximum likelihood and minimum variance unbiased estimators, uniformly most powerful unbiased tests, uniformly most accurate confidence intervals and expected cover tolerance regions.