Multiparameter Exponential Family Research Articles

Sequential detection of transient signal by moving likelihood ratio statistic in an exponential family

ABSTRACT We first consider the sequential detection of transient signals by generalizing the moving average chart to exponential family and study the false detection probability (FDP) and power of detection (POD) in the steady state. Then windowed adjusted signed (or modified directed) likelihood ratio chart is studied by treating it as approximate normal random variable. In the multi-parameter exponential family, the detection of the transient change of one of the canonical parameters or a function of canonical parameters is considered by using the generalized adjusted signed likelihood ratio chart. Comparisons with window restricted CUSUM and Shiryayev-Roberts (S-R) procedures show that the generalized signed likelihood ratio chart performs quite well. Several important examples including the mean or variance change under normal model and a real-time example are used for illustration.

A bivariate geometric distribution via conditional specification: properties and applications

In this article, we discuss a bivariate geometric distribution whose conditionals are geometric distributions and the marginals are not geometric and exhibits negative correlation. Several useful structural properties of the bivariate geometric distribution namely marginals, moments, generating functions, stochastic ordering are investigated. Simple proofs of negative correlation, marginal over-dispersion, distribution of sum, and distribution of the conditional given the sum are also derived. The distribution is shown to be a member of the multi-parameter exponential family and some natural but useful consequences are also outlined. A characterization via conditional failure rates is provided. An extensive simulation study is also carried out to explore the flexibility of the bivariate geometric distribution with various choices of the model parameters. Finally, the distribution is fitted to one bivariate count data set with an inherent negative correlation to illustrate its suitability.

A new bivariate Poisson distribution via conditional specification: properties and applications

ABSTRACT In this article, we discuss a bivariate Poisson distribution whose conditionals are univariate Poisson distributions and the marginals are not Poisson which exhibits negative correlation. Some useful structural properties of this distribution namely marginals, moments, generating functions, stochastic ordering are investigated. Simple proofs of negative correlation, marginal over-dispersion, distribution of sum and conditional given the sum are also derived. The distribution is shown to be a member of the multi-parameter exponential family and some natural but useful consequences are also outlined. Parameter estimation with maximum likelihood is implemented. Copula-based simulation experiments are carried out using Bivariate Normal and the Farlie–Gumbel–Morgenstern copulas to assess how the model behaves in dealing with the situation. Finally, the distribution is fitted to seven bivariate count data sets with an inherent negative correlation to illustrate suitability.

Estimating the error variance in a high-dimensional linear model

SummaryThe lasso has been studied extensively as a tool for estimating the coefficient vector in the high-dimensional linear model; however, considerably less is known about estimating the error variance in this context. In this paper, we propose the natural lasso estimator for the error variance, which maximizes a penalized likelihood objective. A key aspect of the natural lasso is that the likelihood is expressed in terms of the natural parameterization of the multi-parameter exponential family of a Gaussian with unknown mean and variance. The result is a remarkably simple estimator of the error variance with provably good performance in terms of mean squared error. These theoretical results do not require placing any assumptions on the design matrix or the true regression coefficients. We also propose a companion estimator, called the organic lasso, which theoretically does not require tuning of the regularization parameter. Both estimators do well empirically compared to pre-existing methods, especially in settings where successful recovery of the true support of the coefficient vector is hard. Finally, we show that existing methods can do well under fewer assumptions than previously known, thus providing a fuller story about the problem of estimating the error variance in high-dimensional linear models.

Detecting changes in a multiparameter exponential family by using adaptive CUSUM procedure

ABSTRACTAn adaptive cumulative sum (CUSUM) procedure is proposed to monitor parameter changes in a multiparameter exponential family where the change-point and postchange parameters are estimated adaptively. Approximations for average run lengths are derived. Monitoring changes in both mean and variance in the normal case is considered as an illustration. The conditional biases of the estimations for the change-point and postchange mean and variance is studied by simulation comparison with several other CUSUM procedures. An adaptive dam process by modifying the adaptive CUSUM process is used to detect and identify change points and change segments by using Citibank stock prices from 30 Dow Jones Industry Index.

Model selection for the segmentation of multiparameter exponential family distributions

We consider the segmentation problem of univariate distributions from the exponential fam- ily with multiple parameters. In segmentation, the choice of the number of segments remains a dicult issue due to the discrete nature of the change-points. In this general exponential family distribution framework, we propose a penalized log-likelihood estimator where the penalty is inspired by papers of L. Birg e and P. Massart. The resulting estimator is proved to satisfy an oracle inequality. We then further study the particular case of categorical variables by comparing the values of the key constants when derived from the specication of our general approach and when obtained by working directly with the characteristics of this distribution. Finally, a simulation study is conducted to assess the performance of our criterion for the exponential distribution, and an application on real data modeled by the categorical distribution is provided.

Homogeneity testing via weighted affinity in multiparameter exponential families

Based on stochastically independent samples with underlying density functions from the same multiparameter exponential family, a weighted version of Matusita’s affinity is applied as test statistic in a homogeneity test of identical densities as well as in a discrimination problem. Asymptotic distributions of the test statistics are stated, and the impact of weights on the deviation of actual and required type I error for finite sample sizes is examined in a simulation study.

Asymptotic normality of unbiased estimators in the case of an absolutely continuous multiparameter exponential family

Sufficient conditions are obtained for the asymptotic normality of unbiased estimators for the distribution density of a random vector with independent identically distributed components and for the linear integral functional of this density in the case of an exponential family of distributions. To analyze the limit behavior of unbiased estimators, multidimensional local limit theorems for density are used. Under certain regularity conditions, the limit law variance is demonstrated to coincide with the Cramer–Rao lower bound.

A simple Bayesian approach to multiple change-points

After a brief review of previous frequentist and Bayesian approaches to multiple change-points, we describe a Bayesian model for multiple parameter changes in a multiparameter exponential family. This model has attractive sta- tistical and computational properties and yields explicit recursive formulas for the Bayes estimates of the piecewise constant parameters. Efficient estimators of the hyperparameters of the Bayesian model for the parameter jumps can be used in conjunction, yielding empirical Bayes estimates. The empirical Bayes approach is also applied to solve long-standing frequentist problems such as significance testing of the null hypothesis of no change-points versus multiple change-point alterna- tives, and inference on the number and locations of change-points that partition the unknown parameter sequence into segments of equal values. Simulation studies of performance and an illustrative application to the British coal mine data are also given. Extensions from the exponential family to general parametric families and from independent observations to genearlized linear time series models are then provided.

Sequential Change-Point Detection When the Pre- and Post-Change Parameters are Unknown

We describe asymptotically optimal Bayesian and frequentist solutions to the problem of sequential change-point detection in multiparameter exponential families when the pre- and post-change parameters are unknown. In this connection we also address certain issues recently raised by Mei (2008) concerning performance criteria for detection rules in this setting.

Objective Bayes and conditional inference in exponential families

Objective Bayes methodology is considered for conditional frequentist inference about a canonical parameter in a multi-parameter exponential family. A condition is derived under which posterior Bayes quantiles match the conditional frequentist coverage to a higher-order approximation in terms of the sample size. This condition is on the model, not on the prior, and it ensures that any first-order probability matching prior in the unconditional sense automatically yields higher-order conditional probability matching. Objective Bayes methods are compared to parametric bootstrap and analytic methods for higher-order conditional frequentist inference.

Estimation procedures with delayed observations

The statistical model is considered in which the collection of data from several independent populations is available only at random times determined by order statistics of lifetimes of a given number of objects. Each of the populations is distributed according to a general multiparameter exponential family. The problem is to estimate the mean value vector parameter of the multiparameter exponential family of distributions of the forthcoming observations. Under the loss function involving a weighted squared error loss, the cost proportional to the events appeared and a cost of observing the process, a class of optimal sequential procedures is established. The procedures are derived in two situations: when the lifetime distribution is completely known and in the case when it is unknown but assumed to belong to an exponential subfamily with an unknown failure rate parameter.

A Bayesian Approach to Sequential Surveillance in Exponential Families

We describe herein a Bayesian change-point model and the associated recursive formulas for the estimated time-varying parameters and the posterior probability that a change-point has occurred at a particular time. The proposed model is a variant of that of Chernoff and Zacks (1964) for the case of normal means with known common variance. It considers more generally the multiparameter exponential family and addresses the complex statistical issues due to multiple change-points and unknown pre- and post-change system parameters in sequential surveillance. A sequential detection rule based on the proposed model is also introduced and its false alarm rate and mean detection delay are studied in the multiple change-point setting.

Region-Based Active Contours with Exponential Family Observations

In this paper, we focus on statistical region-based active contour models where image features (e.g. intensity) are random variables whose distribution belongs to some parametric family (e.g. exponential) rather than confining ourselves to the special Gaussian case. In the framework developed in this paper, we consider the general case of region-based terms involving functions of parametric probability densities, for which the anti-log-likelihood function is a special case. Using shape derivative tools, our effort focuses on constructing a general expression for the derivative of the energy (with respect to a domain), and on deriving the corresponding evolution speed. More precisely, we first show by an example that the estimator of the distribution parameters is crucial for the derived speed expression. On the one hand, when using the maximum likelihood (ML) estimator for these parameters, the evolution speed has a closed-form expression that depends simply on the probability density function. On the other hand, complicating additive terms appear when using other estimators, e.g. method of moments. We then proceed by stating a general result within the framework of multi-parameter exponential family. This result is specialized to the case of the anti-log-likelihood function with the ML estimator and to the case of the relative entropy. Experimental results on simulated data confirm our expectations that using the appropriate noise model leads to the best segmentation performance. We also report preliminary experiments on real life Synthetic Aperture Radar (SAR) images to demonstrate the potential applicability of our approach.

Generalized Likelihood Ratio Statistics and Uncertainty Adjustments in Efficient Adaptive Design of Clinical Trials

A new approach to adaptive design of clinical trials is proposed in a general multiparameter exponential family setting, based on generalized likelihood ratio statistics and optimal sequential testing theory. These designs are easy to implement, maintain the prescribed Type I error probability, and are asymptotically efficient. Practical issues involved in clinical trials allowing mid-course adaptation and the large literature on this subject are discussed, and comparisons between the proposed and existing designs are presented in extensive simulation studies of their finite-sample performance, measured in terms of the expected sample size and power functions.

Efficient adaptive designs with mid‐course sample size adjustment in clinical trials

Adaptive designs have been proposed for clinical trials in which the nuisance parameters or alternative of interest are unknown or likely to be misspecified before the trial. Although most previous works on adaptive designs and mid-course sample size re-estimation have focused on two-stage or group-sequential designs in the normal case, we consider here a new approach that involves at most three stages and is developed in the general framework of multiparameter exponential families. This approach not only maintains the prescribed type I error probability but also provides a simple but asymptotically efficient sequential test whose finite-sample performance, measured in terms of the expected sample size and power functions, is shown to be comparable to the optimal sequential design, determined by dynamic programming, in the simplified normal mean case with known variance and prespecified alternative, and superior to the existing two-stage designs and also to adaptive group-sequential designs when the alternative or nuisance parameters are unknown or misspecified.

IMPROVED ESTIMATORS OF THE NATURAL PARAMETERS IN CONTINUOUS MULTIPARAMETER EXPONENTIAL FAMILIES

ABSTRACT In this paper we first present general solutions to a differential inequality. These results include some of those of Berger, Ghosh and Parsian, and Ghosh, Hwang and Tsui[1], 6-7 as special cases. We then apply these results to the problem of estimating the natural parameter vector under a quadratic loss in continuous multiparameter exponential families. Also, we construct some classes of estimators dominating (linear functions of) the uniformly minimum variance unbiased estimator (UMVUE) or an estimator which is a known constant multiple of the UMVUE in cases of sampling from normal, gamma and generalized gamma distributions. Finally, in the case of sampling from normal distribution, we compare dominating estimators in each class with themselves in some special cases and demonstrate where our method can be failed in the case when the initial estimator is known to be admissible.

A computer algebra package for approximate conditional inference

This paper presents a set of REDUCE procedures that make a number of existing higher-order asymptotic results available for both theoretical and practical research. Attention has been restricted to the context of exact and approximate inference for a parameter of interest conditionally either on an ancillary statistic or on a statistic partially sufficient for the nuisance parameter. In particular, the procedures apply to regression-scale models and multiparameter exponential families. Most of them support algebraic computation as well as numerical calculation for a given data set. Examples illustrate the code.

Asymptotic approximations for error probabilities of sequential or fixed sample size tests in exponential families

Asymptotic approximations for the error probabilities of sequential tests of composite hypotheses in multiparameter exponential families are developed herein for a general class of test statistics, including generalized likelihood ratio statistics and other functions of the sufficient statistics. These results not only generalize previous approximations for Type I error probabilities of sequential generalized likelihood ratio tests, but also pro- vide a unified treatment of both sequential and fixed sample size tests and of Type I and Type II error probabilities. Geometric arguments involving integration over tubes play an important role in this unified theory.

Asymptotic Distribution of Estimated Affinity between Multiparameter Exponential Families

Let F1,...,F J be the distributions of J independent multiparameter exponential families, and ϱ J (F1,...,F J ) denote the affinity between F1,...,F J . We consider the problem of estimating ϱ J on the basis of independent random samples from these distributions. Subject to some mild regularity conditions, we derive the asymptotic distribution of the maximum likelihood estimator of ϱ J . Applications to hypothesis testing and discriminant analysis are discussed, and an example is provided.