One-parameter Exponential Family Research Articles

Robust estimation for general integer-valued autoregressive models based on the exponential-polynomial divergence

In this study, we develop a robust estimator for integer-valued one-parameter exponential family autoregressive models, named general integer-valued autoregressive models. This model accommodates a broad class of integer-valued time series models. In particular, we propose a robust estimation method that minimizes the exponential-polynomial divergence (EPD) belonging to the Brègman divergence family. EPD subsumes the density power divergence (DPD), which has been extensively studied by many authors for the past decades. Under regularity conditions, the minimum EPD estimator (MEPDE) is shown to be consistent and asymptotically normal. Comparing the performance of MEPDE with the minimum DPD estimator, we substantiate the validity of MEPDE through a simulation study and real data analysis.

Robust estimation for the one-parameter exponential family integer-valued GARCH(1,1) models based on a modified Tukey’s biweight function

Robust estimation for the one-parameter exponential family integer-valued GARCH(1,1) models based on a modified Tukey’s biweight function

Modeling and inference for multivariate time series of counts based on the INGARCH scheme

Modeling multivariate time series of counts using the integer-valued generalized autoregressive conditional heteroscedastic (INGARCH) scheme is proposed. The key idea is to model each component of the time series with a univariate INGARCH model, where the conditional distribution is modeled with a one-parameter exponential family distribution, and to use a (nonlinear) parametric function of all components to recursively produce the conditional means. It is shown that the proposed multivariate INGARCH (MINGARCH) model is strictly stationary and ergodic. For inference, the quasi-maximum likelihood estimator (QMLE) and the minimum density power divergence estimator (MDPDE) for robust estimation are adopted, and their consistency and asymptotic normality are verified. As an application, the change point test based on the QMLE and MDPDE is illustrated. The Monte Carlo simulation study and real data analysis using the number of weekly syphilis cases in the United States are conducted to confirm the validity of the proposed method.

Discussion of “Exponential-Family Embedding With Application to Cell Developmental Trajectories for Single-Cell RNA-seq Data”

Exponential-family singular value decomposition (eSVD) is a new approach for embedding multivariate data into a lower-dimensional space. It provides an elegant dimension reduction framework with flexibility to handle one-parameter exponential family distributions and proven consistency. This approach adds a valuable new tool to the toolbox of data analysts. Here we discuss a number of open problems and challenges that remain to be addressed in the future in order to unleash the full potential of eSVD and other similar approaches.

The adaptive Wynn algorithm in generalized linear models with univariate response

For a nonlinear regression model, the information matrices of designs depend on the parameter of the model. The adaptive Wynn algorithm for D-optimal design estimates the parameter at each step on the basis of the observed responses and employed design points so far, and selects the next design point as in the classical Wynn algorithm for D-optimal design. The name “Wynn algorithm” is in honor of Henry P. Wynn who established the latter “classical” algorithm in his 1970 paper (Ann. Math. Stat. 41 (1970) 1655–1664). The asymptotics of the sequences of designs and maximum likelihood estimates generated by the adaptive algorithm is studied for an important class of nonlinear regression models: generalized linear models whose (univariate) response variables follow a distribution from a one-parameter exponential family. Under the assumptions of compactness of the experimental region and of the parameter space together with some natural continuity assumptions, it is shown that the adaptive ML-estimators are strongly consistent and the design sequence is asymptotically locally D-optimal at the true parameter point. If the true parameter point is an interior point of the parameter space, then under some smoothness assumptions the asymptotic normality of the adaptive ML-estimators is obtained.

Modeling and inference for counts time series based on zero-inflated exponential family INGARCH models

This study considers statistical inferences such as parameter estimation and change tests for counts time series models where the conditional density of present observations over past information follow a zero-inflated one-parameter exponential family. We verify that the zero-inflated exponential family (ZIEF) INGARCH process is stationary and ergoic, and the maximum likelihood estimator is consistent and asymptotically normal. We then construct CUSUM tests based on (standardized) residuals and squares of (standardized) residuals as a parameter change test. Their null distributions is shown to converge to the sup of a Brownian bridge in distribution. The validity of the CUSUM tests is confirmed through a simulation study and real data analysis.

Minimax estimation of a restricted mean for a one-parameter exponential family

For one-parameter exponential families, we provide a unified minimax result for estimating the mean under weighted squared error losses in the presence of a lower-bound restriction. The finding recovers cases for which the result is known, as well as others which are new such as for a negative binomial model. We also study a related self-minimaxity property, obtaining several non-minimax results. Finally, for discrete models such as Poisson and negative binomial, we obtain classes of minimax estimators of a lower-bound restriction on the mean, which include range preserving solutions.

Robust Change Point Test for General Integer-Valued Time Series Models Based on Density Power Divergence.

In this study, we consider the problem of testing for a parameter change in general integer-valued time series models whose conditional distribution belongs to the one-parameter exponential family when the data are contaminated by outliers. In particular, we use a robust change point test based on density power divergence (DPD) as the objective function of the minimum density power divergence estimator (MDPDE). The results show that under regularity conditions, the limiting null distribution of the DPD-based test is a function of a Brownian bridge. Monte Carlo simulations are conducted to evaluate the performance of the proposed test and show that the test inherits the robust properties of the MDPDE and DPD. Lastly, we demonstrate the proposed test using a real data analysis of the return times of extreme events related to Goldman Sachs Group stock.

Predictively consistent prior effective sample sizes.

Determining the sample size of an experiment can be challenging, even more so when incorporating external information via a prior distribution. Such information is increasingly used to reduce the size of the control group in randomized clinical trials. Knowing the amount of prior information, expressed as an equivalent prior effective sample size (ESS), clearly facilitates trial designs. Various methods to obtain a prior's ESS have been proposed recently. They have been justified by the fact that they give the standard ESS for one-parameter exponential families. However, despite being based on similar information-based metrics, they may lead to surprisingly different ESS for nonconjugate settings, which complicates many designs with prior information. We show that current methods fail a basic predictive consistency criterion, which requires the expected posterior-predictive ESS for a sample of size N to be the sum of the prior ESS and N. The expected local-information-ratio ESS is introduced and shown to be predictively consistent. It corrects the ESS of current methods, as shown for normally distributed data with a heavy-tailed Student-t prior and exponential data with a generalized Gamma prior. Finally, two applications are discussed: the prior ESS for the control group derived from historical data and the posterior ESS for hierarchical subgroupanalyses.

Recovering Fisher-Information from the MGF Alone without Requiring Explicit PMF or PDF from a One-Parameter Exponential Family

It is well-known that a finite moment generating function (m.g.f.) corresponds to a unique probability distribution. So, an important question arises: Is it possible to obtain an expression of Fisher-information, IX(Ɵ); using the m.g.f. alone, that is without requiring explicitly a probability mass function (p.m.f.) or probability density function (p.d.f.), given that the p.m.f. or p.d.f came from a one-parameter exponential family? We revisit the core of statistical inference by developing a clear link (Theorem 1.1) between the m.g.f. and IX(Ɵ). Illustrations are included.

Nonparametric screening under conditional strictly convex loss for ultrahigh dimensional sparse data

Sure screening technique has been considered as a powerful tool to handle the ultrahigh dimensional variable selection problems, where the dimensionality $p$ and the sample size $n$ can satisfy the NP dimensionality $\log p=O(n^{a})$ for some $a>0$ [J. R. Stat. Soc. Ser. B. Stat. Methodol. 70 (2008) 849–911]. The current paper aims to simultaneously tackle the “universality” and “effectiveness” of sure screening procedures. For the “universality,” we develop a general and unified framework for nonparametric screening methods from a loss function perspective. Consider a loss function to measure the divergence of the response variable and the underlying nonparametric function of covariates. We newly propose a class of loss functions called conditional strictly convex loss, which contains, but is not limited to, negative log likelihood loss from one-parameter exponential families, exponential loss for binary classification and quantile regression loss. The sure screening property and model selection size control will be established within this class of loss functions. For the “effectiveness,” we focus on a goodness-of-fit nonparametric screening (Goffins) method under conditional strictly convex loss. Interestingly, we can achieve a better convergence probability of containing the true model compared with related literature. The superior performance of our proposed method has been further demonstrated by extensive simulation studies and some real scientific data example.

Robust estimation for general integer-valued time series models

In this study, we consider a robust estimation method for general integer-valued time series models whose conditional distribution belongs to the one-parameter exponential family. As a robust estimator, we employ the minimum density power divergence estimator, and we demonstrate this is strongly consistent and asymptotically normal under certain regularity conditions. A simulation study is carried out to evaluate the performance of the proposed estimator. A real data analysis using the return times of extreme events of the Goldman Sachs Group stock is also provided as an illustration.

Selection-Corrected Statistical Inference for Region Detection With High-Throughput Assays

ABSTRACTScientists use high-dimensional measurement assays to detect and prioritize regions of strong signal in spatially organized domain. Examples include finding methylation-enriched genomic regions using microarrays, and active cortical areas using brain-imaging. The most common procedure for detecting potential regions is to group neighboring sites where the signal passed a threshold. However, one needs to account for the selection bias induced by this procedure to avoid diminishing effects when generalizing to a population. This article introduces pin-down inference, a model and an inference framework that permit population inference for these detected regions. Pin-down inference provides nonasymptotic point and confidence interval estimators for the mean effect in the region that account for local selection bias. Our estimators accommodate nonstationary covariances that are typical of these data, allowing researchers to better compare regions of different sizes and correlation structures. Inference is provided within a conditional one-parameter exponential family per region, with truncations that match the selection constraints. A secondary screening-and-adjustment step allows pruning the set of detected regions, while controlling the false-coverage rate over the reported regions. We apply the method to genomic regions with differing DNA-methylation rates across tissue. Our method provides superior power compared to other conditional and nonparametric approaches. Supplementary materials for this article are available online.

CUSUM test for general nonlinear integer-valued GARCH models: comparison study

This study considers the problem of testing a parameter change in general nonlinear integer-valued time series models where the conditional distribution of current observations is assumed to follow a one-parameter exponential family. We consider score-, (standardized) residual-, and estimate-based CUSUM tests and show that their limiting null distributions take the form of the functions of Brownian bridges. Based on the obtained results, we then conduct a comparison study of the performance of CUSUM tests through the use of Monte Carlo simulations. Our findings demonstrate that the standardized residual-based CUSUM test largely outperforms the others.

The sample size required in importance sampling

The goal of importance sampling is to estimate the expected value of a given function with respect to a probability measure $\nu$ using a random sample of size $n$ drawn from a different probability measure $\mu$. If the two measures $\mu$ and $\nu$ are nearly singular with respect to each other, which is often the case in practice, the sample size required for accurate estimation is large. In this article, it is shown that in a fairly general setting, a sample of size approximately $\exp(D(\nu\parallel\mu))$ is necessary and sufficient for accurate estimation by importance sampling, where $D(\nu\parallel\mu)$ is the Kullback–Leibler divergence of $\mu$ from $\nu$. In particular, the required sample size exhibits a kind of cut-off in the logarithmic scale. The theory is applied to obtain a general formula for the sample size required in importance sampling for one-parameter exponential families (Gibbs measures).

Inference in Ising models

The Ising spin glass is a one-parameter exponential family model for binary data with quadratic sufficient statistic. In this paper, we show that given a single realization from this model, the maximum pseudolikelihood estimate (MPLE) of the natural parameter is $\sqrt{a_{N}}$-consistent at a point whenever the log-partition function has order $a_{N}$ in a neighborhood of that point. This gives consistency rates of the MPLE for ferromagnetic Ising models on general weighted graphs in all regimes, extending the results of Chatterjee (Ann. Statist. 35 (2007) 1931–1946) where only $\sqrt{N}$-consistency of the MPLE was shown. It is also shown that consistent testing, and hence estimation, is impossible in the high temperature phase in ferromagnetic Ising models on a converging sequence of simple graphs, which include the Curie–Weiss model. In this regime, the sufficient statistic is distributed as a weighted sum of independent $\chi^{2}_{1}$ random variables, and the asymptotic power of the most powerful test is determined. We also illustrate applications of our results on synthetic and real-world network data.

Application of Unbiased Estimators to Group Classification Risk Estimation

We consider the problem of construction of point and interval estimators for the Bayesian risk of a group classification decision rule provided that the elements of training samples have distributions belonging to the same one-parameter exponential family. We propose a solution to this problem using the asymptotic normality and efficiency of an unbiased risk estimator. An example of application of the obtained theoretical results is given.

Maximum likelihood estimator of the parameter for a continuous one-parameter exponential family under the optimal ranked set sampling

This paper studies a maximum likelihood estimator (MLE) of the parameter for a continuous one-parameter exponential family under ranked set sampling (RSS). The authors first find the optimal RSS according to the character of the family, viz, arrange the RSS based on quasi complete and sufficient statistic of independent and identically distributed (iid) samples. Then under this RSS, some sufficient conditions for the existence and uniqueness of the MLE, which are easily used in practice, are obtained. Using these conditions, the existence and uniqueness of the MLEs of the parameters for some usual distributions in this family are proved. Numerical simulations for these distributions fully support the result from the above two step optimizations of the sampling and the estimation method.

Special issue in honor of Kathryn Chaloner

We are honored to be guest editors for this special issue of Applied Stochastic Models in Business and Industry, dedicated to Kathryn Chaloner's life and achievements. Contributions from her colleagues and friends in this volume are devoted to Kathryn's early research interest in Bayesian optimal design. She was a pioneer in this field and had a broad impact on the subsequent literature on experimental design. Kathryn was internationally known for her research in Bayesian Statistics, which included design of experiments, outlier detection, prior elicitation, and clinical trials, and for her advancements in the study of HIV/AIDS. She was an accomplished researcher, teacher, and mentor. She actively encouraged women and under-represented minority students to join the field of biostatistics and provided mentorship, support, and encouragement. The special issue contains seven papers. The first three papers present some of the recent work in Bayesian optimal designs for nonlinear models, binary responses, and experiments with adversarial components. This is followed by two articles on Bayesian designs in clinical trials. The last two articles of the special issue involve engineering design problems. The paper by Giovagnoli considers the binary response models of Chaloner and Larntz (1989, Journal of Statistical Planning and Inference, 21:191–208) and develops an adaptive version of A-optimal Bayesian designs that are revised as additional data become available during the experiment. Adaptive Bayesian compound designs is also discussed as another extension. Konstantinou and Dette develop approximate Bayesian D-optimal designs for nonlinear regression models that involve covariates that cannot be observed directly. Applications to specific nonlinear models including the exponential regression model are considered and characterizations of the D-optimal saturated designs are presented. Optimal sample size selection for experiments is considered by De Santis and Gubbiotti in an adversarial setting where the decision makers have different prior opinions while having common utility functions and data. Using a Bayesian decision approach with a quadratic loss function, the authors present results for the one-parameter exponential family and investigate effect of the priors on the optimal sample size. Müller, Xu, and Thall discuss advantages and limitations of Bayesian decision theoretic approach in design of clinical trials. In doing so, the authors present a specific case study and point out the limitations such as the computational difficulties involved in solving sequential problems as well as selection of utility functions capturing different stakeholder interests. The article by Ventz, Parmigiani, and Trippa also consider limitations of the Bayesian framework in design of clinical trials and propose a strategy to alleviate these. The proposed strategy involves combining Bayesian designs with frequentist metrics such as error rates, confidence intervals, etc. that are commonly used by medical community and regulatory agencies. The authors show that this can be achieved by an inclusion of frequentist constraints into the Bayesian formulation. The constrained decision theoretic framework is illustrated via several applications and computational algorithms are presented for implementing the proposed approach. Nakamura, Seepaul, Kadane, and Reeja-Jayan discuss design of experiments in materials science. The authors consider use of a batch-sequential experiment and a regression model to determine optimal levels of input variables for synthesis of ceramic materials. It is illustrated that the optimal settings produce more reliable results than other experiments. The final article by Polson and Soyer considers Bayesian design of accelerated life tests for reliability assessment. An augmented probability simulation (APS) approach using Lindley's (1976, Annals of Statistics, 4:1–10) conjugate utility functions is proposed to compute optimal designs in an efficient manner. The proposed approach is illustrated using single and multiple point designs. Use of particle based methods for APS is also discussed as an alternative to Markov chain Monte Carlo methods.

Distribution-free detection of a submatrix

We consider the problem of detecting the presence of a submatrix with larger-than-usual values in a large data matrix. This problem was considered in Butucea and Ingster (2013) under a one-parameter exponential family, and one of the procedures they analyzed is the scan test. Taking a nonparametric stance, we show that a calibration by permutation leads to the same (first-order) asymptotic performance. This is true for the two types of permutations we consider. We also study the corresponding rank-based variants and quantify precisely the loss in asymptotic power.