Zero-inflated Poisson Distribution Research Articles

Type I multivariate zero-inflated Poisson distribution with applications

Motivated from the stochastic representation of the univariate zero-inflated Poisson (ZIP) random variable, the authors propose a multivariate ZIP distribution, called as Type I multivariate ZIP distribution, to model correlated multivariate count data with extra zeros. The distributional theory and associated properties are developed. Maximum likelihood estimates for parameters of interest are obtained by Fisher’s scoring algorithm and the expectation–maximization (EM) algorithm, respectively. Asymptotic and bootstrap confidence intervals of parameters are provided. Likelihood ratio test and score test are derived and are compared via simulation studies. Bayesian methods are also presented if prior information on parameters is available. Two real data sets are used to illustrate the proposed methods. Under both AIC and BIC, our analysis of the two data sets supports the Type I multivariate zero-inflated Poisson model as a much less complex alternative with feasibility to the existing multivariate ZIP models proposed by Li et al. (Technometrics, 29–38, Vol 41, 1999).

A spatially-explicit count data regression for modeling the density of forest cockchafer (Melolontha hippocastani) larvae in the Hessian Ried (Germany)

In this paper, a regression model for predicting the spatial distribution of forest cockchafer larvae in the Hessian Ried region (Germany) is presented. The forest cockchafer, a native biotic pest, is a major cause of damage in forests in this region particularly during the regeneration phase. The model developed in this study is based on a systematic sample inventory of forest cockchafer larvae by excavation across the Hessian Ried. These forest cockchafer larvae data were characterized by excess zeros and overdispersion. Using specific generalized additive regression models, different discrete distributions, including the Poisson, negative binomial and zero-inflated Poisson distributions, were compared. The methodology employed allowed the simultaneous estimation of non-linear model effects of causal covariates and, to account for spatial autocorrelation, of a 2-dimensional spatial trend function. In the validation of the models, both the Akaike information criterion (AIC) and more detailed graphical procedures based on randomized quantile residuals were used. The negative binomial distribution was superior to the Poisson and the zero-inflated Poisson distributions, providing a near perfect fit to the data, which was proven in an extensive validation process. The causal predictors found to affect the density of larvae significantly were distance to water table and percentage of pure clay layer in the soil to a depth of 1 m. Model predictions showed that larva density increased with an increase in distance to the water table up to almost 4 m, after which it remained constant, and with a reduction in the percentage of pure clay layer. However this latter correlation was weak and requires further investigation. The 2-dimensional trend function indicated a strong spatial effect, and thus explained by far the highest proportion of variation in larva density. As such the model can be used to support forest practitioners in their decision making for regeneration and forest protection planning in the Hessian Ried. However, the application of the model for predicting future spatial patterns of the larva density is still somewhat limited because the causal effects are comparatively weak.

Selection of single sampling plans by attributes under the conditions of zero-inflated Poisson distribution

Purpose– The purpose of this paper is to determine single sampling plans (SSPs) by attributes when the number of nonconformities is distributed according to a zero-inflated Poisson (ZIP) distribution.Design/methodology/approach– Manufacturing processes have now-a-days been aligned properly and are monitored well, so that the occurrence of nonconformities would be a rare phenomenon. The information related to number of nonconformities per product will have more number of zeros. Under such circumstances, the appropriate probability distribution of the number of nonconformities is a ZIP distribution. The operating characteristic function of the sampling plan is derived.Findings– Parameters of the sampling plans are obtained for some sets of values of (p1,α,p2,β). Numerical examples are given to illustrate the selection of SSPs under ZIP distribution and to study its advantages over Poisson SSP.Originality/value– Results obtained in this paper are original and has been done for the first time in this regard. Parameters of the sampling plans are essential to make decisions either to accept or reject the lots based on the inspection of the samples.

First-order mixed integer-valued autoregressive processes with zero-inflated generalized power series innovations

To model zero-inflated time series of counts, we propose a first-order mixed integer-valued autoregressive process with zero-inflated generalized power series innovations. These innovations contain the commonly used zero-inflated Poisson and geometric distributions. Strict stationarity, ergodicity of the process, and some important probabilistic properties such as the transition probabilities, the k-step ahead conditional mean and variance are obtained. The conditional maximum likelihood estimators for the parameters in this process are derived and the performances of the estimators are studied via simulation. As illustration, an application to an offence data set is given to show the effectiveness of the proposed model.

A Combination of CUSUM Charts for Monitoring a Zero-Inflated Poisson Process

The Zero-inflated Poisson distribution (ZIP) is used to model the defects in processes with a large number of zeros. We propose a control charting procedure using a combination of two cumulative sum (CUSUM) charts to detect increases in the parameters of ZIP process, one is a conforming run length (CRL) CUSUM chart and another is a zero truncated Poisson (ZTP) CUSUM chart. The control limits of the control charts are obtained using both Markov chain-based methods and simulations. Simulation experiments show that the proposed method outperforms an existing method. Finally, a real example is presented.

Objective Bayes analysis of zero-inflated Poisson distribution with application to healthcare data

In this article, non-informative priors are investigated for a zero-inflated Poisson distribution with two parameters: the probability of zeros and the mean of the Poisson part. Both the reference prior and the Jeffreys prior are derived and shown to be second-order matching priors when only the mean of the Poisson part is of interest. However, when the probability of zeros is of interest, the reference prior is still a second-order matching prior, whereas the Jeffreys prior is not so. Furthermore, when both parameters are of interest, the reference prior is a unique second-order matching prior. Frequentist coverage probabilities of the posterior confidence sets based on the Jeffreys and reference priors are compared with each other using Monte Carlo simulations and with confidence sets based on the maximum likelihood estimation.

Random Effects Modeling and the Zero-Inflated Poisson Distribution

Overdispersion due to a large proportion of zero observations in data sets is a common occurrence in many applications of many fields of research; we consider such scenarios in count panel (longitudinal) data. A well-known and widely implemented technique for handling such data is that of random effects modeling, which addresses the serial correlation inherent in panel data, as well as overdispersion. To deal with the excess zeros, a zero-inflated Poisson distribution has come to be canonical, which relaxes the equal mean-variance specification of a traditional Poisson model and allows for the larger variance characteristic of overdispersed data. A natural proposal then to approach count panel data with overdispersion due to excess zeros is to combine these two methodologies, deriving a likelihood from the resulting conditional probability. In performing simulation studies, we find that this approach in fact poses problems of identifiability. In this article, we construct and explain in full detail why a model obtained from the marriage of two classical and well-established techniques is unidentifiable and provide results of simulation studies demonstrating this effect. A discussion on alternative methodologies to resolve the problem is provided in the conclusion.

Análise de frequência de máximos anuais baseada em séries de duração parcial. Combinação das distribuições de Poisson inflacionada de zeros e generalizada de Pareto, modelo ZIP-GP

In frequency analysis of hydrological extremes under a peaks-over-threshold (PoT) approach, the model with Poisson arrival counts and Generalized Pareto (GP) distributed exceedances – Poisson-GP model is widely used. Such model requires the validation of the hypothesis that the distribution of the annual number of extreme events may be described by a Poisson distribution (Poisson hypothesis), which is not always verifiable in practical terms. The present study concerns the use of an alternative distribution for modeling the annual number of hydrological extremes the Zero-Inflated Poisson (ZIP) distribution with two parameters. A ZIP-GP model for extreme frequency analysis is proposed. This model is less restrictive than the Poisson-GP model since it allows for a more accurate description of the occurrence process in a PoT framework if the fraction of years with no exceedances is significantly higher than the theoretical mass at zero of the Poisson distribution. An application of the ZIP-GP model to flood data from Northern Portugal and the evaluation of its performance relative to the Poisson-GP model is presented. The results show that the ZIP-GP model outperforms the Poisson-GP model, especially for lower quantiles, hence the ZIP distribution is a valid alternative to the Poisson distribution for modeling the annual occurrence counts of peaks in a PoT approach for flood frequency analysis.

On peaks-over-threshold modeling of floods with zero-inflated Poisson arrivals under stationarity and nonstationarity

The peaks-over-threshold (POT) model with Poisson arrivals and generalized Pareto (GP) distributed exceedances remains a popular and useful tool for modelling hydrologic extremes. The use of the Poisson–GP model for flood frequency analysis requires the validation of the hypothesis that the distribution of the annual number of flood events may be described by a Poisson distribution. Such hypothesis is not always valid in practical applications. The present study concerns the use of an alternative distribution for modelling the annual number of floods—the zero-inflated Poisson (ZIP) distribution with two parameters. A ZIP–GP model for flood frequency analysis is proposed. This model is less restrictive than the Poisson–GP model since it allows for a more accurate description of the occurrence process in a POT framework if the fraction of years with no exceedances is significantly higher than the theoretical mass at zero of the Poisson distribution. Furthermore, a nonstationary model (NSZIP–GP) is presented, in which the parameters of the ZIP are allowed to change in time as a function of a covariate, which, even for stationary peak magnitudes, affects the annual maximum flood quantiles with a given non-exceedance probability. Applications of the ZIP–GP model to flood data from Northern Portugal and the evaluation of its performance relative to the Poisson–GP model, including assessments of quantile uncertainty, are presented. An illustrative application of the NSZIP–GP model, using the North Atlantic Oscillation as a covariate is also presented. The applications of both models include assessment of quantile uncertainty.

A Note on a Stata Plugin for Estimating Group-based Trajectory Models

Group-based trajectory models are used to investigate population differences in the developmental courses of behaviors or outcomes. This note introduces a new Stata command, traj, for fitting to longitudinal data finite (discrete) mixture models designed to identify clusters of individuals following similar progressions of some behavior or outcome over age or time. Normal, Censored normal, Poisson, Zero-inflated Poisson, and Logistic distributions are supported.

Determination of Single Sampling Plans by Attributes Under the Conditions of Zero-Inflated Poisson Distribution

When the manufacturing process is well monitored, occurrence of nondefects would be a frequent event in sampling inspection. The appropriate probability distribution of the number of defects is a zero-inflated Poisson (ZIP) distribution. In this article, determination of single sampling plans (SSPs) by attributes using unity values is considered, when the number of defects follows a ZIP distribution. The operating characteristic (OC) function of the sampling plan is derived. Plan parameters are obtained for some sets of values of (p1, α, p2, β). Numerical illustrations are given to describe the determination of SSP under ZIP distribution and to study its performance in comparison with Poisson SSP.

Zero-inflated and overdispersed: what's one to do?

Zero-inflated Poisson (ZIP) and zero-inflated negative binomial (ZINB) models are recommended for handling excessive zeros in count data. For various reasons, researchers may not address zero inflation. This paper helps educate researchers on (1) the importance of accounting for zero inflation and (2) the consequences of misspecifying the statistical model. Using simulations, we found that when the zero inflation in the data was ignored, estimation was poor and statistically significant findings were missed. When overdispersion within the zero-inflated data was ignored, poor estimation and inflated Type I errors resulted. Recommendations on when to use the ZINB and ZIP models are provided. In an illustration using a two-step model selection procedure (likelihood ratio test and the Vuong test), the ZIP model was correctly identified only when the distributions had moderate means and sample sizes and did not correctly identify the ZINB model or the zero inflation in the ZIP and ZINB distributions.

Multivariate models for correlated count data

In this study, we deal with the problem of overdispersion beyond extra zeros for a collection of counts that can be correlated. Poisson, negative binomial, zero-inflated Poisson and zero-inflated negative binomial distributions have been considered. First, we propose a multivariate count model in which all counts follow the same distribution and are correlated. Then we extend this model in a sense that correlated counts may follow different distributions. To accommodate correlation among counts, we have considered correlated random effects for each individual in the mean structure, thus inducing dependency among common observations to an individual. The method is applied to real data to investigate variation in food resources use in a species of marsupial in a locality of the Brazilian Cerrado biome.

Effectiveness of sampling plans by attributes based on mixture distributions characterising microbial clustering in food

As in many cases, pathogenic microorganisms contaminate the food material as clusters or group of individual cells; the effectiveness of sampling plans based on mixture distributions representing bacterial agglomeration was assessed. In general, sampling plans that do not take into account such consideration lead to higher probabilities of accepting defective lots. Since quite often no scientific data are available in order to determine the degree of over-dispersion or clustering of the target microorganisms, in this theoretical study we compare the variance-to-mean ratio and the reciprocal of the exponent k of the negative binomial distribution (NB) as measures of dispersion. The mixture Poisson-logarithmic (Plog) model is proposed as a special case of the NB distribution, where the bacterial clusters are Poisson distributed while the individuals in each cluster follow a logarithmic distribution. In order to describe microbial data characterised by an excess of zero counts (1−π), we assess the zero-inflated Poisson (ZIP) and zero-inflated negative binomial (ZINB) distributions as alternative statistical models. The Operating Characteristic (OC) curves generated on the basis of the zero-inflated distributions were compared for fixed values of the variance-to-mean ratio and the parameter π at any mean level of contamination and sample weight adopted. The results show that assuming fixed 1/k and π for the NB and ZIP distributions, respectively, both models converge to a Poisson distribution at the producer's quality level. In contrast, the consumer's quality level is highly affected by assuming fixed values of 1/k and π since it increases. The OC curves generated for the NB and ZIP distributions assuming fixed values of the variance-to-mean ratio at any mean level of contamination and sample weight adopted, reveal that both the consumer's and producer's quality level are affected, as they both increase. Within the ZINB distribution, a separate investigation is conducted to determine which parameters are mostly responsible for describing microbial over-dispersion. As a general conclusion, for the design of sampling plans based on any statistical distribution, OC curves that reflect microbial agglomeration should be constructed considering that variance is not constant but dependant on the level of microbial concentration of the lot.

Markov regression models for count time series with excess zeros: A partial likelihood approach

Count data with excess zeros are common in many biomedical and public health applications. The zero-inflated Poisson (ZIP) regression model has been widely used in practice to analyze such data. In this paper, we extend the classical ZIP regression framework to model count time series with excess zeros. A Markov regression model is presented and developed, and the partial likelihood is employed for statistical inference. Partial likelihood inference has been successfully applied in modeling time series where the conditional distribution of the response lies within the exponential family. Extending this approach to ZIP time series poses methodological and theoretical challenges, since the ZIP distribution is a mixture and therefore lies outside the exponential family. In the partial likelihood framework, we develop an EM algorithm to compute the maximum partial likelihood estimator (MPLE). We establish the asymptotic theory of the MPLE under mild regularity conditions and investigate its finite sample behavior in a simulation study. The performances of different partial-likelihood based model selection criteria are compared in the presence of model misspecification. Finally, we present an epidemiological application to illustrate the proposed methodology.

Non nested model selection for spatial count regression models with application to health insurance

In this paper we consider spatial regression models for count data. We examine not only the Poisson distribution but also the generalized Poisson capable of modeling over-dispersion, the negative Binomial as well as the zero-inflated Poisson distribution which allows for excess zeros as possible response distribution. We add random spatial effects for modeling spatial dependency and develop and implement MCMC algorithms in $$R$$ for Bayesian estimation. The corresponding R library ‘spatcounts’ is available on CRAN. In an application the presented models are used to analyze the number of benefits received per patient in a German private health insurance company. Since the deviance information criterion is only appropriate for exponential family models, we use in addition the Vuong and Clarke test with a Schwarz correction to compare possibly non nested models. We illustrate how they can be used in a Bayesian context.

Conditional tests for homogeneity of zero-inflated Poisson and Poisson-hurdle distributions

We develop two conditional tests for homogeneity of zero-inflated Poisson (ZIP) and Poisson-hurdle distributions. A Monte Carlo method is proposed for approximating the reference distributions of these tests. The techniques are applied to two examples.

Modelling homogeneous and heterogeneous microbial contaminations in a powdered food product

The actual physical distribution of microorganisms within a batch of food influences quantification of microorganisms in the batch, resulting from sampling and enumeration by microbiological tests. Quantification may be most accurate for batches in which microorganisms are distributed homogeneously. However, when the distribution is non-homogeneous, quantification may result in an under-, or overestimation. In the case of pathogens being non-homogeneously distributed, this heterogeneity will impact on public health. Enumeration data are commonly modelled by the Lognormal distribution. Although the Lognormal distribution can model heterogeneity, it does not allow for complete absence of microorganisms. Studies that validate the appropriateness of using Lognormal or other statistical distributions are scarce.This study systematically investigated laboratory and industrial scale batches of powdered infant formula, modelled the enumeration data using a range of statistical distributions, and assessed the appropriateness of individual models. For laboratory scale experiments, batches of milk powder were contaminated by distributing similar numbers of cells of Cronobacter sakazakii either homogeneously throughout a batch of milk powder or by distributing the cells in a localised part of the batch. Each batch was then systematically sampled and the distribution determined by enumerating the samples. By also enumerating the remainder of the batch, a balance could be made of the total number of microorganisms added and of the number retrieved from a batch. Discrete, as well as continuous statistical distributions, were fitted to enumeration data and the parameters estimated by Maximum Likelihood. The data were fitted both as censored and uncensored data. Enumeration data obtained for an industrial batch of powdered infant formula were investigated in this way as well. It was found that Normal, Poisson and Zero-Inflated Poisson distributions fitted the data sets very poorly. In case of homogeneous contamination, there was not a notable difference between the ability of Negative Binomial, Poisson-Lognormal, Weibull, Gamma, and Lognormal distributions to model the data. Overall, either the Negative Binomial distribution or the Poisson-Lognormal distribution fitted the data best in the 10 batches studied, especially when part of a data set contained zeros and/or the numbers were low. The Negative Binomial fitted the laboratory batches best and the Poisson-Lognormal fitted the industrial batch best.

Operational Value-at-Risk in Case of Zero-Inflated Frequency

In this paper we analyze operational risk in case of zero-inflated frequency data. We show that standard Poisson distribution does not fit correctly excess zero count data. Alternatively, Zero-inflated Poisson (ZIP) distribution fits better such data. To assess the benefits of the use of ZIP distribution in case of excess zero on operational risk management, we develop two separate aggregate distributions. The first one is based on standard Poisson distribution and the second on ZIP distribution. Note that the severity model is the same for both aggregation models. Results show that operational capital charge based on standard Poisson distribution is underestimated by 5% at probability 99.99%.

A Comparison of Two Test Statistics for Poisson Overdispersion/Underdispersion

Within the family of zero-inflated Poisson distributions, the data has Poisson distribution if any only if the mean equals the variance. In this paper we compare two closely related test statistics constructed based on this idea. Our results show that although these two tests are asymptotically equivalent under the null hypothesis and are equally efficient, one test is always more efficient than the other one for small and medium sample sizes.