Large Number Of Zeros Research Articles

A zero-inflated Poisson integer-valued autoregressive model with time-varying coefficients covariates

For integer-valued time series, we may be interested in how a covariate affects the series. In the existing models, it is commonly assumed that the coefficients of the covariates are constant, and the innovation process follows a Poisson distribution. However, in practice, the influence of covariates may vary with time, and there may be a large number of zeros. To address this situation, this paper proposes an integer-valued autoregressive model with time-varying coefficients on covariates, incorporating a zero-inflated Poisson innovation process. The binomial thinning parameters and unknown functions of the proposed model are estimated based on approximate conditional least squares and the cubic spline method. The consistency and asymptotic properties of the binomial thinning parameters are investigated. Simulation studies and real data analysis support the flexibility and good performance of the proposed model.

Score Matching for Compositional Distributions

Compositional data are challenging to analyse due to the non-negativity and sum-to-one constraints on the sample space. With real data, it is often the case that many of the compositional components are highly right-skewed, with large numbers of zeros. Major limitations of currently available models for compositional data include one or more of the following: insufficient flexibility in terms of distributional shape; difficulty in accommodating zeros in the data in estimation; and lack of computational viability in moderate to high dimensions. In this article, we propose a new model, the polynomially tilted pairwise interaction (PPI) model, for analysing compositional data. Maximum likelihood estimation is difficult for the PPI model. Instead, we propose novel score matching estimators, which entails extending the score matching approach to Riemannian manifolds with boundary. These new estimators are available in closed form and simulation studies show that they perform well in practice. As our main application, we analyse real microbiome count data with fixed totals using a multinomial latent variable model with a PPI model for the latent variable distribution. We prove that, under certain conditions, the new score matching estimators are consistent for the parameters in the new multinomial latent variable model.

Generalized zero-inated Poisson regression mixture model for fitting health-related data

In many bioscience studies, it is common to encounter count data with a large number of zeros that Poisson regression model or standard zero-inflated Poisson (ZIP) regression model do not fit well. Generalized zero-inflated Poisson (GZIP) regression mixture model can handle the data with excess zeros and overdispersion caused by unobserved heterogeneity. For the parameter estimation, expectation-maximization (EM) algorithm with iteratively reweighted least sqaures (IRLS) method is used. We applied GZIP regression mixture model into two health-related data, Behavioral Risk Factor Surveillance System (BRFSS) data and Integrated Public Use Microdata Series (IPUMS) census data, and compared the performance of the models using AIC and BIC to find the best mixture model.

Accuracy of Zero Inflated Generalized Poisson Exponentially Moving Average Control Chart

The Zero-Inflated Generalized Poisson (ZIGP) distribution is a case-based distribution where the discrete data has a large number of zeros and an overdispersion occurs, i.e. the variance is greater than the mean value. The purpose of this study is to determine the Exponential Weight Moving Average (EWMA) control chart with the assumption that the data has a Zero-Inflated Generalized Poisson (ZIP) distribution. The results show that the ARL value of the ARL ZIGP EWMA control chart has better accuracy when compared to when using the ZIP EWMA control chart on ZIGP distributed data. This is indicated by the smaller ARL value compared to the ZIP EWMA control chart, namely when φ = 1.4, and φ = 0.6. So that the ARL ZIGP EWMA control chart has a fairly good accuracy in detecting out of control conditions for ZIGP distributed data. In addition, the modified ARL shows the same values ​​before and after the modification for the underdispersion data and shows a larger or negative value for the overdispersion data. This can eliminate or reduce errors in analyzing the accuracy of the control chart.

Modeling Hospitalization Decision and Utilization for the Elderly in China

Getting medical services has become more difficult and expensive in China, which led to a problem of illness not being treated and a large number of zeros in the statistics of being hospitalized for the elderly. Traditional classic models such as the Poisson model and the negative binomial model cannot fit this kind of data well. One aim of this study was to use zero-inflated and hurdle models to better solve the problem of excess zeros. Another aim was to discover the factors affecting the decision-making behavior of the elderly being hospitalized and hospitalization service utilization. Therefore, the XGBoost model was firstly introduced to rank the importance of influencing factors in this paper. It was found that the zero-inflated negative binomial model performed best. The results showed that the elderly who had enjoyed NRCM or ERBMI/URBMI were more likely to have a higher number of hospitalizations. This indicated that the high cost of hospitalization had prevented the willingness of the elderly being hospitalized, but the basic medical insurance had increased the times of their repeated hospital readmissions. Policy efforts should be made to improve the level of basic medical insurance.

Models for Overdispersion Count Data with Generalized Distribution: An Application to Parasites Intensity

The Poisson regression model is widely used for count data. This model assumes equidispersion. In practice, equidispersion is seldom reflected in data. However, in real-life data, the variance usually exceeds the mean. This situation is known as overdispersion. Negative binomial distribution and other Poisson mix models are often used to model overdispersion count data. Another extension of the negative binomial distribution in another model for count data is the univariate generalized Waring. In addition, the model developed by Famoye can be used in the analysis of count data. When the count data contains a large number of zeros, it is necessary to use zero-inflated models. In this study, different generalized regression models are emphasized for the analysis of excessive zeros count data. For this purpose, a real data set was analysed with the generalized Poisson model, generalized negative binomial model, generalized negative binomial Famoye, generalized Waring model, and the foregoing zero-inflated models. Log-likelihood, Akaike information criterion, Bayes information criterion, Vuong statistics were used for model comparisons.

Log-ratio analysis of microbiome data with many zeroes is library size dependent.

Microbiome composition data collected through amplicon sequencing are count data on taxa in which the total count per sample (the library size) is an artefact of the sequencing platform, and as a result, such data are compositional. To avoid library size dependency, one common way of analysing multivariate compositional data is to perform a principal component analysis (PCA) on data transformed with the centred log‐ratio, hereafter called a log‐ratio PCA. Two aspects typical of amplicon sequencing data are the large differences in library size and the large number of zeroes. In this study, we show on real data and by simulation that, applied to data that combine these two aspects, log‐ratio PCA is nevertheless heavily dependent on the library size. This leads to a reduction in power when testing against any explanatory variable in log‐ratio redundancy analysis. If there is additionally a correlation between the library size and the explanatory variable, then the type 1 error becomes inflated. We explore putative solutions to this problem.

Measuring temporal patterns in ecology: The case of mast seeding.

Properly assessing temporal patterns is a central issue in ecology in order to understand ecosystem processes and their mechanisms. Mast seeding has traditionally been described as a reproductive behavior consisting of highly variable and synchronized reproductive events. The most common metric used to measure temporal variability and thus infer masting behavior, the coefficient of variation (CV), however, has been repeatedly suggested to improperly estimate temporal variability. Biases of CV estimates are especially problematic for non‐normally distributed data and/or data sets with a high number of zeros.Some recent studies have already adopted new metrics to measure temporal variability, but most continue to use CV. This controversy has started a strong debate about what metrics to use.We here summarize the problems of CV when assessing temporal variability, particularly across data sets containing a large number of zeros, and highlight the benefits of using other metrics of temporal variability, such as proportional variability (PV) and consecutive disparity (D). We also suggest a new way to look at reproductive behavior, by separating temporal variability from frequency of reproduction, to allow better comparison of data sets with different characteristics.We suggest future studies to properly describe the temporal patterns in fully scientific and measurable terms that do not lead to confusion, such as variability and frequency of reproduction, using robust and fully comparable metrics.

Comparison of zero replacement strategies for compositional data with large numbers of zeros

Modern applications in chemometrics and bioinformatics result in compositional data sets with a high proportion of zeros. An example are microbiome data, where zeros refer to measurements below the detection limit of one count. When building statistical models, it is important that zeros are replaced by sensible values. Different replacement techniques from compositional data analysis are considered and compared by a simulation study and examples. The comparison also includes a recently proposed method (Templ, 2020) [1] based on deep learning. Detailed insights into the appropriateness of the methods for a problem at hand are provided, and differences in the outcomes of statistical results are discussed.

Too many zeros and/or highly skewed? A tutorial on modelling health behaviour as count data with Poisson and negative binomial regression

ABSTRACT Background Dependent variables in health psychology are often counts, for example, of a behaviour or number of engagements with an intervention. These counts can be very strongly skewed, and/or contain large numbers of zeros as well as extreme outliers. For example, ‘How many cigarettes do you smoke on an average day?’ The modal answer may be zero but may range from 0 to 40+. The same can be true for minutes of moderate-to-vigorous physical activity. For some people, this may be near zero, but take on extreme values for someone training for a marathon. Typical analytical strategies for this data involve explicit (or implied) transformations (smoker v. non-smoker, log transformations). However, these data types are ‘counts’ (i.e. non-negative whole numbers) or quasi-counts (time is ratio but discrete minutes of activity could be analysed as a count), and can be modelled using count distributions – including the Poisson and negative binomial distribution (and their zero-inflated and hurdle extensions, which alloweven more zeros). Methods In this tutorial paper I demonstrate (in R, Jamovi, and SPSS) the easy application of these models to health psychology data, and their advantages over alternative ways of analysing this type of data using two datasets – one highly dispersed dependent variable (number of views on YouTube, and another with a large number of zeros (number of days on which symptoms were reported over a month). Results The negative binomial distribution had the best fit for the overdispersed number of views on YouTube. Negative binomial, and zero-inflated negative binomial were both good fits for the symptom data with over-abundant zeros. Conclusions In both cases, count distributions provided not just a better fit but would lead to different conclusions compared to the poorly fitting traditional regression/linear models.

A review of models of forest fire occurrence prediction in China

We summarized research progress of forest fire occurrence prediction model in China based on the literature review, from the prospects of forest fire drivers, models of forest fire occurrence probability, models of forest fire occurrence frequency and model validation methods. The main conclusions are: 1) Meteorology, terrain, vegetation, fuel and human activities were the main driving factors of forest fire occurrence and model prediction accuracy. 2) In the models of forest fire occurrence probability, the geographically weighted logistic regression model considered the spatial correlation between model variables, the Gompit regression model could fit the asymmetric structure fire data. The random forest algorithm had a high prediction accuracy without the requirement of multicollinearity test and excessive fitting, which made it as one of the optimal methods of forest fire occurrence probability prediction. 3) Among all the forest fire occurrence frequency models, the negative binomial regression model was suitable for fitting the over discrete data, the zero-inflated model and hurdle model could deal with fire data that contained a large number of zeros. 4) ROC test, AIC test, likelihood ratio test, and Wald test were the most common methods for evaluating the accuracy of fire occurrence probability and frequency models. The study of forest fire occurrence prediction model should be the main focus of the forest fire management. Model selection should base on fire data structure of different forests. More influencing factors should be taken into account to improve the prediction accuracy of model. In addition, it was necessary to further explore the application of other mathematical methods in forest fire prediction, to improve the accuracy of the models.

A four-parameter negative binomial-Lindley distribution for modeling over and underdispersed count data with excess zeros

Count data often exhibits the property of dispersion and have large number of zeros. In order to take these properties into account, a new generalized negative binomial-Lindley distribution with four parameters is proposed, of which the two-parameter and three-parameter negative binomial-Lindley distributions are special cases. Several statistical properties of the proposed distribution are presented. The dispersion index for the proposed distribution is derived and based on the index, it is clear that the proposed distribution can adequately fit the data with properties of overdispersion or underdispersion depending on the choice of the parameters. The proposed distribution is fitted to three overdispersed datasets with large proportion of zeros. The best fitted model is selected based on the values of AIC, mean absolute error and root mean squared error. From the model fittings, it can be concluded that the proposed distribution outperforms Poisson and negative binomial distributions in fitting the count data with overdispersion and large number of zeros.

Quasi-binomial zero-inflated regression model suitable for variables with bounded support

In recent years, a variety of regression models, including zero-inflated and hurdle versions, have been proposed to explain the case of a dependent variable with respect to exogenous covariates. Apart from the classical Poisson, negative binomial and generalised Poisson distributions, many proposals have appeared in the statistical literature, perhaps in response to the new possibilities offered by advanced software that now enables researchers to implement numerous special functions in a relatively simple way. However, we believe that a significant research gap remains, since very little attention has been paid to the quasi-binomial distribution, which was first proposed over fifty years ago. We believe this distribution might constitute a valid alternative to existing regression models, in situations in which the variable has bounded support. Therefore, in this paper we present a zero-inflated regression model based on the quasi-binomial distribution, taking into account the moments and maximum likelihood estimators, and perform a score test to compare the zero-inflated quasi-binomial distribution with the zero-inflated binomial distribution, and the zero-inflated model with the homogeneous model (the model in which covariates are not considered). This analysis is illustrated with two data sets that are well known in the statistical literature and which contain a large number of zeros.

Preserving the distribution function in surveys in case of imputation for zero inflated data

Item non-response in surveys is usually handled by single imputation, whose main objective is to reduce the non-response bias. Imputation methods need to be adapted to the study variable. For instance, in business surveys, the interest variables often contain a large number of zeros. Motivated by a mixture regression model, we propose two imputation procedures for such data and study their statistical properties. We show that these procedures preserve the distribution function if the imputation model is well specified. The results of a simulation study illustrate the good performance of the proposed methods in terms of bias and mean square error.

A latent allocation model for the analysis of microbial composition and disease

BackgroundEstablishing the relationship between microbiota and specific diseases is important but requires appropriate statistical methodology. A specialized feature of microbiome count data is the presence of a large number of zeros, which makes it difficult to analyze in case-control studies. Most existing approaches either add a small number called a pseudo-count or use probability models such as the multinomial and Dirichlet-multinomial distributions to explain the excess zero counts, which may produce unnecessary biases and impose a correlation structure taht is unsuitable for microbiome data.ResultsThe purpose of this article is to develop a new probabilistic model, called BERnoulli and MUltinomial Distribution-based latent Allocation (BERMUDA), to address these problems. BERMUDA enables us to describe the differences in bacteria composition and a certain disease among samples. We also provide a simple and efficient learning procedure for the proposed model using an annealing EM algorithm.ConclusionWe illustrate the performance of the proposed method both through both the simulation and real data analysis. BERMUDA is implemented with R and is available from GitHub (https://github.com/abikoushi/Bermuda).

Fat-Tailed Regression Modeling with Spliced Distributions

Insurance claims data usually contains a large number of zeros and exhibits fat-tail behavior. Misestimation of one end of the tail impacts the other end of the tail of the claims distribution; such can affect both the adequacy of premiums and needed reserves to hold. In addition, insured policyholders in a portfolio are naturally non-homogeneous. It is an ongoing challenge for actuaries to be able to build a predictive model that will simultaneously capture these peculiar characteristics of claims data and policyholder heterogeneity. Such models can help make improved predictions and thereby ease the decision making process. This paper proposes the use of spliced regression models for fitting insurance loss data. A primary advantage of spliced distributions is its flexibility to accommodate modeling different segments of the claims distribution with different parametric models. The threshold that breaks the segments is assumed to be a parameter and this presents an additional challenge in the estimation. Our simulation study demonstrates the effectiveness of using multi-stage optimization for likelihood inference and at the same time, the repercussions of model misspecification. For purposes of illustration, we consider three-component spliced regression models: the first component contains zeros, the second component models the middle segment of the loss data, and the third component models the tail segment of the loss data. We calibrate these proposed models and evaluate their performance using a Singapore auto insurance claims dataset. The estimation results show that the spliced regression model performs better than the Tweedie regression model in terms of tail fitting and prediction accuracy.

Fat-Tailed Regression Modeling with Spliced Distributions

Insurance claims data usually contain a large number of zeros and exhibits fat-tail behavior. Misestimation of one end of the tail impacts the other end of the tail of the claims distribution and can affect both the adequacy of premiums and needed reserves to hold. In addition, insured policyholders in a portfolio are naturally non-homogeneous. It is an ongoing challenge for actuaries to be able to build a predictive model that will simultaneously capture these peculiar characteristics of claims data and policyholder heterogeneity. Such models can help make improved predictions and thereby ease the decision-making process. This article proposes the use of spliced regression models for fitting insurance loss data. A primary advantage of spliced distributions is their flexibility to accommodate modeling different segments of the claims distribution with different parametric models. The threshold that breaks the segments is assumed to be a parameter, and this presents an additional challenge in the estimation. Our simulation study demonstrates the effectiveness of using multistage optimization for likelihood inference and at the same time the repercussions of model misspecification. For purposes of illustration, we consider three-component spliced regression models: the first component contains zeros, the second component models the middle segment of the loss data, and the third component models the tail segment of the loss data. We calibrate these proposed models and evaluate their performance using a Singapore auto insurance claims dataset. The estimation results show that the spliced regression model performs better than the Tweedie regression model in terms of tail fitting and prediction accuracy.

New Forecasting Technique for Intermittent Demand, Based on Stochastic Simulation. An Alternative to Croston’s Method

The main aim of the article is to present a new forecasting technique, applicable in case of intermittent demand. To present properties of this new technique, the accuracy of the predictions generated by the Croston’s method and by the author’s method (based on stochastic simulation) was analyzed. For comparison, methods such as moving average and simple exponential smoothing are as well used as a reference. Also the SBA method, a modification of Croston’s method, is applied. Croston’s method is an extension of adaptive methods. It separates the interval between the (non‑zero) sales and the sales level. Its purpose is to better forecast intermittent (sporadic) demand. The second prognostic method is the author’s proposal which relies on two stages. In the first stage, based on stochastic simulation, it determines if an event (sale) occurs in a given period. In the second stage, the sales level is estimated (if the previous stage shows that the sales will occur). Due to the strong asymmetry of the sales, the sales level is determined on the basis of the corresponding quantiles. The basis for forecasting are weekly sales series of about fourteen thousand products (real data). The analyzed time series can be defined as atypical, which is manifested by a small number of non‑zero observations (high number of zeros), high volatility and randomness (randomness tests indicate white noise). Forecast error measures are used to characterize both the bias and the efficiency. The forecast error measures will be characterized so that they can be applied to a time series with a large number of zeros (including the author’s forecast error measure proposal). Forecasts were evaluated with respect to the distributions of four ex post errors, such as mean error (ME), mean absolute deviation (MAD), mean absolute scaled error (MASE) and the author’s proposal (error D). The proposed technique, based on stochastic simulation, seems to be the least biased and most efficient. The Croston’s method gives positively biased predictions with rather low efficiency. The proposed forecasting technique might support decisions in enterprises facing the problem of forecasting intermittent demand. The more accurate forecasts could increase the quality of customer service and optimize the inventory level.

Modeling change trajectories with count and zero-inflated outcomes: Challenges and recommendations

The goal of this article is to describe models to examine change over time with an outcome that represents a count, such as the number of alcoholic drinks per day. Common challenges encountered with this type of data are: (1) the outcome is discrete, may have a large number of zeroes, and may be overdispersed, (2) the data are clustered (multiple observations within each individual), (3) the researchers needs to carefully consider and choose an appropriate time metric, and (4) the researcher needs to identify both a proper individual (potentially nonlinear) change model and an appropriate distributional form that captures the properties of the data. In this article, we provide an overview of generalized linear models, generalized estimating equation models, and generalized latent variable (mixed-effects) models for longitudinal count outcomes focusing on the Poisson, negative binomial, zero-inflated, and hurdle distributions. We review common challenges and provide recommendations for identifying an appropriate change trajectory while determining an appropriate distributional form for the outcome (e.g., determining zero-inflation and overdispersion). We demonstrate the process of fitting and choosing a model with empirical longitudinal data on alcohol intake across adolescence collected as part of the National Longitudinal Survey of Youth 1997.

An effective numerical method for solving fractional pantograph differential equations using modification of hat functions

In this work, a spectral method based on a modification of hat functions (MHFs) is proposed to solve the fractional pantograph differential equations. Some basic properties of fractional calculus and the operational matrices of MHFs are utilized to reduce the considered problem to a system of linear algebraic equations. The greatest advantage of using MHFs is the large number of zeros in their operational matrix of fractional integration, product operational matrix and also pantograph operational matrix. This property makes these functions computationally attractive. Some illustrative examples are included to show the high performance and applicability of the proposed method and a comparison is made with the existing results. These examples confirm that the method leads to the results of convergence order O(h3).