Large Number Of Zeros Research Articles

Joint Species Distribution Modeling: Dimension Reduction Using Dirichlet Processes

Species distribution models are used to evaluate the variables that affect the distribution and abundance of species and to predict biodiversity. Historically, such models have been fitted to each species independently. While independent models can provide useful information regarding distribution and abundance, they ignore the fact that, after accounting for environmental covariates, residual interspecies dependence persists. With stacking of individual models, misleading behaviors, may arise. In particular, individual models often imply too many species per location. Recently developed joint species distribution models have application to presence–absence, continuous or discrete abundance, abundance with large numbers of zeros, and discrete, ordinal, and compositional data. Here, we deal with the challenge of joint modeling for a large number of species. To appreciate the challenge in the simplest way, with just presence/absence (binary) response and say, S species, we have an S-way contingency table with 2S cell probabilities. Even if S is as small as 100 this is an enormous table, infeasible to work with without some structure to reduce dimension. We develop a computationally feasible approach to accommodate a large number of species (say order 103) that allows us to: 1) assess the dependence structure across species; 2) identify clusters of species that have similar dependence patterns; and 3) jointly predict species distributions. To do so, we build hierarchical models capturing dependence between species at the first or “data” stage rather than at a second or “mean” stage. We employ the Dirichlet process for clustering in a novel way to reduce dimension in the joint covariance structure. This last step makes computation tractable. We use Forest Inventory Analysis (FIA) data in the eastern region of the United States to demonstrate our method. It consists of presence–absence measurements for 112 tree species, observed east of the Mississippi. As a proof of concept for our dimension reduction approach, we also include simulations using continuous and binary data.

Mathematical modeling of operating modes of underground gas storage facilities

Underground gas storage facilities (UGS) are considered, which are technologically inseparable objects of a single gas transportation system (GTS) and provide reliable supply and gas transit. The system analysis of the problems arising in the process of UGS operation, as a separate technological facility, and as part of the gas transportation system, led to the search for ways to solve them. The existing methodological support, which has been transferred from gas production without changes, and metrological support does not provide high-quality information support for dispatching systems. Now most of the existing problems with the least resource costs can be solved by means of modeling and optimization. To this end, mathematical models of all the main technological objects involved in the process of injecting into reservoirs and withdrawing from gas layers are proposed. The models are adapted to real data – the values of the gas flow parameters in the area of their possible change. The process of constructing an integrated model and its analysis shows the instability of the work of the implemented methods at the junction of heterogeneous objects (instability of the process of «cross-linking» the parameters of gas-dynamic processes) to ensure, with a given accuracy, the equality of balance equations. In order to avoid this effect, new methods for solving systems with different mathematical representations of the equations are proposed, ensuring a stable obtaining of the result with guaranteed accuracy. In addition, a method for solving systems of equations is implemented, the matrix of the numerical model of the reservoir is sparse (contains a large number of zeros), which allowed to speed up the process of obtaining results by several orders of magnitude. To simulate compressor stations, an imitation (algorithmic) model is proposed. This approach makes it possible to take into account the actual state of each gas compressor unit and set optimization tasks according to the criteria for the stability of the compressor station and its optimality by the energy criterion.

Optimal Representation of Large-Scale Graph Data Based on K2-Tree

Graph is widely used to model data in various applications. With the rapid growth of many emerging applications such as Internet of Things, it is urgent to require the processing capability on large scale graphs with billions of vertices. Web graph is a typical case of graph data that is widely used for analyzing the structure, behavior and evolution of the World Wide Web. In this paper, we focus on optimal representation of large-scale Web graphs. Our work is motivated by the need of fit large-scale graphs into the main memory and carry out analyze on them. By analyzing the adjacency matrix of Web graphs, we find two characteristics on the distribution of 1s in the matrix. Firstly, only a very small proportion of elements in the matrix are 1s. Secondly, majority of 1s gather around the principal diagonal and form a few number of clusters in the matrix. Based on these characteristics, we first develop a clustering mechanism to locate the clusters of 1s in the adjacency matrix. Then, we combine this clustering mechanism with a structure named K2-tree and propose an approach for representing large-scale Web graphs compactly. Basic idea of the approach is trying to compress a large number of zeros as a single zero. Experimental results show that, our approach not only reduces the space for representing a Web graph, but also reduces the time consumption for operations such as retrieving neighbors of any nodes on the graph; compared with existing approaches, our approach achieves the best space/time tradeoff.

<b>Modeling the incidence of citrus canker in leaves of the sweet orange variety ‘Pera’

Citrus canker, caused by the bacterium Xanthomonas citri subsp. citri , is one of the most important diseases of citrus. The use of resistant genotypes plays an important role in the management and control of the disease and is the most environmentally sustainable approach to disease control. Citrus canker incidence was recorded in an experiment on nine genotypes of the sweet orange variety ‘Pera’ grafted on four rootstocks. The experiment was started in 2010 and the incidence of citrus canker on the leaves was recorded on a quarterly basis. The incidence data from the experiment were analyzed using a zero-inflated Beta regression model (RBIZ), which is the appropriate method to describe data with large numbers of zeros. Based on the residual analysis, the data fit the model well. The discrete component of the explanatory variable, rootstock, was not significant as a factor affecting the onset of disease, in contrast with the continuous component, genotype, which was significant in explaining the incidence of citrus canker.

The employment history of people with and without asthma

Introduction People with asthma may have a greater likelihood of leaving employment due to ill health, spending more time in unemployment and changing jobs more frequently because of their disease. Some studies have reported that people with childhood asthma were disadvantaged in later employment. The aim of this analysis is to investigate the effects of asthma on employment history from early-to mid-adult life (age 42) in a British birth cohort. Methods Members of the National Child Development Study (NCDS) cohort with information on asthma and an employment history from ages 16 to 42 were included (n=9881). Asthma was defined as childhood asthma (0-16), severe childhood asthma, current asthma at age 16 and adult onset asthma. Outcomes were ever having a period out of employment, total periods of employment and unemployment and total time out of employment. The associations between asthma and the outcomes were assessed separately for men and women using Mann-Whitney or Kruskal Wallis tests, with significant results explored using a logistic regression model for binary outcomes and a negative binomial regression model (with a zero inflated model for outcomes with large numbers of zeros) for count data. Results By age 42, 43% of the cohort had experienced at least one period out of employment and the median number of employment periods was 6, with women reporting on average one more job by age 42 than men. Women reporting adult onset asthma were likely to have had more periods of employment when compared to women who never had asthma (p=0.003). For both men and women with adult onset asthma there was a doubling of the risk of ever being out of employment due to ill health compared to those without asthma (men OR 1.94 [95% CI 1.24-3.04], women OR 2.00 [95% CI 1.43-2.80]). Conclusion There is evidence that adult onset (but not childhood) asthma affects employment history, particularly for women. It is not known whether this reflects the experience of more recent cohorts.

Zero-Inflated Beta Regression for Differential Abundance Analysis with Metagenomics Data.

Metagenomics data have been growing rapidly due to the advances in NGS technologies. One goal of human microbial studies is to detect abundance differences across clinical conditions. Besides small sample size and high dimension, metagenomics data are usually represented as compositions (proportions) with a large number of zeros and skewed distribution. Efficient tools for handling such compositional data need to be developed. We propose a zero-inflated beta regression approach (ZIBSeq) for identifying differentially abundant features between multiple clinical conditions. The proposed method takes the sparse nature of metagenomics data into account and handle the compositional data efficiently. Compared with other available methods, the proposed approach demonstrates better performance with large AUC values for most simulation studies. When applied to a human metagenomics data, it also identifies biologically important taxa reported from previous studies. The software in R is available upon request from the first author.

Time Series of Zero‐Inflated Counts and their Coherent Forecasting

Poisson integer‐valued auto‐regressive process of order 1 (PINAR(1)) due to Al‐Osh and Alzaid (Journal of Time Series Analysis 1987; 8(3): 261–275) and McKenzie (Advances in Applied Probability 1988; 20(4): 822–835) has received a significant attention in modelling low‐count time series during the last two decades because of its simplicity. But in many practical scenarios, the process appears to be inadequate, especially when data are overdispersed in nature. This overdispersion occurs mainly for three reasons: presence of some extreme values, large number of zeros, and presence of both extreme values with a large number of zeros. In this article, we develop a zero‐inflated Poisson INAR(1) process as an alternative to the PINAR(1) process when the number of zeros in the data is larger than the expected number of zeros by the Poisson process. We investigate some important properties such as stationarity, ergodicity, autocorrelation structure, and conditional distribution, with a detailed study on h‐step‐ahead coherent forecasting. A comparative study among different methods of parameter estimation is carried out using some simulated data. One real dataset is analysed for practical illustration. Copyright © 2015 John Wiley & Sons, Ltd.

Exploring the application of the Negative Binomial–Generalized Exponential model for analyzing traffic crash data with excess zeros

In order to analyze crash data, many new analysis tools are being developed by transportation safety analysts. The Negative Binomial–Generalized Exponential distribution (NB–GE) is such a tool that was recently introduced to handle datasets characterized by a large number of zero counts and is over-dispersed. As the name suggests, this three-parameter distribution is a combination of both Negative binomial and Generalized Exponential distributions. So far, nobody has used this distribution in the context of a regression model for analyzing datasets with excess zeros. This paper therefore describes the application of the NB–GE generalized linear model (GLM). The distribution and GLM were applied to four datasets known to have large dispersion and/or a large number of zeros. The NB–GE was compared to the Poisson, NB as well as the Negative Binomial–Lindley (NB–L) model, another three-parameter recently introduced in the safety literature. The study results show that for datasets characterized by a sizable over-dispersion and contain a large number of zeros, the NB–GE performs similarly as the NB–L, but significantly outclass the traditional NB model. Furthermore, the NB-GE model has a simpler modeling framework than the NB–L, which makes its application relatively straight forward.

A two-stage statistical procedure for feature selection and comparison in functional analysis of metagenomes

With the advance of new sequencing technologies producing massive short reads data, metagenomics is rapidly growing, especially in the fields of environmental biology and medical science. The metagenomic data are not only high dimensional with large number of features and limited number of samples but also complex with a large number of zeros and skewed distribution. Efficient computational and statistical tools are needed to deal with these unique characteristics of metagenomic sequencing data. In metagenomic studies, one main objective is to assess whether and how multiple microbial communities differ under various environmental conditions. We propose a two-stage statistical procedure for selecting informative features and identifying differentially abundant features between two or more groups of microbial communities. In the functional analysis of metagenomes, the features may refer to the pathways, subsystems, functional roles and so on. In the first stage of the proposed procedure, the informative features are selected using elastic net as reducing the dimension of metagenomic data. In the second stage, the differentially abundant features are detected using generalized linear models with a negative binomial distribution. Compared with other available methods, the proposed approach demonstrates better performance for most of the comprehensive simulation studies. The new method is also applied to two real metagenomic datasets related to human health. Our findings are consistent with those in previous reports. R code and two example datasets are available at http://cals.arizona.edu/∼anling/software.htm. Supplementary file is available at Bioinformatics online.

A Combination of CUSUM Charts for Monitoring a Zero-Inflated Poisson Process

The Zero-inflated Poisson distribution (ZIP) is used to model the defects in processes with a large number of zeros. We propose a control charting procedure using a combination of two cumulative sum (CUSUM) charts to detect increases in the parameters of ZIP process, one is a conforming run length (CRL) CUSUM chart and another is a zero truncated Poisson (ZTP) CUSUM chart. The control limits of the control charts are obtained using both Markov chain-based methods and simulations. Simulation experiments show that the proposed method outperforms an existing method. Finally, a real example is presented.

THE NEGATIVE BINOMIAL-ERLANG DISTRIBUTION WITH APPLICATIONS

This paper introduces a new three-parameter of the mixed negative binomial distribution which is called the negative binomial-Erlang distribution. This distribution obtained by mixing the negative binomial distribution with the Erlang distribution. The negative binomial-Erlang distribution can be used to describe count data with a large number of zeros. The negative binomial- exponential is presented as special cases of the negative binomial-Erlang distri- bution. In addition, we present some properties of the negative binomial-Erlang distribution including factorial moments, mean, variance, skewness and kurto- sis. The parameter estimation for negative binomial-Erlang distribution by the maximum likelihood estimation are provided. Applications of the the nega- tive binomial-Erlang distribution are carried out two real count data sets. The result shown that the negative binomial-Erlang distribution is better than fit when compared the Poisson and negative binomial distributions.

The negative binomial-generalized exponential (NB-GE) distribution

This paper introduced a new mixed distribution, namely the negative binomial-generalized exponential (NB-GE) distribution, which it obtained by mixing the negative binomial (NB) distribution with a generalized exponential (GE) distribution. The NB-GE distribution offers the advantage of being able to handle this kind of data sets, while still maintaining similar characteristics as the traditional negative binomial. The closed form and the factorial moment of the NB-GE distribution are derived. The NB-GE distribution appropriate when we find the one related to count data which have a large number of zeros and overdispersion in Poisson distribution. In addition, we present the basic properties of the new distribution such as mean, variance, skewness and kurtosis. Including, the negative binomial-exponential distributions are presented as special cases of this NB-GE distribution. Parameters estimation is also implemented using maximum likelihood method and the usefulness of the NB-GE distribution is illustrated by real data set. When using a goodness of fit test, the results show that the NB-GE distribution can provide a better fit than the NB distribution and Poisson distribution for count data that contain a large number of zeros.

Identifiability of zero-inflated Poisson models

Zero-inflated Poisson (ZIP) models, which are mixture models, have been popularly used for count data that often contain large numbers of zeros, but their identifiability has not yet been thoroughly explored. In this work, we systematically investigate the identifiability of the ZIP models under a number of different assumptions. More specifically, we show the identifiability of a parametric ZIP model in which the incidence probability p(x) and Poisson mean λ(x) are modeled parametrically as p(x) = exp(β0 + β1x)/[1 + exp(β0 + β1x)] and λ(x) = exp(α0+α1x) for x being a continuous covariate in a closed interval. A semiparametric ZIP regression model is shown to be identifiable in which (i) p(x) = exp(β0 + β1x)/[1 + exp(β0 + β1x)] and λ(x) = exp[s(x)], (ii) p(x) = exp[r(x)]/{1 + exp[r(x)]} and λ(x) = exp(α0 + α1x), or (iii) p(x) = exp[r(x)]/{1 + exp[r(x)]} and λ(x) = exp[s(x)] for r(x) and s(x) being unspecified smooth functions.

Implementation of a finite‐difference micromagnetic model on GPU hardware

SUMMARYWe have developed a micromagnetic simulator for graphical processing units (GPU), using the CUDA framework. In this paper, we discuss the optimization of the effective field calculation, both from a mathematical and from a hardware‐specific point of view. By using a finite‐difference discretization scheme, the long‐range magnetostatic field can be calculated using fast Fourier transforms, an approach well suited for the GPU. We show how the implementation can be tuned to the GPU hardware and how the performance can be further increased by dealing with the large number of zeros that typically occurs in the micromagnetic field computation. Additionally, we show how the ferromagnetic exchange interaction can be readily included in the magnetostatic field calculation without any additional computational cost. The resulting high‐performance software can be used to run large‐scale simulations that would have been very time‐consuming on regular CPU hardware. As an example, we present a case study on the de‐pinning of domain walls in racetrack memory devices. Copyright © 2012 John Wiley & Sons, Ltd.

Semiparametric mixture of binomial regression with a degenerate component

Many datasets contain a large number of zeros, and cannot be modeled directly using a single distribution. Motivated by rain data from a global climate model, we study a semiparametric mixture of binomial regressions, in which both the component proportions and the success probabilities depend on the predictors nonparametrically. An EM algorithm is proposed to estimate this semiparametric mixture model by maximizing the local likelihood function. We also consider a special case in which the component proportions are constant while the component success probabilities still depend on the predictors nonparametrically. This model is estimated by a one-step backfitting procedure, and the estimates are shown to achieve the optimal convergence rates. The asymptotic properties of the estimates for both models are established. The proposed procedures are applied to rain data from a global climate model and historical rain data from Edmonton, Canada. Simulation studies show that satisfactory estimates are obtained for the proposed models for finite samples.

A two‐stage model for incidence and prevalence in point‐level spatial count data

We consider the problem of modeling point‐level spatial count data with a large number of zeros. We develop a model that is compatible with scientific assumptions about the underlying data‐generating process. We utilize a two‐stage spatial generalized linear mixed model framework for the counts, modeling incidence, resulting in 0–1 outcomes, and prevalence, resulting in positive counts, as separate but dependent processes and utilize a Gaussian process model for characterizing the underlying spatial dependence. We describe a Bayesian approach and study several variants of our two‐stage model. We fit the models via Markov chain Monte Carlo methods. We study several Markov chain Monte Carlo algorithms, including a version of the Langevin–Hastings algorithm, for exploring the complicated posterior distribution efficiently and recommend an algorithm that is fairly efficient. Finally, we demonstrate the application of our modeling and computational approach on both simulated data and real data from an ecological field survey. Copyright © 2011 John Wiley & Sons, Ltd.

The negative binomial-Lindley generalized linear model: Characteristics and application using crash data

There has been a considerable amount of work devoted by transportation safety analysts to the development and application of new and innovative models for analyzing crash data. One important characteristic about crash data that has been documented in the literature is related to datasets that contained a large amount of zeros and a long or heavy tail (which creates highly dispersed data). For such datasets, the number of sites where no crash is observed is so large that traditional distributions and regression models, such as the Poisson and Poisson-gamma or negative binomial (NB) models cannot be used efficiently. To overcome this problem, the NB-Lindley (NB-L) distribution has recently been introduced for analyzing count data that are characterized by excess zeros. The objective of this paper is to document the application of a NB generalized linear model with Lindley mixed effects (NB-L GLM) for analyzing traffic crash data. The study objective was accomplished using simulated and observed datasets. The simulated dataset was used to show the general performance of the model. The model was then applied to two datasets based on observed data. One of the dataset was characterized by a large amount of zeros. The NB-L GLM was compared with the NB and zero-inflated models. Overall, the research study shows that the NB-L GLM not only offers superior performance over the NB and zero-inflated models when datasets are characterized by a large number of zeros and a long tail, but also when the crash dataset is highly dispersed.

The negative binomial–Lindley distribution as a tool for analyzing crash data characterized by a large amount of zeros

The modeling of crash count data is a very important topic in highway safety. As documented in the literature, given the characteristics associated with crash data, transportation safety analysts have proposed a significant number of analysis tools, statistical methods and models for analyzing such data. Among the data issues, we find the one related to crash data which have a large amount of zeros and a long or heavy tail. It has been found that using this kind of dataset could lead to erroneous results or conclusions if the wrong statistical tools or methods are used. Thus, the purpose of this paper is to introduce a new distribution, known as the negative binomial-Lindley (NB-L), which has very recently been introduced for analyzing data characterized by a large number of zeros. The NB-L offers the advantage of being able to handle this kind of datasets, while still maintaining similar characteristics as the traditional negative binomial (NB). In other words, the NB-L is a two-parameter distribution and the long-term mean is never equal to zero. To examine this distribution, simulated and observed data were used. The results show that the NB-L can provide a better statistical fit than the traditional NB for datasets that contain a large amount of zeros.

Comparing samples with large numbers of zeros

Comparing samples with large numbers of zeros

Tobit or OLS? An Empirical Evaluation Under Different Diary Window Lengths

Time use researchers frequently debate whether it is more appropriate to fit censored regression (Tobit) models using maximum likelihood estimation or linear models using ordinary least squares (OLS) to explain individuals’ allocations of time to different activities as recorded in time-diary data. One side argues that estimation of Tobit models addresses the significant censoring (i.e., large numbers of zeros) typically found in time-diary data and that OLS estimation leads to biased and inconsistent estimates. The opposing side argues that optimization occurs over a longer period than that covered by the typical time diary, and thus that reported zeros represent measurement error rather than true non-participation in the activity, in which case OLS is preferred. We use the Australian Time Use Surveys, which record information for two consecutive diary days, to estimate censored and linear versions of a parental child care model for both 24-hour and 48-hour windows of observation in order to determine the empirical consequences of estimation technique and diary length. We find a moderate amount of measurement error when we use the 24-hour window compared to the 48-hour window, but a large number of zeros in the shorter window remain zeroes when we double the window length. Most of the qualitative conclusions we draw are similar for the two windows of observation and the two estimation methods, although there are some slight differences in the magnitudes and statistical significance of the estimates. Importantly, Tobit estimates appear to be more sensitive to window length than OLS estimates.