Multivariate Poisson Distribution Research Articles

Fast Integration of Poisson Distributions for Dead Sensor Marginalization

In dual-phase time-projection chambers there are photosensor arrays arranged to allow for inference of the positions of interactions within the detector. If there is a gap in data left by a broken or saturated photosensors, the inference of the position is less precise and less accurate. As we are unable to repair or replace photosensors once the experiment has begun, we develop methods to estimate the missing signals. Our group is developing a probabilistic graphical model of the correlations between the number of photons detected by adjacent photosensors that represents the probability distribution over photons detected as a Poisson distribution. Determining the posterior probability distribution over a number of photons detected by a sensor then requires integration over a multivariate Poisson distribution, which is computationally intractable for high-dimensions. In this work, we present an approach to quickly calculate and integrate over a multidimensional Poisson distribution. Our approach uses Zarr, a Python array compression package, to manage large multi-dimensional arrays and approximates the log factorial to quickly calculate the Poisson distribution without overflow.

Optimal proportional and excess-of-loss reinsurance for multiple classes of insurance business

In this paper we consider a reinsurance strategy which combines a proportional and an excess-of-loss reinsurance in a risk model with multiple dependent classes of insurance business. Under the assumption that the claim number of the classes has a multivariate Poisson distribution, the aim is to maximize the expected utility of terminal wealth. In a general setting, after deriving the corresponding Hamilton–Jacobi–Bellman equation, we prove a Verification Theorem and identify sufficient conditions for the optimality. Then, in a special case with exponential utility, an explicit solution is found by solving an intricate associated static constrained optimization problem.

Two approaches to monitoring multivariate Poisson counts: Simple and accurate

We consider the monitoring of multivariate correlated count data, which have many applications in practice. Although there are quite a few methods for the statistical process control of Multivariate Poisson (MP) counts, they are either too complicated or too simple to provide a satisfactory tool for efficient online monitoring. In addition, they mostly focus on only the mean vector of multivariate counts and ignore the correlations among them. In this article, we adopt the multivariate Poisson distribution with a two-way covariance structure for modeling MP counts, which has marginal Poisson distributions in each dimension and allows for pairwise correlations. Based on this, we develop two control charts to simultaneously monitor the mean vector and covariance matrix of MP counts. The first chart enjoys a simple charting statistic and is computationally fast, whereas the second one is accurate and provides a gold standard for monitoring MP counts. We also give recommendations on choice between them. Numerical simulations have demonstrated the advantages of the proposed two charts, and in non-Poisson cases we also test their robustness against underdispersion and overdispersion that are encountered often in count data.

Interaction networks from discrete event data by Poisson multivariate mutual information estimation and information flow with applications from gene expression data

In this work, we introduce a new methodology for inferring the interaction structure of discrete valued time series which are Poisson distributed. While most related methods are premised on continuous state stochastic processes, in fact, discrete and counting event oriented stochastic process are natural and common, so called time-point processes. An important application that we focus on here is gene expression, where it is often assumed that the data is generated from a multivariate Poisson distribution. Nonparameteric methods such as the popular k-nearest neighbors are slow converging for discrete processes, and thus data hungry. Now, with the new multi-variate Poisson estimator developed here as the core computational engine, the causation entropy (CSE) principle, together with the associated greedy search algorithm optimal CSE (oCSE) allows us to efficiently infer the true network structure for this class of stochastic processes that were previously not practical. We illustrate the power of our method, first in benchmarking with synthetic datum, and then by inferring the genetic factors network from a breast cancer micro-ribonucleic acid sequence count data set. We show the Poisson oCSE gives the best performance among the tested methods and discovers previously known interactions on the breast cancer data set.

Extreme Compressed Sensing of Poisson Rates from Multiple Measurements.

Compressed sensing (CS) is a signal processing technique that enables the efficient recovery of a sparse high-dimensional signal from low-dimensional measurements. In the multiple measurement vector (MMV) framework, a set of signals with the same support must be recovered from their corresponding measurements. Here, we present the first exploration of the MMV problem where signals are independently drawn from a sparse, multivariate Poisson distribution. We are primarily motivated by a suite of biosensing applications of microfluidics where analytes (such as whole cells or biomarkers) are captured in small volume partitions according to a Poisson distribution. We recover the sparse parameter vector of Poisson rates through maximum likelihood estimation with our novel Sparse Poisson Recovery (SPoRe) algorithm. SPoRe uses batch stochastic gradient ascent enabled by Monte Carlo approximations of otherwise intractable gradients. By uniquely leveraging the Poisson structure, SPoRe substantially outperforms a comprehensive set of existing and custom baseline CS algorithms. Notably, SPoRe can exhibit high performance even with one-dimensional measurements and high noise levels. This resource efficiency is not only unprecedented in the field of CS but is also particularly potent for applications in microfluidics in which the number of resolvable measurements per partition is often severely limited. We prove the identifiability property of the Poisson model under such lax conditions, analytically develop insights into system performance, and confirm these insights in simulated experiments. Our findings encourage a new approach to biosensing and are generalizable to other applications featuring spatial and temporal Poisson signals.

Multivariate zero-inflated Bell distribution and its inference and applications

In this paper we introduce a multivariate family of distributions for multivariate count data with excess zeros, which is a multivariate extension of the univariate zero-inflated Bell distribution. We derive various general properties of this multivariate distribution. In particular, the marginal distributions are univariate zero-inflated Bell distributions. The model parameters are estimated using the traditional maximum likelihood estimation method. In addition, we develop a simple EM algorithm to compute the maximum likelihood estimates of the parameters of the new multivariate distribution with closed-form expressions for the maximum likelihood estimators. Empirical applications that employ real multivariate count data are considered to illustrate the usefulness of the new class of multivariate distributions, and comparisons with the multivariate zero-inflated Poisson distribution, multivariate zero-adjusted Poisson distributions, and multivariate zero-inflated generalized Poisson distribution are made.

Type I multivariate zero-inflated COM-Poisson regression model.

In this paper, we present the Type I multivariate zero-inflated Conway-Maxwell-Poisson distribution, whose development is based on the extension of the Type I multivariate zero-inflated Poisson distribution. We developed important properties of the distribution and present a regression model. The AIC and BIC criteria are used to select the best fitted model. Two real data sets have been used to illustrate the proposed model. Moreover, we conclude by stating that the Type I multivariate zero-inflated Conway-Maxwell-Poisson distribution produces a better fitted model for multivariate count data with excess of zeros.

An MEWMA control chart based on multivariate Poisson distribution data

Count data appears widely in finance, biomedical science, psychology, actuarial insurance, road safety and many other areas of life, and it is often multivariate. In this paper, we consider using the multivariate exponentially weighted moving average (MEWMA) control chart method to monitor count data under the assumption of multivariate Poisson distribution, which is derived by He [1] from another form of multivariate gamma distribution. We illustrate the proposed scheme by numerical simulation. In this paper, we provide two tables of average run length (ARL) values of the MEWMA control chart with different parameter shifts. The performance of the proposed control chart is evaluated by the ARL, given a parameter shift occurred, which is able to deliver satisfactory control efficiency.

A Flexible Multivariate Distribution for Correlated Count Data

Multivariate count data are often modeled via a multivariate Poisson distribution, but it contains an underlying, constraining assumption of data equi-dispersion (where its variance equals its mean). Real data are oftentimes over-dispersed and, as such, consider various advancements of a negative binomial structure. While data over-dispersion is more prevalent than under-dispersion in real data, however, examples containing under-dispersed data are surfacing with greater frequency. Thus, there is a demonstrated need for a flexible model that can accommodate both data types. We develop a multivariate Conway–Maxwell–Poisson (MCMP) distribution to serve as a flexible alternative for correlated count data that contain data dispersion. This structure contains the multivariate Poisson, multivariate geometric, and the multivariate Bernoulli distributions as special cases, and serves as a bridge distribution across these three classical models to address other levels of over- or under-dispersion. In this work, we not only derive the distributional form and statistical properties of this model, but we further address parameter estimation, establish informative hypothesis tests to detect statistically significant data dispersion and aid in model parsimony, and illustrate the distribution’s flexibility through several simulated and real-world data examples. These examples demonstrate that the MCMP distribution performs on par with the multivariate negative binomial distribution for over-dispersed data, and proves particularly beneficial in effectively representing under-dispersed data. Thus, the MCMP distribution offers an effective, unifying framework for modeling over- or under-dispersed multivariate correlated count data that do not necessarily adhere to Poisson assumptions.

Compound Poisson Approximations in $$\ell _p$$-norm for Sums of Weakly Dependent Vectors

The distribution of the sum of 1-dependent lattice vectors with supports on coordinate axes is approximated by a multivariate compound Poisson distribution and by signed compound Poisson measure. The local and $$\ell _\alpha $$ -norms are used to obtain the error bounds. The Heinrich method is used for the proofs.

Modeling and Change Detection for Count-Weighted Multilayer Networks

In a typical network with a set of individuals, it is common to have multiple types of interactions between two individuals. In practice, these interactions are usually sparse and correlated, which is not sufficiently accounted for in the literature. This article proposes a multilayer weighted stochastic block model (MZIP-SBM) based on a multivariate zero-inflated Poisson (MZIP) distribution to characterize the sparse and correlated multilayer interactions of individuals. A variational-EM algorithm is developed to estimate the parameters in this model. We further propose a monitoring statistic based on the score test of MZIP-SBM model parameters for change detection in multilayer networks. The proposed model and monitoring scheme are validated using extensive simulation studies and the case study from Enron E-mail network.

Regression for doubly inflated multivariate Poisson distributions

ABSTRACTDependent multivariate count data occur in several research studies. These data can be modelled by a multivariate Poisson or Negative binomial distribution constructed using copulas. However, when some of the counts are inflated, that is, the number of observations in some cells are much larger than other cells, then the copula-based multivariate Poisson (or Negative binomial) distribution may not fit well and it is not an appropriate statistical model for the data. There is a need to modify or adjust the multivariate distribution to account for the inflated frequencies. In this article, we consider the situation where the frequencies of two cells are higher compared to the other cells and develop a doubly inflated multivariate Poisson distribution function using multivariate Gaussian copula. We also discuss procedures for regression on covariates for the doubly inflated multivariate count data. For illustrating the proposed methodologies, we present real data containing bivariate count observations with inflations in two cells. Several models and linear predictors with log link functions are considered, and we discuss maximum likelihood estimation to estimate unknown parameters of the models.

Global alcohol exposure between 1990 and 2017 and forecasts until 2030: a modelling study

Alcohol use is a leading risk factor for global disease burden, and data on alcohol exposure are crucial to evaluate progress in achieving global non-communicable disease goals. We present estimates on the main indicators of alcohol exposure for 189 countries from 1990-2017, with forecasts up to 2030. Adult alcohol per-capita consumption (the consumption in L of pure alcohol per adult [≥15 years]) in a given year was based on country-validated data up to 2016. Forecasts up to 2030 were obtained from multivariate log-normal mixture Poisson distribution models. Using survey data from 149 countries, prevalence of lifetime abstinence and current drinking was obtained from Dirichlet regressions. The prevalence of heavy episodic drinking (30-day prevalence of at least one occasion of 60 g of pure alcohol intake among current drinkers) was estimated with fractional response regressions using survey data from 118 countries. Between 1990 and 2017, global adult per-capita consumption increased from 5·9 L (95% CI 5·8-6·1) to 6·5 L (6·0-6·9), and is forecasted to reach 7·6 L (6·5-10·2) by 2030. Globally, the prevalence of lifetime abstinence decreased from 46% (42-49) in 1990 to 43% (40-46) in 2017, albeit this was not a significant reduction, while the prevalence of current drinking increased from 45% (41-48) in 1990 to 47% (44-50) in 2017. We forecast both trends to continue, with abstinence decreasing to 40% (37-44) by 2030 (annualised 0·2% decrease) and the proportion of current drinkers increasing to 50% (46-53) by 2030 (annualised 0·2% increase). In 2017, 20% (17-24) of adults were heavy episodic drinkers (compared with 1990 when it was estimated at 18·5% [15·3-21·6%], and this prevalence is expected to increase to 23% (19-27) in 2030. Based on these data, global goals for reducing the harmful use of alcohol are unlikely to be achieved, and known effective and cost-effective policy measures should be implemented to reduce alcohol exposure. Centre for Addiction and Mental Health and the WHO Collaborating Center for Addiction and Mental Health at the Centre for Addiction and Mental Health.

Construction of a new multivariate Poisson distribution through the conditional probability with its variance covariance matrix: application to the simulation of Poisson regressions

Construction of a new multivariate Poisson distribution through the conditional probability with its variance covariance matrix: application to the simulation of Poisson regressions

An attribute control chart for multivariate Poisson distribution using multiple dependent state repetitive sampling

AbstractIn this paper, an attribute control chart for a multivariate Poisson distribution using multiple dependent state repetitive sampling (MDSRS) is presented. The evaluation of the proposed control chart is given through the average run length (ARL). The proposed control chart performs better than the existing control chart based on repetitive sampling and that using multiple dependent state sampling in terms of ARLs. A real example and a simulation study are added to explain the procedure and to demonstrate the power of the proposed control chart.

Multivariate Poisson hidden Markov models with a case study of modelling seismicity

SummaryWhile the literature on multivariate models for continuous data flourishes, there is a lack of models for multivariate counts. We aim to contribute to this framework by extending the well known class of univariate hidden Markov models to the multidimensional case, by introducing multivariate Poisson hidden Markov models. Each state of the extended model is associated with a different multivariate discrete distribution. We consider different distributions with Poisson marginals, starting from the multivariate Poisson distribution and then extending to copula based distributions to allow flexible dependence structures. An EM type algorithm is developed for maximum likelihood estimation. A real data application is presented to illustrate the usefulness of the proposed models. In particular, we apply the models to the occurrence of strong earthquakes (surface wave magnitude ≥5), in three seismogenic subregions in the broad region of the North Aegean Sea for the time period from 1 January 1981 to 31 December 2008. Earthquakes occurring in one subregion may trigger events in adjacent ones and hence the observed time series of events are cross‐correlated. It is evident from the results that the three subregions interact with each other at times differing by up to a few months. This migration of seismic activity is captured by the model as a transition to a state of higher seismicity.

Interpoint Distance Classification of High Dimensional Discrete Observations

SummaryClassification is a multivariate technique that is concerned with allocating new observations to two or more groups. We use interpoint distances to measure the closeness of the samples and construct new rules for high dimensional classification of discrete observations. Applicable to high dimensional data, the new method is non‐parametric and uses test‐based classification with permutation testing. We propose a modification of a test‐based rule to use relative values with respect to the training samples baseline. We compare the proposed rule with parametric methods, such as likelihood ratio rule and modified linear discriminate function, and non‐parametric techniques such as support vector machine, nearest neighbour and depth‐based classification, under multivariate Bernoulli, multinomial and multivariate Poisson distributions.

Type I multivariate zero-truncated/adjusted Poisson distributions with applications

Although there is substantial literature on the univariate zero-truncated Poisson (ZTP) distributions, few works have been done for developing multivariate ZTP distributions with dimension larger than two. In this paper, we first propose a new Type I multivariate ZTP distribution to model multivariate zero-truncated count data and develop its important distributional properties, while providing efficient statistical inference methods. The key idea for extending the univariate ZTP distribution to a multivariate version is to identify a new stochastic representation of the ZTP random variable, resulting in a novel expectation–maximization algorithm with a feature that the introduced latent variables are independent of the observed variables. Since the ZTP distribution is closely related with the zero-adjusted Poisson (ZAP) distribution, the second objective of this paper is to propose a new family of Type I multivariate ZAP distributions (including the Type I multivariate ZTP distribution, the Type I multivariate zero-inflated Poisson (ZIP) distribution, the Type I multivariate zero-deflated Poisson (ZDP) distribution as its special cases) by accounting for truncation, inflation and deflation at zero. The corresponding distributional properties are explored and the associated statistical methods are provided. Finally, three real data sets are used to illustrate the proposed distributions and corresponding analysis methods.

A new multivariate zero-adjusted Poisson model with applications to biomedicine.

Recently, although advances were made on modeling multivariate count data, existing models really has several limitations: (i) The multivariate Poisson log-normal model (Aitchison and Ho, 1989) cannot be used to fit multivariate count data with excess zero-vectors; (ii) The multivariate zero-inflated Poisson (ZIP) distribution (Li etal., 1999) cannot be used to model zero-truncated/deflated count data and it is difficult to apply to high-dimensional cases; (iii) The Type I multivariate zero-adjusted Poisson (ZAP) distribution (Tian etal., 2017) could only model multivariate count data with a special correlation structure for random components that are all positive or negative. In this paper, we first introduce a new multivariate ZAP distribution, based on a multivariate Poisson distribution, which allows the correlations between components with a more flexible dependency structure, that is some of the correlation coefficients could be positive while others could be negative. We then develop its important distributional properties, and provide efficient statistical inference methods for multivariate ZAP model with or without covariates. Two real data examples in biomedicine are used to illustrate the proposed methods.

A Review of Multivariate Distributions for Count Data Derived from the Poisson Distribution.

The Poisson distribution has been widely studied and used for modeling univariate count-valued data. Multivariate generalizations of the Poisson distribution that permit dependencies, however, have been far less popular. Yet, real-world high-dimensional count-valued data found in word counts, genomics, and crime statistics, for example, exhibit rich dependencies, and motivate the need for multivariate distributions that can appropriately model this data. We review multivariate distributions derived from the univariate Poisson, categorizing these models into three main classes: 1) where the marginal distributions are Poisson, 2) where the joint distribution is a mixture of independent multivariate Poisson distributions, and 3) where the node-conditional distributions are derived from the Poisson. We discuss the development of multiple instances of these classes and compare the models in terms of interpretability and theory. Then, we empirically compare multiple models from each class on three real-world datasets that have varying data characteristics from different domains, namely traffic accident data, biological next generation sequencing data, and text data. These empirical experiments develop intuition about the comparative advantages and disadvantages of each class of multivariate distribution that was derived from the Poisson. Finally, we suggest new research directions as explored in the subsequent discussion section.