Poisson Dirichlet Distribution Research Articles

Multiple partition structures and harmonic functions on branching graphs

We introduce and study multiple partition structures which are sequences of probability measures on families of Young diagrams subjected to a consistency condition. The multiple partition structures are generalizations of Kingman's partition structures, and are motivated by a problem of population genetics. They are related to harmonic functions and coherent systems of probability measures on a certain branching graph. The vertices of this graph are multiple Young diagrams (or multiple partitions), and the edges depend on the Jack parameter. If the value of the Jack parameter is equal to one the branching graph under considerations reflects the branching rule for the irreducible representations of the wreath product of a finite group with the symmetric group. If the value of the Jack parameter is zero then the coherent systems of probability measures are precisely the multiple partition structures. We establish a bijective correspondence between the set of harmonic functions on the graph and probability measures on the generalized Thoma set. The correspondence is determined by a canonical integral representation of harmonic functions. As a consequence we obtain a representation theorem for multiple partition structures.We give an example of a multiple partition structure which is expected to be relevant for a model of population genetics for the genetic variation of a sample of gametes from a large population. Namely, we construct a probability measure on the wreath product of a finite group with the symmetric group. If the finite group contains the identity element only then it coincides with the well-known Ewens probability measure on the symmetric group. The constructed probability measure defines a multiple partition structure which is a generalization of the Ewens partition structure studied by Kingman. We show that this multiple partition structure can be represented in terms of a multiple analogue of the Poisson-Dirichlet distribution called the multiple Poisson-Dirichlet distribution in the paper.

Multiplicative chaos of the Brownian loop soup

We construct a measure on the thick points of a Brownian loop soup in a bounded domain D of the plane with given intensity $\theta>0$, which is formally obtained by exponentiating the square root of its occupation field. The measure is constructed via a regularisation procedure, in which loops are killed at a fix rate, allowing us to make use of the Brownian multiplicative chaos measures previously considered in [BBK94, AHS20, Jeg20a], or via a discrete loop soup approximation. At the critical intensity $\theta = 1/2$, it is shown that this measure coincides with the hyperbolic cosine of the Gaussian free field, which is closely related to Liouville measure. This allows us to draw several conclusions which elucidate connections between Brownian multiplicative chaos, Gaussian free field and Liouville measure. For instance, it is shown that Liouville-typical points are of infinite loop multiplicity, with the relative contribution of each loop to the overall thickness of the point being described by the Poisson--Dirichlet distribution with parameter $\theta = 1/2$. Conversely, the Brownian chaos associated to each loop describes its microscopic contribution to Liouville measure. Along the way, our proof reveals a surprising exact integrability of the multiplicative chaos associated to a killed Brownian loop soup. We also obtain some estimates on the discrete and continuous loop soups which may be of independent interest.

Learning the two parameters of the Poisson–Dirichlet distribution with a forensic application

AbstractIn forensic science, the rare type match problem arises when the matching characteristic from the suspect and the crime scene is not in the reference database; hence, it is difficult to evaluate the likelihood ratio that compares the defense and prosecution hypotheses. A recent solution consists of modeling the ordered population probabilities according to the two‐parameter Poisson–Dirichlet distribution, which is a well‐known Bayesian nonparametric prior, and plugging the maximum likelihood estimates of the parameters into the likelihood ratio. We demonstrate that this approximation produces a systematic bias that fully Bayesian inference avoids. Motivated by this forensic application, we consider the need to learn the posterior distribution of the parameters that governs the two‐parameter Poisson–Dirichlet using two sampling methods: Markov Chain Monte Carlo and approximate Bayesian computation. These methods are evaluated in terms of accuracy and efficiency. Finally, we compare the likelihood ratio that is obtained by our proposal with the existing solution using a database of Y‐chromosome haplotypes.

Single-Block Recursive Poisson–Dirichlet Fragmentations of Normalized Generalized Gamma Processes

Dong, Goldschmidt and Martin (2006) (DGM) showed that, for 0<α<1, and θ>−α, the repeated application of independent single-block fragmentation operators based on mass partitions following a two-parameter Poisson–Dirichlet distribution with parameters (α,1−α) to a mass partition having a Poisson–Dirichlet distribution with parameters (α,θ) leads to a remarkable nested family of Poisson—Dirichlet distributed mass partitions with parameters (α,θ+r) for r=0,1,2,⋯. Furthermore, these generate a Markovian sequence of α-diversities following Mittag-Leffler distributions, whose ratios lead to independent Beta-distributed variables. These Markov chains are referred to as Mittag-Leffler Markov chains and arise in the broader literature involving Pólya urn and random tree/graph growth models. Here we obtain explicit descriptions of properties of these processes when conditioned on a mixed Poisson process when it equates to an integer n, which has interpretations in a species sampling context. This is equivalent to obtaining properties of the fragmentation operations of (DGM) when applied to mass partitions formed by the normalized jumps of a generalized gamma subordinator and its generalizations. We focus primarily on the case where n=0,1.

Poisson-Dirichlet asymptotics in condensing particle systems

We study measures on random partitions, arising from condensing stochastic particle systems with stationary product distributions. We provide fairly general conditions on the stationary weights, which lead to Poisson-Dirichlet statistics of the condensed phase in the thermodynamic limit. The Poisson-Dirichlet distribution is known to be the unique reversible measure of split-merge dynamics for random partitions, which we use to characterize the limit law. We also establish concentration results for the macroscopic phase, using size-biased sampling techniques and the equivalence of ensembles to characterize the bulk distribution of the system.

Asymptotic behavior of some factorizations of random words

AbstractIn this paper, we consider the normalized lengths of the factors of some factorizations of random words. First, for the Lyndon factorization of finite random words with n independent letters drawn from a finite or infinite totally ordered alphabet according to a general probability distribution, we prove that the limit law of the normalized lengths of the smallest Lyndon factors is a variant of the stickbreaking process. Convergence of the distribution of the lengths of the longest factors to a Poisson–Dirichlet distribution follows. Second, we consider the standard factorization of random Lyndon word: we prove that the distribution of the normalized length of the standard right factor of a random n‐letters long Lyndon word, derived from such an alphabet, converges, when n is large, to: in which denotes the probability of the smallest letter of the alphabet.

Gibbs partitions, Riemann–Liouville fractional operators, Mittag–Leffler functions, and fragmentations derived from stable subordinators

AbstractPitman (2003), and subsequently Gnedin and Pitman (2006), showed that a large class of random partitions of the integers derived from a stable subordinator of index $\alpha\in(0,1)$ have infinite Gibbs (product) structure as a characterizing feature. The most notable case are random partitions derived from the two-parameter Poisson–Dirichlet distribution, $\textrm{PD}(\alpha,\theta)$, whose corresponding $\alpha$-diversity/local time have generalized Mittag–Leffler distributions, denoted by $\textrm{ML}(\alpha,\theta)$. Our aim in this work is to provide indications on the utility of the wider class of Gibbs partitions as it relates to a study of Riemann–Liouville fractional integrals and size-biased sampling, and in decompositions of special functions, and its potential use in the understanding of various constructions of more exotic processes. We provide characterizations of general laws associated with nested families of $\textrm{PD}(\alpha,\theta)$ mass partitions that are constructed from fragmentation operations described in Dong et al. (2014). These operations are known to be related in distribution to various constructions of discrete random trees/graphs in [n], and their scaling limits. A centerpiece of our work is results related to Mittag–Leffler functions, which play a key role in fractional calculus and are otherwise Laplace transforms of the $\textrm{ML}(\alpha,\theta)$ variables. Notably, this leads to an interpretation within the context of $\textrm{PD}(\alpha,\theta)$ laws conditioned on Poisson point process counts over intervals of scaled lengths of the $\alpha$-diversity.

Stein’s method for the Poisson–Dirichlet distribution and the Ewens sampling formula, with applications to Wright–Fisher models

We provide a general theorem bounding the error in the approximation of a random measure of interest—for example, the empirical population measure of types in a Wright–Fisher model—and a Dirichlet process, which is a measure having Poisson–Dirichlet distributed atoms with i.i.d. labels from a diffuse distribution. The implicit metric of the approximation theorem captures the sizes and locations of the masses, and so also yields bounds on the approximation between the masses of the measure of interest and the Poisson–Dirichlet distribution. We apply the result to bound the error in the approximation of the stationary distribution of types in the finite Wright–Fisher model with infinite-alleles mutation structure (not necessarily parent independent) by the Poisson–Dirichlet distribution. An important consequence of our result is an explicit upper bound on the total variation distance between the random partition generated by sampling from a finite Wright–Fisher stationary distribution, and the Ewens sampling formula. The bound is small if the sample size n is much smaller than N1/6log(N)−1/2, where N is the total population size. Our analysis requires a result of separate interest, giving an explicit bound on the second moment of the number of types of a finite Wright–Fisher stationary distribution. The general approximation result follows from a new development of Stein’s method for the Dirichlet process, which follows by viewing the Dirichlet process as the stationary distribution of a Fleming–Viot process, and then applying Barbour’s generator approach.

Exact simulation of two-parameter Poisson-Dirichlet random variables

Consider a random vector $(V_{1}, \dots , V_{n})$ where $\{V_{k}\}_{k=1, \dots , n}$ are the first $n$ components of a two-parameter Poisson-Dirichlet distribution $PD(\alpha , \theta )$. In this paper, we derive a decomposition for the components of the random vector, and propose an exact simulation algorithm to sample from the random vector. Moreover, a special case arises when $\theta /\alpha $ is a positive integer, for which we present a very fast modified simulation algorithm using a compound geometric representation of the decomposition. Numerical examples are provided to illustrate the accuracy and effectiveness of our algorithms.

A note on residual allocation models

Residual allocation models (RAMs) arise in many subjects including Bayesian statistics, combinatorics, ecology, finance, information theory, machine learning, and population genetics. In this paper, we give a brief review of RAM and presents a few examples where the model arises. An extended discussion will focus a concrete model, the GEM distribution, and its ordered analogue, the Poisson-Dirichlet distribution. The paper concludes with a discussion of the GEM process.

A Nonparametric Bayesian Approach to the Rare Type Match Problem

The “rare type match problem” is the situation in which, in a criminal case, the suspect’s DNA profile, matching the DNA profile of the crime stain, is not in the database of reference. Ideally, the evaluation of this observed match in the light of the two competing hypotheses (the crime stain has been left by the suspect or by another person) should be based on the calculation of the likelihood ratio and depends on the population proportions of the DNA profiles that are unknown. We propose a Bayesian nonparametric method that uses a two-parameter Poisson Dirichlet distribution as a prior over the ranked population proportions and discards the information about the names of the different DNA profiles. This model is validated using data coming from European Y-STR DNA profiles, and the calculation of the likelihood ratio becomes quite simple thanks to an Empirical Bayes approach for which we provided a motivation.

A Berry–Esseen theorem for Pitman’s $\alpha $-diversity

This paper is concerned with the study of the random variable $K_n$ denoting the number of distinct elements in a random sample $(X_1, \dots, X_n)$ of exchangeable random variables driven by the two parameter Poisson-Dirichlet distribution, $PD(\alpha,\theta)$. For $\alpha\in(0,1)$, Theorem 3.8 in \cite{Pit(06)} shows that $\frac{K_n}{n^{\alpha}}\stackrel{\text{a.s.}}{\longrightarrow} S_{\alpha,\theta}$ as $n\rightarrow+\infty$. Here, $S_{\alpha,\theta}$ is a random variable distributed according to the so-called scaled Mittag-Leffler distribution. Our main result states that $$ \sup_{x \geq 0} \Big| \ppsf\Big[\frac{K_n}{n^{\alpha}} \leq x \Big] - \ppsf[S_{\alpha,\theta} \leq x] \Big| \leq \frac{C(\alpha, \theta)}{n^{\alpha}} $$ holds with an explicit constant $C(\alpha, \theta)$. The key ingredients of the proof are a novel probabilistic representation of $K_n$ as compound distribution and new, refined versions of certain quantitative bounds for the Poisson approximation and the compound Poisson distribution.

Structure of the Condensed Phase in the Inclusion Process

We establish a complete picture of condensation in the inclusion process in the thermodynamic limit with vanishing diffusion, covering all scaling regimes of the diffusion parameter and including large deviation results for the maximum occupation number. We make use of size-biased sampling to study the structure of the condensed phase, which can extend over more than one lattice site and exhibit an interesting hierarchical structure characterized by the Poisson–Dirichlet distribution. While this approach is established in other areas including population genetics or random permutations, we show that it also provides a powerful tool to analyse homogeneous condensation in stochastic particle systems with stationary product distributions. We discuss the main mechanisms beyond inclusion processes that lead to the interesting structure of the condensed phase, and the connection to other generic particle systems. Our results are exact, and we present Monte-Carlo simulation data and recursive numerics for partition functions to illustrate the main points.

Limiting Distributions of Generalised Poisson–Dirichlet Distributions Based on Negative Binomial Processes

The $$\text {PD}_\alpha ^{(r)}$$ distribution, a two-parameter distribution for random vectors on the infinite simplex, generalises the $$\text {PD}_\alpha $$ distribution introduced by Kingman, to which it reduces when $$r=0$$ . The parameter $$\alpha \in (0,1)$$ arises from its construction based on ratios of ordered jumps of an $$\alpha $$ -stable subordinator, and the parameter $$r>0$$ signifies its connection with an underlying negative binomial process. Herein, it is shown that other distributions on the simplex, including the Poisson–Dirichlet distribution $$\text {PD}(\theta )$$ , occur as limiting cases of $$\text {PD}_\alpha ^{(r)}$$ , as $$r\rightarrow \infty $$ . As a result, a variety of connections with species and gene sampling models, and many other areas of probability and statistics, are made.

An Integral Characterization of the Dirichlet Process

We give a new integral characterization of the Dirichlet process on a general phase space. To do so, we first prove a characterization of the nonsymmetric Beta distribution via size-biased sampling. Two applications are a new characterization of the Dirichlet distribution and a marked version of a classical characterization of the Poisson–Dirichlet distribution via invariance and independence properties.

A spatio-temporal point process model for particle growth

AbstractA spatio-temporal model of particle or star growth is defined, whereby new unit masses arrive sequentially in discrete time. These unit masses are referred to as candidate stars, which tend to arrive in mass-dense regions and then either form a new star or are absorbed by some neighbouring star of high mass. We analyse the system as time increases, and derive the asymptotic growth rate of the number of stars as well as the size of a randomly chosen star. We also prove that the size-biased mass distribution converges to a Poisson–Dirichlet distribution. This is achieved by embedding our model into a continuous-time Markov process, so that new stars arrive according to a marked Poisson process, with locations as marks, whereas existing stars grow as independent Yule processes. Our approach can be interpreted as a Hoppe-type urn scheme with a spatial structure. We discuss its relevance for and connection to models of population genetics, particle aggregation, image segmentation, epidemic spread, and random graphs with preferential attachment.

Reversible measure-valued processes associated with the Poisson-Dirichlet distribution

The Poisson-Dirichlet distribution is a probability on the space of infinite dimensional simplex. It is the distribution of a discrete-probability-valued random variable.The closely associated random measures include the GEM distribution, the Dirichlet process and their two-parameter generalizations.One of the active areas for research in modern probability is concerned with the construction of reversible measure-valued processes associated with these random measures.This article will survey recent development in this area and list a few interesting problems for further consideration.

Ratios of ordered points of point processes with regularly varying intensity measures

We study limiting properties of ratios of ordered points of point processes whose intensity measures have regularly varying tails, giving a systematic treatment which points the way to “large-trimming” properties of extremal processes and a variety of applications. Our point process approach facilitates a connection with the negative binomial process of Gregoire (1984) and consequently to certain generalised versions of the Poisson–Dirichlet distribution.

Moderate Deviations for Ewens-Pitman Sampling Models

Consider a population of individuals belonging to an infinity number of types, and assume that type proportions follow the Poisson-Dirichlet distribution with parameter α ∈ [0,1) and 𝜃 > −α. Given a sample of size n from the population, two important statistics are the number Kn of different types in the sample, and the number Ml,n of different types with frequency l in the sample. We establish moderate deviation principles for (Kn)n≥ 1 and (Ml,n)n≥ 1. Corresponding rate functions are explicitly identified, which help in revealing a critical scale and in understanding the exact role of the parameters α and 𝜃.

When do combinatorial structures obey the Poisson-Dirichlet law?

We attempt to soften the conditions under which the component vector of a random decomposable structure obeys the Poisson-Dirichlet distribution. The so-called Logarithmic Condition is relaxed to an averaged requirement. The results are focused on permutations sampled according to a generalized Ewens probability.