Discrete Probability Measures Research Articles

Minimax Estimation of the <inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math> </inline-formula> Distance

We consider the problem of estimating the $L_1$ distance between two discrete probability measures $P$ and $Q$ from empirical data in a nonasymptotic and large alphabet setting. When $Q$ is known and one obtains $n$ samples from $P$, we show that for every $Q$, the minimax rate-optimal estimator with $n$ samples achieves performance comparable to that of the maximum likelihood estimator (MLE) with $n\ln n$ samples. When both $P$ and $Q$ are unknown, we construct minimax rate-optimal estimators whose worst case performance is essentially that of the known $Q$ case with $Q$ being uniform, implying that $Q$ being uniform is essentially the most difficult case. The \emph{effective sample size enlargement} phenomenon, identified in Jiao \emph{et al.} (2015), holds both in the known $Q$ case for every $Q$ and the $Q$ unknown case. However, the construction of optimal estimators for $\|P-Q\|_1$ requires new techniques and insights beyond the approximation-based method of functional estimation in Jiao \emph{et al.} (2015).

Recursive construction of continuum random trees

We introduce a general recursive method to construct continuum random trees (CRTs) from independent copies of a random string of beads, that is, any random interval equipped with a random discrete probability measure, and from related structures. We prove the existence of these CRTs as a new application of the fixpoint method for recursive distribution equations formalised in high generality by Aldous and Bandyopadhyay. We apply this recursive method to show the convergence to CRTs of various tree growth processes. We note alternative constructions of existing self-similar CRTs in the sense of Haas, Miermont and Stephenson, and we give for the first time constructions of random compact $\mathbb{R}$-trees that describe the genealogies of Bertoin’s self-similar growth fragmentations. In forthcoming work, we develop further applications to embedding problems for CRTs, providing a binary embedding of the stable line-breaking construction that solves an open problem of Goldschmidt and Haas.

Density regression using repulsive distributions

ABSTRACTFlexible regression is a traditional motivation for the development of non-parametric Bayesian models. A popular approach for this involves a joint model for responses and covariates, from which the desired result arises by conditioning on the covariates. Many such models involve the convolution of a continuous kernel with some discrete random probability measure defined as an infinite mixture of i.i.d. atoms. Following this strategy, we propose a flexible model that involves the concept of repulsion between atoms. We show that this results in a more parsimonious representation of the regression than the i.i.d. counterpart. The key aspect is that repulsion discourages mixture components that are near each other, thus favouring parsimony. We show that the conditional model retains the repulsive features, thus facilitating interpretation of the resulting flexible regression, and with little or no sacrifice of model fit compared to the infinite mixture case. We show the utility of the methodology by way of a small simulation study and an application to a well-known data set.

Quantitative logarithmic equidistribution of the crucial measures

Let K be an algebraically closed field of characteristic 0 that is complete with respect to a non-Archimedean absolute value, and let $$\phi \in K(z)$$ with $$\deg (\phi )\ge 2$$ . Recently, Rumely introduced a family of discrete probability measures $$\{\nu _{\phi ^n}\}$$ on the Berkovich line $$\mathbf{P }^1_{\text {K}}$$ over K which carry information about the reduction of conjugates of $$\phi $$ . In a previous article, the author showed that the measures $$\nu _{\phi ^n}$$ converge weakly to the canonical measure $$\mu _\phi $$ . In this article, we extend this result to allow test functions which may have logarithmic singularities at the boundary of $$\mathbf{P }^1_{\text {K}}$$ . These integrands play a key role in potential theory, and we apply our main results to show the potential functions attached to $$\nu _{\phi ^n}$$ converge to the potential function attached to $$\mu _\phi $$ , as well as an approximation result for the Lyapunov exponent of $$\phi $$ .

Spectrality of Moran measures with four-element digit sets

Let δE=1#E∑a∈Eδa denote the uniformly discrete probability measure on a finite set E. We prove that the infinite convolution (Moran measure)μb,{Dk}=δb−1D1⁎δb−2D2⁎⋯ admits an orthonormal basis of exponential provided that {Dk}k=1∞ is a uniformly bounded sequence of 4-digit spectral sets, b=2l+1q with q>1 an odd integer, and l sufficiently large (depends on Dk). We also give some examples to illustrate the result.

Multi-Armed Bandit for Species Discovery: A Bayesian Nonparametric Approach

ABSTRACTLet (P1, …, PJ) denote J populations of animals from distinct regions. A priori, it is unknown which species are present in each region and what are their corresponding frequencies. Species are shared among populations and each species can be present in more than one region with its frequency varying across populations. In this article, we consider the problem of sequentially sampling these populations to observe the greatest number of different species. We adopt a Bayesian nonparametric approach and endow (P1, …, PJ) with a hierarchical Pitman–Yor process prior. As a consequence of the hierarchical structure, the J unknown discrete probability measures share the same support, that of their common random base measure. Given this prior choice, we propose a sequential rule that, at every time step, given the information available up to that point, selects the population from which to collect the next observation. Rather than picking the population with the highest posterior estimate of producing a new value, the proposed rule includes a Thompson sampling step to better balance the exploration–exploitation trade-off. We also propose an extension of the algorithm to deal with incidence data, where multiple observations are collected in a time period. The performance of the proposed algorithms is assessed through a simulation study and compared to three other strategies. Finally, we compare these algorithms using a dataset of species of trees, collected from different plots in South America. Supplementary materials for this article are available online.

Dependent Species Sampling Models for Spatial Density Estimation

We consider a novel Bayesian nonparametric model for density estimation with an underlying spatial structure. The model is built on a class of species sampling models, which are discrete random probability measures that can be represented as a mixture of random support points and random weights. Specifically, we construct a collection of spatially dependent species sampling models and propose a mixture model based on this collection. The key idea is the introduction of spatial dependence by modeling the weights through a conditional autoregressive model. We present an extensive simulation study to compare the performance of the proposed model with competitors. The proposed model compares favorably to these alternatives. We apply the method to the estimation of summer precipitation density functions using Climate Prediction Center Merged Analysis of Precipitation data over East Asia.

From probability monads to commutative effectuses

Effectuses have recently been introduced as categorical models for quantum computation, with probabilistic and Boolean (classical) computation as special cases. These ‘probabilistic’ models are called commutative effectuses, and are the focus of attention here. The paper describes the main known ‘probability’ monads: the monad of discrete probability measures, the Giry monad, the expectation monad, the probabilistic power domain monad, the Radon monad, and the Kantorovich monad. It also introduces successive properties that a monad should satisfy so that its Kleisli category is a commutative effectus. The main properties are: partial additivity, strong affineness, and commutativity. It is shown that the resulting commutative effectus provides a categorical model of probability theory, including a logic using effect modules with parallel and sequential conjunction, predicate- and state-transformers, normalisation and conditioning of states.

Gaussian phase transitions and conic intrinsic volumes: Steining the Steiner formula

Intrinsic volumes of convex sets are natural geometric quantities that also play important roles in applications, such as linear inverse problems with convex constraints, and constrained statistical inference. It is a well-known fact that, given a closed convex cone $C\subset\mathbb{R}^{d}$, its conic intrinsic volumes determine a probability measure on the finite set $\{0,1,\ldots,d\}$, customarily denoted by $\mathcal{L}(V_{C})$. The aim of the present paper is to provide a Berry–Esseen bound for the normal approximation of $\mathcal{L}(V_{C})$, implying a general quantitative central limit theorem (CLT) for sequences of (correctly normalised) discrete probability measures of the type $\mathcal{L}(V_{C_{n}})$, $n\geq1$. This bound shows that, in the high-dimensional limit, most conic intrinsic volumes encountered in applications can be approximated by a suitable Gaussian distribution. Our approach is based on a variety of techniques, namely: (1) Steiner formulae for closed convex cones, (2) Stein’s method and second-order Poincare inequality, (3) concentration estimates and (4) Fourier analysis. Our results explicitly connect the sharp phase transitions, observed in many regularised linear inverse problems with convex constraints, with the asymptotic Gaussian fluctuations of the intrinsic volumes of the associated descent cones. In particular, our findings complete and further illuminate the recent breakthrough discoveries by Amelunxen, Lotz, McCoy and Tropp [Inf. Inference 3 (2014) 224–294] and McCoy and Tropp [Discrete Comput. Geom. 51 (2014) 926–963] about the concentration of conic intrinsic volumes and its connection with threshold phenomena. As an additional outgrowth of our work we develop total variation bounds for normal approximations of the lengths of projections of Gaussian vectors on closed convex sets.

Simple and large equivalence relations

We construct ergodic discrete probability measure preserving equivalence relations $\cR$ that has no proper ergodic normal subequivalence relations and no proper ergodic finite-index subequivalence relations. We show that every treeable equivalence relation satisfying a mild ergodicity condition and cost $>1$ surjects onto every countable group with ergodic kernel. Lastly, we provide a simple characterization of normality for subequivalence relations and an algebraic description of the quotient.

Probability approximation schemes for stochastic programs with distributionally robust second-order dominance constraints

Since the pioneering work by Dentcheva and Ruszczyński [Optimization with stochastic dominance constraints, SIAM J. Optim. 14 (2003), pp. 548–566], stochastic programs with second-order dominance constraints (SPSODC) have received extensive discussions over the past decade from theory of optimality to numerical schemes and practical applications. In this paper, we investigate discrete approximation of SPSODC when (a) the true probability is known but continuously distributed and (b) the true probability distribution is unknown but it lies within an ambiguity set of distributions. Differing from the well-known Monte Carlo discretization method, we propose a deterministic discrete approximation scheme due to Pflug and Pichler [Approximations for Probability Distributions and Stochastic Optimization Problems, International Series in Operations Research & Management Science, Vol. 163, Springer, New York, 2011, pp. 343–387] and demonstrate that the discrete probability measure and the ambiguity set of discrete probability measures approximate their continuous counterparts under the Kantorovich metric. Stability analysis of the optimal value and optimal solutions of the resulting discrete optimization problems is presented and some comparative numerical test results are reported.

Spectrality of a class of infinite convolutions

For a finite set D⊂Z and an integer b≥2, we say that (b,D) is compatible with C⊂Z if [e−2πidc/b]d∈D,c∈C is a Hadamard matrix. Let δE=1#E∑a∈Eδa denote the uniformly discrete probability measure on E. We prove that the class of infinite convolution (Moran measure) μb,{Dk}=δb−1D1⁎δb−2D2⁎⋯ is a spectral measure provided that there is a common C⊂Z+ compatible to all the (b,Dk) and C+C⊆{0,1,…,b−1}. We also give some examples to illustrate the hypotheses and results, in particular, the last condition on C is essential.

Spectrality of infinite Bernoulli convolutions

Let { b k − a k } k = 1 ∞ be a sequence of positive integers with upper bound and let δ E be the uniformly discrete probability measure on the finite set E . For 0 < ρ < 1 , the infinite convolution μ ρ , { a k , b k } = δ ρ { a 1 , b 1 } ⁎ δ ρ 2 { a 2 , b 2 } ⁎ ⋯ ⋯ is called an infinite Bernoulli convolution. In this paper, we investigate whenever there exists Λ such that { e − 2 π i λ x } λ ∈ Λ is an orthonormal basis for L 2 ( μ ρ , { a k , b k } ) .

Are Gibbs-Type Priors the Most Natural Generalization of the Dirichlet Process?

Discrete random probability measures and the exchangeable random partitions they induce are key tools for addressing a variety of estimation and prediction problems in Bayesian inference. Here we focus on the family of Gibbs-type priors, a recent elegant generalization of the Dirichlet and the Pitman-Yor process priors. These random probability measures share properties that are appealing both from a theoretical and an applied point of view: (i) they admit an intuitive predictive characterization justifying their use in terms of a precise assumption on the learning mechanism; (ii) they stand out in terms of mathematical tractability; (iii) they include several interesting special cases besides the Dirichlet and the Pitman-Yor processes. The goal of our paper is to provide a systematic and unified treatment of Gibbs-type priors and highlight their implications for Bayesian nonparametric inference. We deal with their distributional properties, the resulting estimators, frequentist asymptotic validation and the construction of time-dependent versions. Applications, mainly concerning mixture models and species sampling, serve to convey the main ideas. The intuition inherent to this class of priors and the neat results they lead to make one wonder whether it actually represents the most natural generalization of the Dirichlet process.

Adaptive multi-element polynomial chaos with discrete measure: Algorithms and application to SPDEs

We develop a multi-element probabilistic collocation method (ME-PCM) for arbitrary discrete probability measures with finite moments and apply it to solve partial differential equations with random parameters. The method is based on numerical construction of orthogonal polynomial bases in terms of a discrete probability measure. To this end, we compare the accuracy and efficiency of five different constructions. We develop an adaptive procedure for decomposition of the parametric space using the local variance criterion. We then couple the ME-PCM with sparse grids to study the Korteweg–de Vries (KdV) equation subject to random excitation, where the random parameters are associated with either a discrete or a continuous probability measure. Numerical experiments demonstrate that the proposed algorithms lead to high accuracy and efficiency for hybrid (discrete–continuous) random inputs.

On a class of $$\sigma $$ σ -stable Poisson–Kingman models and an effective marginalized sampler

We investigate the use of a large class of discrete random probability measures, which is referred to as the class $$\mathcal {Q}$$Q, in the context of Bayesian nonparametric mixture modeling. The class $$\mathcal {Q}$$Q encompasses both the the two-parameter Poisson---Dirichlet process and the normalized generalized Gamma process, thus allowing us to comparatively study the inferential advantages of these two well-known nonparametric priors. Apart from a highly flexible parameterization, the distinguishing feature of the class $$\mathcal {Q}$$Q is the availability of a tractable posterior distribution. This feature, in turn, leads to derive an efficient marginal MCMC algorithm for posterior sampling within the framework of mixture models. We demonstrate the efficacy of our modeling framework on both one-dimensional and multi-dimensional datasets.

Some results of multidimensional discrete probability measures represented by Euler products

Some results of multidimensional discrete probability measures represented by Euler products

Modeling with Normalized Random Measure Mixture Models

The Dirichlet process mixture model and more general mixtures based on discrete random probability measures have been shown to be flexible and accurate models for density estimation and clustering. The goal of this paper is to illustrate the use of normalized random measures as mixing measures in nonparametric hierarchical mixture models and point out how possible computational issues can be successfully addressed. To this end, we first provide a concise and accessible introduction to normalized random measures with independent increments. Then, we explain in detail a particular way of sampling from the posterior using the Ferguson-Klass representation. We develop a thorough comparative analysis for location-scale mixtures that considers a set of alternatives for the mixture kernel and for the nonparametric component. Simulation results indicate that normalized random measure mixtures potentially represent a valid default choice for density estimation problems. As a byproduct of this study an R package to fit these models was produced and is available in the Comprehensive R Archive Network (CRAN).

On the conditional independence implication problem: A lattice-theoretic approach

Conditional independence is a crucial notion in the development of probabilistic systems which are successfully employed in areas such as computer vision, computational biology, and natural language processing. We introduce a lattice-theoretic framework that permits the study of the conditional independence (CI) implication problem relative to the class of discrete probability measures. Semi-lattices are associated with CI statements and a finite, sound and complete inference system relative to semi-lattice inclusions is presented. This system is shown to be (1) sound and complete for inferring general from saturated CI statements and (2) complete for inferring general from general CI statements. We also show that the general probabilistic CI implication problem can be reduced to that for elementary CI statements. The completeness of the inference system together with its lattice-theoretic characterization yields a criterion we can use to falsify instances of the probabilistic CI implication problem as well as several heuristics that approximate this falsification criterion in polynomial time. We also propose a validation criterion based on representing constraints and sets of constraints as sparse 0–1 vectors which encode their semi-lattices. The validation algorithm works by finding solutions to a linear programming problem involving these vectors and matrices. We provide experimental results for this algorithm and show that it is more efficient than related approaches.

Sound approximate reasoning about saturated conditional probabilistic independence under controlled uncertainty

Knowledge about complex events is usually incomplete in practice. We distinguish between random variables that can be assigned a designated marker to model missing data values, and certain random variables to which the designated marker cannot be assigned. The ability to specify an arbitrary set of certain random variables provides an effective mechanism to control the uncertainty in form of missing data values. A finite axiomatization for the implication problem of saturated conditional independence statements is established under controlled uncertainty, relative to discrete probability measures. The completeness proof utilizes special probability models where two assignments have probability one half. The special probability models enable us to establish an equivalence between the implication problem and that of a propositional fragment in Cadoli and Schaerfʼs S-3 logic. Here, the propositional variables in S correspond to the random variables specified to be certain. The duality leads to an almost linear time algorithm to decide implication. It is shown that this duality cannot be extended to cover general conditional independence statements. All results subsume classical reasoning about saturated conditional independence statements as the idealized special case where every random variable is certain. Under controlled uncertainty, certain random variables allow us to soundly approximate classical reasoning about saturated conditional independence statements.