Discrete Probability Measures Research Articles

The Wasserstein Metric between a Discrete Probability Measure and a Continuous One

This paper examines the Wasserstein metric between the empirical probability measure of n discrete random variables and a continuous uniform measure in the d-dimensional ball, providing an asymptotic estimation of their expectations as n approaches infinity. Furthermore, we investigate this problem within a mixed process framework, where n discrete random variables are generated by the Poisson process.

Hypercontractivity meets random convex hulls: analysis of randomized multivariate cubatures

Given a probability measure μ on a set X and a vector-valued function φ , a common problem is to construct a discrete probability measure on X such that the push-forward of these two probability measures under φ is the same. This construction is at the heart of numerical integration methods that run under various names such as quadrature, cubature or recombination. A natural approach is to sample points from μ until their convex hull of their image under φ includes the mean of φ . Here, we analyse the computational complexity of this approach when φ exhibits a graded structure by using so-called hypercontractivity. The resulting theorem not only covers the classical cubature case of multivariate polynomials, but also integration on pathspace, as well as kernel quadrature for product measures.

A quantitative Heppes theorem and multivariate Bernoulli distributions

Abstract Using some extensions of a theorem of Heppes on finitely supported discrete probability measures, we address the problems of classification and testing based on projections. In particular, when the support of the distributions is known in advance (as for instance for multivariate Bernoulli distributions), a single suitably chosen projection determines the distribution. Several applications of these results are considered.

Finite-dimensional Discrete Random Structures and Bayesian Clustering

Discrete random probability measures stand out as effective tools for Bayesian clustering. The investigation in the area has been very lively, with a strong emphasis on nonparametric procedures based on either the Dirichlet process or on more flexible generalizations, such as the normalized random measures with independent increments (NRMI). The literature on finite-dimensional discrete priors is much more limited and mostly confined to the standard Dirichlet-multinomial model. While such a specification may be attractive due to conjugacy, it suffers from considerable limitations when it comes to addressing clustering problems. In order to overcome these, we introduce a novel class of priors that arise as the hierarchical compositions of finite-dimensional random discrete structures. Despite the analytical hurdles such a construction entails, we are able to characterize the induced random partition and determine explicit expressions of the associated urn scheme and of the posterior distribution. A detailed comparison with (infinite-dimensional) NRMIs is also provided: indeed, informative bounds for the discrepancy between the partition laws are obtained. Finally, the performance of our proposal over existing methods is assessed on a real application where we study a publicly available dataset from the Italian education system comprising the scores of a mandatory nationwide test.

A hierarchically low-rank optimal transport dissimilarity measure for structured data

We develop a class of hierarchically low-rank, scalable optimal transport dissimilarity measures for structured data, bringing the current state-of-the-art optimal transport solvers to a higher level of performance. Given two n-dimensional discrete probability measures supported on two structured grids in \({\mathbb {R}}^d\), we present a fast method for computing an entropically regularized optimal transport distance, referred to as the debiased Sinkhorn distance. The method combines Sinkhorn’s matrix scaling iteration with a low-rank hierarchical representation of the scaling matrices to achieve a near-linear complexity \({{\mathscr {O}}}(n \ln ^4 n)\). This provides a fast, scalable, and easy-to-implement algorithm for computing a class of optimal transport dissimilarity measures, enabling their applicability to large-scale optimization problems, where the computation of the classical Wasserstein metric is not feasible. We carry out a rigorous error-complexity analysis for the proposed algorithm and present several numerical examples to verify the accuracy and efficiency of the algorithm and to demonstrate its applicability in tackling real-world problems.

Semi-discrete optimal transport: hardness, regularization and numerical solution

Semi-discrete optimal transport problems, which evaluate the Wasserstein distance between a discrete and a generic (possibly non-discrete) probability measure, are believed to be computationally hard. Even though such problems are ubiquitous in statistics, machine learning and computer vision, however, this perception has not yet received a theoretical justification. To fill this gap, we prove that computing the Wasserstein distance between a discrete probability measure supported on two points and the Lebesgue measure on the standard hypercube is already #P-hard. This insight prompts us to seek approximate solutions for semi-discrete optimal transport problems. We thus perturb the underlying transportation cost with an additive disturbance governed by an ambiguous probability distribution, and we introduce a distributionally robust dual optimal transport problem whose objective function is smoothed with the most adverse disturbance distributions from within a given ambiguity set. We further show that smoothing the dual objective function is equivalent to regularizing the primal objective function, and we identify several ambiguity sets that give rise to several known and new regularization schemes. As a byproduct, we discover an intimate relation between semi-discrete optimal transport problems and discrete choice models traditionally studied in psychology and economics. To solve the regularized optimal transport problems efficiently, we use a stochastic gradient descent algorithm with imprecise stochastic gradient oracles. A new convergence analysis reveals that this algorithm improves the best known convergence guarantee for semi-discrete optimal transport problems with entropic regularizers.

Superrigidity, measure equivalence, and weak Pinsker entropy

We show that the class $\\mathscr{B}$, of discrete groups which satisfy the conclusion of Popa’s cocycle superrigidity theorem for Bernoulli actions, is invariant under measure equivalence. We generalize this to the setting of discrete probability measure preserving (p.m.p.) groupoids, and as a consequence we deduce that any nonamenable lattice in a product of two noncompact, locally compact second countable groups must belong to $\\mathscr{B}$. We also introduce a measure-conjugacy invariant called weak Pinsker entropy and show that, if $G$ is a group in the class $\\mathscr{B}$, then weak Pinsker entropy is an orbit-equivalence invariant of every essentially free p.m.p. action of $G$.

On the Structure of Optimal Transportation Plans between Discrete Measures

It is well known that the optimal transportation plan between two probability measures mu and nu is induced by a transportation map whenever mu is an absolutely continuous measure supported over a compact set in the Euclidean space and the cost function is a strictly convex function of the Euclidean distance. However, when mu and nu are both discrete, this result is generally false. In this paper, we prove that, given any pair of discrete probability measures and a cost function, there exists an optimal transportation plan that can be expressed as the sum of two deterministic plans, i.e., plans induced by transportation maps. As an application, we estimate the infinity-Wasserstein distance between two discrete probability measures mu and nu with the p-Wasserstein distance, times a constant depending on mu , on nu , and on the fixed cost function.

A column generation approach to the discrete barycenter problem

The discrete Wasserstein barycenter problem is a minimum-cost mass transport problem for a set of discrete probability measures. Although an exact barycenter is computable through linear programming, the underlying linear program can be extremely large. For worst-case input, a best known linear programming formulation is exponential in the number of variables, but has a low number of constraints, making it an interesting candidate for column generation.In this paper, we devise and study two column generation strategies: a natural one based on a simplified computation of reduced costs, and one through a Dantzig–Wolfe decomposition. For the latter, we produce efficiently solvable subproblems, namely, a pricing problem in the form of a classical transportation problem. The two strategies begin with an efficient computation of an initial feasible solution. While the structure of the constraints leads to the computation of the reduced costs of all remaining variables for setup, both approaches may outperform a computation using the full program in speed, and dramatically so in memory requirement. In our computational experiments, we exhibit that, depending on the input, either strategy can become a best choice.

Decentralized Algorithms for Wasserstein Barycenters

In this thesis, we consider the Wasserstein barycenter problem of discrete probability measures from computational and statistical sides in two scenarios: (I) the measures are given and we need to compute their Wasserstein barycenter, and (ii) the measures are generated from a probability distribution and we need to calculate the population barycenter of the distribution defined by the notion of Fr\'echet mean. The statistical focus is estimating the sample size of measures necessary to calculate an approximation for Fr\'echet mean (barycenter) of a probability distribution with a given precision. For empirical risk minimization approaches, the question of the regularization is also studied together with proposing a new regularization which contributes to the better complexity bounds in comparison with quadratic regularization. The computational focus is developing algorithms for calculating Wasserstein barycenters: both primal and dual algorithms which can be executed in a decentralized manner. The motivation for dual approaches is closed-forms for the dual formulation of entropy-regularized Wasserstein distances and their derivatives, whereas the primal formulation has closed-form expression only in some cases, e.g., for Gaussian measures. Moreover, the dual oracle returning the gradient of the dual representation for entropy-regularized Wasserstein distance can be computed for a cheaper price in comparison with the primal oracle returning the gradient of the entropy-regularized Wasserstein distance. The number of dual oracle calls, in this case, will also be less, i.e., the square root of the number of primal oracle calls. This explains the successful application of the first-order dual approaches for the Wasserstein barycenter problem.

Asymptotic Optimality of the Triangular Lattice for a Class of Optimal Location Problems

We prove an asymptotic crystallization result in two dimensions for a class of nonlocal particle systems. To be precise, we consider the best approximation with respect to the 2-Wasserstein metric of a given absolutely continuous probability measure f mathrm {d}x by a discrete probability measure sum _i m_i delta _{z_i}, subject to a constraint on the particle sizes m_i. The locations z_i of the particles, their sizes m_i, and the number of particles are all unknowns of the problem. We study a one-parameter family of constraints. This is an example of an optimal location problem (or an optimal sampling or quantization problem) and it has applications in economics, signal compression, and numerical integration. We establish the asymptotic minimum value of the (rescaled) approximation error as the number of particles goes to infinity. In particular, we show that for the constrained best approximation of the Lebesgue measure by a discrete measure, the discrete measure whose support is a triangular lattice is asymptotically optimal. In addition, we prove an analogous result for a problem where the constraint is replaced by a penalization. These results can also be viewed as the asymptotic optimality of the hexagonal tiling for an optimal partitioning problem. They generalise the crystallization result of Bourne et al. (Commun Math Phys, 329: 117–140, 2014) from a single particle system to a class of particle systems, and prove a case of a conjecture by Bouchitté et al. (J Math Pures Appl, 95:382–419, 2011). Finally, we prove a crystallization result which states that optimal configurations with energy close to that of a triangular lattice are geometrically close to a triangular lattice.

Families of Well Approximable Measures

Abstract We provide an algorithm to approximate a finitely supported discrete measure μ by a measure νN corresponding to a set of N points so that the total variation between μ and νN has an upper bound. As a consequence if μ is a (finite or infinitely supported) discrete probability measure on [0, 1] d with a sufficient decay rate on the weights of each point, then μ can be approximated by νN with total variation, and hence star-discrepancy, bounded above by (log N)N− 1. Our result improves, in the discrete case, recent work by Aistleitner, Bilyk, and Nikolov who show that for any normalized Borel measure μ, there exist finite sets whose star-discrepancy with respect to μ is at most ( log N ) d − 1 2 N − 1 {\left( {\log \,N} \right)^{d - {1 \over 2}}}{N^{ - 1}} . Moreover, we close a gap in the literature for discrepancy in the case d =1 showing both that Lebesgue is indeed the hardest measure to approximate by finite sets and also that all measures without discrete components have the same order of discrepancy as the Lebesgue measure.

Asymptotic behavior of the number of distinct values in a sample from the geometric stick-breaking process

Discrete random probability measures are a key ingredient of Bayesian nonparametric inference. A sample generates ties with positive probability and a fundamental object of both theoretical and applied interest is the corresponding number of distinct values. The growth rate can be determined from the rate of decay of the small frequencies implying that, when the decreasingly ordered frequencies admit a tractable form, the asymptotics of the number of distinct values can be conveniently assessed. We focus on the geometric stick-breaking process and we investigate the effect of the distribution for the success probability on the asymptotic behavior of the number of distinct values. A whole range of logarithmic behaviors are obtained by appropriately tuning the prior. A two-term expansion is also derived and illustrated in a comparison with a larger family of discrete random probability measures having an additional parameter given by the scale of the negative binomial distribution.

The equivalence of Fourier-based and Wasserstein metrics on imaging problems

We investigate properties of some extensions of a class of Fourier-based probability metrics, originally introduced to study convergence to equilibrium for the solution to the spatially homogeneous Boltzmann equation. At difference with the original one, the new Fourier-based metrics are well-defined also for probability distributions with different centers of mass, and for discrete probability measures supported over a regular grid. Among other properties, it is shown that, in the discrete setting, these new Fourier-based metrics are equivalent either to the Euclidean-Wasserstein distance $W_2$, or to the Kantorovich-Wasserstein distance $W_1$, with explicit constants of equivalence. Numerical results then show that in benchmark problems of image processing, Fourier metrics provide a better runtime with respect to Wasserstein ones.

Optimal quantization via dynamics

Quantization for probability distributions refers broadly to estimating a given probability measure by a discrete probability measure supported by a finite number of points. We consider general geometric approaches to quantization using stationary processes arising in dynamical systems, followed by a discussion of the special cases of stationary processes: random processes and Diophantine processes. We are interested in how close stationary process can be to giving optimal n-means and nth optimal mean distortion errors. We also consider different ways of measuring the degree of approximation by quantization, and their advantages and disadvantages in these different contexts.

On the inferential implications of decreasing weight structures in mixture models

Bayesian estimation of nonparametric mixture models strongly relies on available representations of discrete random probability measures. In particular, the order of the mixing weights plays an important role for the identifiability of component-specific parameters which, in turn, affects the convergence properties of posterior samplers. The geometric process mixture model provides a simple alternative to models based on the Dirichlet process that effectively addresses these issues. However, the rate of decay of the mixing weights for this model may be too fast for modeling data with a large number of components. The need for different decay rates arises. Some variants of the geometric process featuring different decay behaviors, while preserving the decreasing structure, are presented and investigated. An asymptotic characterization of the number of distinct values in a sample from the corresponding mixing measure is also given, highlighting the inferential implications of different prior specifications. The analysis is completed by a simulation study in the context of density estimation. It shows that by controlling the decaying rate, the mixture model is able to capture data with a large number of components.

Beta-Binomial stick-breaking non-parametric prior

A new class of nonparametric prior distributions, termed Beta-Binomial stick-breaking process, is proposed. By allowing the underlying length random variables to be dependent through a Beta marginals Markov chain, an appealing discrete random probability measure arises. The chain's dependence parameter controls the ordering of the stick-breaking weights, and thus tunes the model's label-switching ability. Also, by tuning this parameter, the resulting class contains the Dirichlet process and the Geometric process priors as particular cases, which is of interest for fast convergence of MCMC implementations. Some properties of the model are discussed and a density estimation algorithm is proposed and tested with simulated datasets.

Spectrality of Moran measures with finite arithmetic digit sets

Let [Formula: see text] be a prime and [Formula: see text] be a sequence of finite arithmetic digit sets in [Formula: see text] with [Formula: see text] uniformly bounded, and let [Formula: see text] be the discrete probability measure on the finite set [Formula: see text] with equal distribution. For [Formula: see text], the infinite Bernoulli convolution [Formula: see text] converges weakly to a Borel probability measure (Moran measure). In this paper, we study the existence of exponential orthonormal basis for [Formula: see text].

Statistical Physics Approach to the Optimal Transport Problem.

Originally defined for the optimal allocation of resources, optimal transport (OT) has found many theoretical and practical applications in multiple domains of science and physics. In this Letter we develop a new method for solving the discrete version of this problem using techniques derived from statistical physics. We derive a strongly concave free energy function that captures the constraints of the OT problem at a finite temperature. Its maximum defines an optimal transport plan, or registration between the two discrete probability measures that are compared, as well as a pseudodistance between those measures that satisfies the triangular inequalities. The computation of this pseudodistance is fast and numerically stable. The temperature dependent OT pseudodistance is shown to decrease monotonically with respect to the inverse of the temperature and to converge to the standard OT distance at zero temperature, providing a robust framework for temperature annealing. We illustrate applications of this framework to the problem of image comparison.

Distribution theory for hierarchical processes

Hierarchies of discrete probability measures are remarkably popular as nonparametric priors in applications, arguably due to two key properties: (i) they naturally represent multiple heterogeneous populations; (ii) they produce ties across populations, resulting in a shrinkage property often described as “sharing of information.” In this paper, we establish a distribution theory for hierarchical random measures that are generated via normalization, thus encompassing both the hierarchical Dirichlet and hierarchical Pitman–Yor processes. These results provide a probabilistic characterization of the induced (partially exchangeable) partition structure, including the distribution and the asymptotics of the number of partition sets, and a complete posterior characterization. They are obtained by representing hierarchical processes in terms of completely random measures, and by applying a novel technique for deriving the associated distributions. Moreover, they also serve as building blocks for new simulation algorithms, and we derive marginal and conditional algorithms for Bayesian inference.