Log-concave Distributions Research Articles

Reliable simulation of extremely-truncated log-concave distributions

Inverse transform sampling is a general method to generate non-uniform-distributed random numbers, but it can be unstable when simulating extremely truncated distributions. Many famous probability distributions are log-concave; this feature is preserved under truncation, so Devroye's automatic rejection sampler is available for this task. This method is more stable than inverse transform sampling and uses a very simple envelope with an acceptance rate greater than 20%, independent of the distribution. The aim of this paper is threefold: firstly, to warn the public against incorrect simulation of truncated distributions; secondly, to motivate a more extensive use of rejection sampling to mitigate these issues; lastly, to propose Devroye's automatic sampler as a practical standard for log-concave distributions. We illustrate the proposal with simulations based on Tweedie distributions due to their relevance in regression analysis. The proposed sampler is shown to work under more extreme truncations than the inverse transform method.

A note on statistical distances for discrete log-concave measures

In this note we explore how standard statistical distances are equivalent for discrete log-concave distributions. Distances include total variation distance, Wasserstein distance, and f-divergences.

Sharp high-dimensional central limit theorems for log-concave distributions

Sharp high-dimensional central limit theorems for log-concave distributions

Kinetic Langevin MCMC sampling without gradient Lipschitz continuity - the strongly convex case

In this article we consider sampling from log concave distributions in Hamiltonian setting, without assuming that the objective gradient is globally Lipschitz. We propose two algorithms based on monotone polygonal (tamed) Euler schemes, to sample from a target measure, and provide non-asymptotic 2-Wasserstein distance bounds between the law of the process of each algorithm and the target measure. Finally, we apply these results to bound the excess risk optimization error of the associated optimization problem.

Automated generation of initial points for adaptive rejection sampling of log-concave distributions

Automated generation of initial points for adaptive rejection sampling of log-concave distributions

On a Conjecture of Feige for Discrete Log-Concave Distributions

On a Conjecture of Feige for Discrete Log-Concave Distributions

Behavior of the Poincaré constant along the Polchinski renormalization flow

We control the behavior of the Poincaré constant along the Polchinski renormalization flow using a dynamic version of [Formula: see text]-calculus. We also treat the case of higher order eigenvalues. Our method generalizes a method introduced by Klartag and Putterman to analyze the evolution of log-concave distributions along the heat flow. Furthermore, we apply it to general [Formula: see text]-measures and discuss the interpretation in terms of transport maps.

Tyler’s and Maronna’s M-estimators: Non-asymptotic concentration results

Tyler’s and Maronna’s M-estimators, as well as their regularized variants, are popular robust methods to estimate the scatter or covariance matrix of a multivariate distribution. In this work, we study the non-asymptotic behavior of these estimators, for data sampled from a distribution that satisfies one of the following properties: (1) independent sub-Gaussian entries, up to a linear transformation; (2) log-concave distributions; (3) distributions satisfying a convex concentration property. Our main contribution is the derivation of tight non-asymptotic concentration bounds of these M-estimators around a suitably scaled version of the data sample covariance matrix. Prior to our work, non-asymptotic bounds were derived only for Elliptical and Gaussian distributions. Our proof uses a variety of tools from non asymptotic random matrix theory and high dimensional geometry. Finally, we illustrate the utility of our results on two examples of practical interest: sparse covariance and sparse precision matrix estimation.

Functional inequalities for perturbed measures with applications to log-concave measures and to some Bayesian problems

We study functional inequalities (Poincaré, Cheeger, log-Sobolev) for probability measures obtained as perturbations. Several explicit results for general measures as well as log-concave distributions are given. The initial goal of this work was to obtain explicit bounds on the constants in view of statistical applications. These results are then applied to the Langevin Monte-Carlo method used in statistics in order to compute Bayesian estimators.

Concave likelihood-based regression with finite-support response variables.

We propose a unified framework for likelihood-based regression modeling when the response variable has finite support. Our work is motivated by the fact that, in practice, observed data are discrete and bounded. The proposed methods assume a model which includes models previously considered for interval-censored variables with log-concave distributions as special cases. The resulting log-likelihood is concave, which we use to establish asymptotic normality of its maximizer as the number of observations n tends to infinity with the number of parameters d fixed, and rates of convergence of L1 -regularized estimators when the true parameter vector is sparse and d and n both tend to infinity with . We consider an inexact proximal Newton algorithm for computing estimates and give theoretical guarantees for its convergence. The range of possible applications is wide, including but not limited to survival analysis in discrete time, the modeling of outcomes on scored surveys and questionnaires, and, more generally, interval-censored regression. The applicability and usefulness of the proposed methods are illustrated in simulations and dataexamples.

Complexity of zigzag sampling algorithm for strongly log-concave distributions

We study the computational complexity of zigzag sampling algorithm for strongly log-concave distributions. The zigzag process has the advantage of not requiring time discretization for implementation, and that each proposed bouncing event requires only one evaluation of partial derivative of the potential, while its convergence rate is dimension independent. Using these properties, we prove that the zigzag sampling algorithm achieves $\varepsilon$ error in chi-square divergence with a computational cost equivalent to $O\bigl(\kappa^2 d^\frac{1}{2}(\log\frac{1}{\varepsilon})^{\frac{3}{2}}\bigr)$ gradient evaluations in the regime $\kappa \ll \frac{d}{\log d}$ under a warm start assumption, where $\kappa$ is the condition number and $d$ is the dimension.

Efficient sampling in spectrahedra and volume approximation

We present algorithmic, complexity, and implementation results on the problem of sampling points from a spectrahedron, that is, the feasible region of a semidefinite program.Our main tool is geometric random walks. We analyze the arithmetic and bit complexity of certain primitive geometric operations that are based on the algebraic properties of spectrahedra and the polynomial eigenvalue problem. This study leads to the implementation of a broad collection of random walks for sampling from spectrahedra that experimentally show faster mixing times than methods currently employed either in theoretical studies or in applications, including the popular family of Hit-and-Run walks. The different random walks offer a variety of advantages, thus allowing us to efficiently sample from general probability distributions, for example the family of log-concave distributions which arise in numerous applications. We focus on two major applications of independent interest: (i) approximate the volume of a spectrahedron, and (ii) compute the expectation of functions coming from robust optimal control.We exploit efficient linear algebra algorithms and implementations to address the aforementioned computations in very high dimension. In particular, we provide a C++ open source implementation of our methods that scales efficiently, for the first time, up to dimension 200. We illustrate its efficiency on various data sets.

Concentration inequalities for log-concave distributions with applications to random surface fluctuations

We derive two concentration inequalities for linear functions of log-concave distributions: an enhanced version of the classical Brascamp–Lieb concentration inequality and an inequality quantifying log-concavity of marginals in a manner suitable for obtaining variance and tail probability bounds. These inequalities are applied to the statistical mechanics problem of estimating the fluctuations of random surfaces of the ∇φ type. The classical Brascamp–Lieb inequality bounds the fluctuations whenever the interaction potential is uniformly convex. We extend these bounds to the case of convex potentials whose second derivative vanishes only on a zero measure set, when the underlying graph is a d-dimensional discrete torus. The result applies, in particular, to potentials of the form U(x)=|x|p with p>1 and answers a question discussed by Brascamp–Lieb–Lebowitz (In Statistical Mechanics (1975) 379–390, Springer). Additionally, new tail probability bounds are obtained for the family of potentials U(x)=|x|p+x2, p>2. This result answers a question mentioned by Deuschel and Giacomin (Stochastic Process. Appl. 89 (2000) 333–354).

Signaling Games for Log-Concave Distributions: Number of Bins and Properties of Equilibria

We investigate the equilibrium behavior for the decentralized cheap talk problem for real random variables and quadratic cost criteria in which an encoder and a decoder have misaligned objective functions. In prior work, it has been shown that the number of bins in any equilibrium has to be countable, generalizing a classical result due to Crawford and Sobel who considered sources with density supported on [0, 1]. In this paper, we first refine this result in the context of log-concave sources. For sources with two-sided unbounded support, we prove that, for any finite number of bins, there exists a unique equilibrium. In contrast, for sources with semi-unbounded support, there may be a finite upper bound on the number of bins in equilibrium depending on certain conditions stated explicitly. Moreover, we prove that for log-concave sources, the expected costs of the encoder and the decoder in equilibrium decrease as the number of bins increases. Furthermore, for strictly log-concave sources with two-sided unbounded support, we prove convergence to the unique equilibrium under best response dynamics which starts with a given number of bins, making a connection with the classical theory of optimal quantization and convergence results of Lloyd’s method. In addition, we consider more general sources which satisfy certain assumptions on the tail(s) of the distribution and we show that there exist equilibria with infinitely many bins for sources with two-sided unbounded support. Further explicit characterizations are provided for sources with exponential, Gaussian, and compactly-supported probability distributions.

Efficient Quantum State Preparation for the Cauchy Distribution Based on Piecewise Arithmetic

The benefits of the quantum Monte Carlo algorithm heavily rely on the efficiency of the superposition state preparation. So far, most reported Monte Carlo algorithms use Grover's state preparation algorithm which is suitable for efficiently integrable distribution functions. Consequently, most reported works are based on log-concave distributions, such as normal distributions. However, non-log-concave distributions still have many uses, such as in financial modeling. Recently, a new method was proposed that does not need integration to calculate the rotation angle for state preparation. However, performing efficient state preparation is still difficult due to the high cost associated with high precision and low error in the calculation for the rotation angle. Many methods of quantum state preparation use polynomial Taylor approximations to reduce the computation cost. However, Taylor approximations do not work well with heavy-tailed distribution functions that are not bounded exponentially. In this work, we present a method of efficient state preparation for heavy-tailed distribution functions. Specifically, we present a quantum gate-level algorithm to prepare quantum superposition states based on the Cauchy distribution which is a non-log-concave heavy-tailed distribution. Our procedure relies on a piecewise polynomial function instead of a single Taylor approximation to reduce computational cost and increase accuracy. The Cauchy distribution is an even function, so the proposed piecewise polynomial contains only a quadratic term and a constant term to maintain the simplest approximation of an even function. Numerical analysis shows that the required number of subdomains increases linearly as the approximation error decreases exponentially. Furthermore, the computation complexity of the proposed algorithm is independent of the number of subdomains in the quantum implementation of the piecewise function due to quantum parallelism. An example of the proposed algorithm based on a simulation conducted in Qiskit is presented to demonstrate its capability to perform state preparation based on the Cauchy distribution.

Concentration inequalities for ultra log-concave distributions

We establish concentration inequalities in the class of ultra log-concave distributions. In particular, we show that ultra log-concave distributions satisfy Poisson concentration bounds. As an application, we derive concentration bounds for the intrinsic

An upper bound on the variance of scalar multilayer perceptrons for log-concave distributions

In this paper, we give an upper bound on the variance of scalar multilayer perceptrons. The distribution of the input is assumed to be the class of log-concave distributions, which includes the well-known Gaussian distribution. The activation functions of the scalar multilayer perceptrons are assumed to be differentiable and Lipschitz continuous.

Mixing of Hamiltonian Monte Carlo on strongly log-concave distributions: Continuous dynamics

We obtain several quantitative bounds on the mixing properties of an “ideal” Hamiltonian Monte Carlo (HMC) Markov chain for a strongly log-concave target distribution π on Rd. Our main result says that the HMC Markov chain generates a sample with Wasserstein error ϵ in roughly O(κ2log(1ϵ)) steps, where the condition number κ=M2 m2 is the ratio of the maximum M2 and minimum m2 eigenvalues of the Hessian of −log(π). In particular, this mixing bound does not depend explicitly on the dimension d. These results significantly extend and improve previous quantitative bounds on the mixing of ideal HMC, and can be used to analyze more realistic HMC algorithms. The main ingredient of our argument is a proof that initially “parallel” Hamiltonian trajectories contract over much longer steps than would be predicted by previous heuristics based on the Jacobi manifold.

Concentration functions and entropy bounds for discrete log-concave distributions

AbstractTwo-sided bounds are explored for concentration functions and Rényi entropies in the class of discrete log-concave probability distributions. They are used to derive certain variants of the entropy power inequalities.

A note on volume thresholds for random polytopes

We study the expected volume of random polytopes generated by taking the convex hull of independent identically distributed points from a given distribution. We show that for log-concave distributions supported on convex bodies, we need at least exponentially many (in dimension) samples for the expected volume to be significant and that super-exponentially many samples suffice for concave measures when their parameter of concavity is positive.