Stein's density approach and information inequalities

Christophe Ley,Yvik Swan

doi:10.1214/ecp.v18-2578

Christophe Ley, Yvik Swan

Open Access

https://doi.org/10.1214/ecp.v18-2578

Copy DOI

Abstract

We provide a new perspective on Stein's so-called density approach by introducing a new operator and characterizing class which are valid for a much wider family of probability distributions on the real line. We prove an elementary factorization property of this operator and propose a new Stein identity which we use to derive information inequalities in terms of what we call the generalized Fisher information distance. We provide explicit bounds on the constants appearing in these inequalities for several important cases. We conclude with a comparison between our results and known results in the Gaussian case, hereby improving on several known inequalities from the literature.

Highlights

Charles Stein’s crafty exploitation of the characterizationX ∼ N (0, 1) ⇐⇒ E f (X) − Xf (X) = 0 for all bounded f ∈ C1(R) (1.1)has given birth to a “method” which is an acclaimed tool both in applied and in theoretical probability
While it has long been known that Stein identities such as (1.1) are related to information theoretic tools and concepts, to the best of our knowledge the only references to explore this connection upfront are [4] in the context of compound Poisson approximation, and more recently [32, 31] for Poisson and Bernoulli approximation
In this paper and the companion paper [22] we extend Stein’s characterization of the Gaussian (1.1) to a broad class of univariate distributions and, in doing so, provide an adequate framework in which the connection with information distances becomes transparent

Summary

Introduction

Has given birth to a “method” which is an acclaimed tool both in applied and in theoretical probability. While it has long been known that Stein identities such as (1.1) are related to information theoretic tools and concepts (see, e.g., [19, 21, 13]), to the best of our knowledge the only references to explore this connection upfront are [4] in the context of compound Poisson approximation, and more recently [32, 31] for Poisson and Bernoulli approximation.

The density approach

Stein-type identities and the generalized Fisher information distance

Bounding the constants

Applications

A Bounds for the supremum norm