On the minimum f-divergence for given total variation

Abstract

We want to find a lower bound for an f-divergence D f in terms of variational distance V which is best possible for any given V. In other words, we want to find L D f ( v ) = inf { D f ( P , Q ) : V ( P , Q ) = v } . In this note we solve this problem for any convex f. Although the form of L D f ( V ) depends on inverting some expressions which may be difficult in general, simplifications can occur when f has some kind of symmetry. For instance, if D f is symmetric in the sense that D f ( P , Q ) = D f ( Q , P ) , we show that L D f ( v ) = 2 − v 2 f ( 2 + v 2 − v ) − f ′ ( 1 ) v . For the Kullback–Leibler divergence K we obtain an expression of L K in terms of the two real branches of Lambert's W function. To cite this article: G.L. Gilardoni, C. R. Acad. Sci. Paris, Ser. I 343 (2006).