Optimal non-asymptotic concentration of centered empirical relative entropy in the high-dimensional regime

Abstract

This document establishes the optimal non-asymptotic concentration of the Kullback–Leibler divergence between the empirical distribution and the true distribution around its mean in the regime of K≫n, where K and n are the alphabet size and sample size, respectively.