A lower bound on compression of unknown alphabets

Abstract

Many applications call for universal compression of strings over large, possibly infinite, alphabets. However, it has long been known that the resulting redundancy is infinite even for i.i.d. distributions. It was recently shown that the redundancy of the strings’ patterns, which abstract the values of the symbols, retaining only their relative precedence, is sublinear in the blocklength n, hence the per-symbol redundancy diminishes to zero. In this paper we show that pattern redundancy is at least ( 1.5 log 2 e ) n 1 / 3 bits. To do so, we construct a generating function whose coefficients lower bound the redundancy, and use Hayman's saddle-point approximation technique to determine the coefficients’ asymptotic behavior.