Maximum Rényi Entropy Rate

Abstract

The supremum of the Renyi entropy rate over the class of discrete-time stationary stochastic processes, whose marginals are supported by some given set and satisfy some given cost constraint, is computed. Unlike the Shannon entropy, the Renyi entropy of a random vector can exceed the sum of the Renyi entropies of its components, and the supremum is, therefore, typically not achieved by memoryless processes. It is nonetheless related to Shannon’s entropy: when the Renyi parameter exceeds one, the supremum is equal to the corresponding supremum of Shannon’s entropy, and when it is smaller than one, the supremum equals the logarithm of the volume of the support set. A Burg-like supremum of the Renyi entropy rate over the class of stochastic processes, whose autocovariance function begins with some given values, is also solved. It is not achieved by Gauss–Markov processes, but it is nonetheless related to Burg’s supremum: the two are equal when the Renyi parameter exceeds one, and the former is infinite otherwise.