Smooth Renyi Entropy Research Articles

On the Conditional Smooth Rényi Entropy and its Applications in Guessing and Source Coding

A novel definition of the conditional smooth Renyi entropy, which is different from that of Renner and Wolf, is introduced. It is shown that our definition of the conditional smooth Renyi entropy is appropriate for providing lower and upper bounds on the optimal guessing moment in a guessing problem where the guesser is allowed to stop guessing and declare an error. Further a general formula for the optimal guessing exponent is presented. In particular, a single-letterized formula for a mixture of i.i.d. sources is obtained. It is also shown that our definition is appropriate to characterize the optimal exponential moment of the codeword length in the problem of source coding with common side-information available at the encoder and decoder under a constraint on the probability of a decoding error.

Improved Bounds on Lossless Source Coding and Guessing Moments via Rényi Measures

This paper provides upper and lower bounds on the optimal guessing moments of a random variable taking values on a finite set when side information may be available. These moments quantify the number of guesses required for correctly identifying the unknown object and, similarly to Arikan’s bounds, they are expressed in terms of the Arimoto-Renyi conditional entropy. Although Arikan’s bounds are asymptotically tight, the improvement of the bounds in this paper is significant in the non-asymptotic regime. Relationships between moments of the optimal guessing function and the MAP error probability are also established, characterizing the exact locus of their attainable values. The bounds on optimal guessing moments serve to improve non-asymptotic bounds on the cumulant generating function of the codeword lengths for fixed-to-variable optimal lossless source coding without prefix constraints. Non-asymptotic bounds on the reliability function of discrete memoryless sources are derived as well. Relying on these techniques, lower bounds on the cumulant generating function of the codeword lengths are derived, by means of the smooth Renyi entropy, for source codes that allow decoding errors.

A New Unified Method for Fixed-Length Source Coding Problems of General Sources

This paper establishes a new unified method for fixed-length source coding problems of general sources. Specifically, we introduce an alternative definition of the smooth Renyi entropy of order zero, and show a unified approach to present the fixed-length coding rate in terms of this information quantity. Our definition of the smooth Renyi entropy has a clear operational meaning, and hence is easy to calculate for finite block lengths. Further, we represent various e-source coding rate and the strong converse property for general sources in terms of the smooth Renyi entropy, and compare them with the results obtained by Han and Renner et al.

Min- and Max-Relative Entropies and a New Entanglement Monotone

Two new relative entropy quantities, called the min- and max-relative entropies, are introduced and their properties are investigated. The well-known min- and max-entropies, introduced by Renner, are obtained from these. We define a new entanglement monotone, which we refer to as the max-relative entropy of entanglement, and which is an upper bound to the relative entropy of entanglement. We also generalize the min- and max-relative entropies to obtain smooth min-and max-relative entropies. These act as parent quantities for the smooth Renyi entropies (ETH Zurich, Ph.D. dissertation, 2005), and allow us to define the analogues of the mutual information, in the smooth Renyi entropy framework. Further, the spectral divergence rates of the information spectrum approach are shown to be obtained from the smooth min- and max-relative entropies in the asymptotic limit.