On the Conditional Rényi Entropy

Abstract

The Renyi entropy of general order unifies the well-known Shannon entropy with several other entropy notions, like the min-entropy or collision entropy. In contrast to the Shannon entropy, there seems to be no commonly accepted definition for the conditional Renyi entropy: several versions have been proposed and used in the literature. In this paper, we reconsider the definition for the conditional Renyi entropy of general order as proposed by Arimoto in the seventies. We show that this particular notion satisfies several natural properties. In particular, we show that it satisfies monotonicity under conditioning, meaning that conditioning can only reduce the entropy, and (a weak form of) chain rule, which implies that the decrease in entropy due to conditioning is bounded by the number of bits one conditions on. None of the other suggestions for the conditional Renyi entropy satisfies both these properties. Finally, we show a natural interpretation of the conditional Renyi entropy in terms of (unconditional) Renyi divergence, and we show consistency with a recently proposed notion of conditional Renyi entropy in the quantum setting.