Twice-universal fixed to variable-length random number generators for finite memory sources

Abstract

We study fixed to variable-length random number generators (FVRs) that input a fixed number of symbols from a finite memory source of arbitrary order and unknown parameters, and output a number uniformly distributed in {0, 1,..., M-1}, where M is also random. We review Elias's FVR in the context of the method of types, and show that it remains universal and optimal in the broad class of k-th order finite memory processes. We precisely characterize, up to an additive constant, the expected output length of the optimal FVR, and show that it includes a model cost term similar to those encountered in universal data compression and universal simulation. We further define twice-universal FVRs, which produce quasi-uniform distributions when the input is a finite memory source of unknown order and parameters. We propose a twice-universal FVR whose expected output length is the same, up to an additive constant, as that of an optimal FVR constructed with knowledge of the order k, with the distance of the output to a uniform distribution vanishing exponentially fast with the input length.