Entrance and escape dynamics for the typical set.

Abstract

According to the asymptotic equipartition property, sufficiently long sequences of random variables converge to a set that is typical. While the size and probability of this set are central to information theory and statistical mechanics, they can often only be estimated accurately in the asymptotic limit due to the exponential growth in possible sequences. Here we derive a time-inhomogeneous dynamics that constructs the properties of the typical set for all finite length sequences of independent and identically distributed random variables. These dynamics link the finite properties of the typical set to asymptotic results and allow the typical set to be applied to small and transient systems. The main result is a geometric mapping-the triangle map-relating sequences of random variables of length n to those of length n+1. We show that the number of points in this map needed to quantify the properties of the typical set grows linearly with sequence length, despite the exponential growth in the number of typical sequences. We illustrate the framework for the Bernoulli process and the Schlögl model for autocatalytic chemical reactions and demonstrate both the convergence to asymptotic limits and the ability to reproduce exact calculations.