The Posterior Distribution of Bayesian Context-Tree Models: Theory and Applications

Abstract

The Context-Tree Weighting (CTW) algorithm and the accompanying collection of ideas and techniques have a long history of statistical applications in discrete time series analysis. CTW was recently revisited from a principled Bayesian statistics point of view, and a general modelling framework called Bayesian Context Trees (BCT) was introduced and found to be very effective in numerous core statistical tasks. In this work, a novel representation of the induced BCT posterior distribution on model space is derived in terms of a simple branching process, and several consequences of this are explored in theory and in practice. First, it is shown that it leads to a simple variable-dimensional Monte Carlo sampler for the joint posterior on models and parameters, which is found to be more efficient than earlier MCMC samplers for the same tasks. Then the branching process representation is used to establish the asymptotic consistency of the BCT posterior, including the derivation of an almost-sure convergence rate.