Stick-Breaking processes, Clumping, and Markov Chain Occupation Laws

Abstract

We connect the empirical or ‘occupation’ laws of certain discrete space time-inhomogeneous Markov chains, related to simulated annealing, to a novel class of ‘stick-breaking’ processes, a ‘nonexchangeable’ generalization of the Dirichlet process used in nonparametric Bayesian statistics. To make this unexpected correspondence, we examine an intermediate ‘clumped’ structure in both the time-inhomogeneous Markov chains and the stick-breaking processes, perhaps of its own interest, which records the sequence of different states visited and the scaled proportions of time spent on them. By matching the associated intermediate structures, we identify the limits of the empirical measures of the time-inhomogeneous Markov chains as types of stick-breaking processes.