Markov chains and processes with a prescribed invariant measure

Abstract

Let (E, E) be a measurable space and let η be a probability measure on E. Denote by I(η) the set of Markov kernels P over (E, E) for which η is an invariant measure: η = ηP. We characterize the extreme points of I(η) in this paper. When E is a finite set, I(η) is a compact, convex set of Markov matrices over E and our characterization generalizes the Birkhoff–von Neumann theorem, which asserts that if η is the uniform distribution on E the extreme points of I(η) are the (# E)! permutation matrices. The number of extreme points of I(η) depends in a complicated manner on the entries of η; the case # F = 3 is enumerated explicitly and general results are given on the maximum and minimum numbers of extreme points. For finite E a similar treatment is given of the convex cone I∗(η) of all generator matrices of Markov processes for which η is invariant: its extremal rays are identified.