Exit Frequency Matrices for Finite Markov Chains

Abstract

Consider a finite irreducible Markov chain on state spaceSwith transition matrixMand stationary distribution π. LetRbe the diagonal matrix of return times,Rii= 1/πi. Given distributions σ, τ andk∈S, the exit frequencyxk(σ, τ) denotes the expected number of times a random walk exits statekbefore an optimal stopping rule from σ to τ halts the walk. For a target distribution τ, we defineXτas then×nmatrix given by (Xτ)ij=xj(i, τ), whereialso denotes the singleton distribution on statei.The dual Markov chain with transition matrix=RM⊤R−1is called thereverse chain. We prove that Markov chain duality extends to matrices of exit frequencies. Specifically, for each target distribution τ, we associate a unique dual distribution τ*. Let$\rX_{\fc{\t}}$denote the matrix of exit frequencies from singletons to τ* on the reverse chain. We show that$\rX_{\fc{\t}} = R (X_{\t}^{\top} - \vb^{\top} \one)R^{-1}$, wherebis a non-negative constant vector (depending on τ). We explore this exit frequency duality and further illuminate the relationship between stopping rules on the original chain and reverse chain.