On classification of states in higher order Markov chains

Abstract

Classification of states in a Markov chain plays an important role for understanding various properties of the chain. In this paper, we introduce a scheme to classify states in a higher order Markov chain and establish some fundamental properties for each type of states. These results are developed using tensors, a natural tool for investigating higher order chains. They not only extend existing results on first order chains to higher order chains, but also shed light on some interesting new phenomena arising in the higher order case.