On censored Markov chains, best augmentations and aggregation/disaggregation procedures

Abstract

Consider the stationary distribution of a Markov chain censored to a subset of the original state space. It can be approximated by augmenting the substochastic matrix T representing transition probabilities in this subset into a stochastic matrix and solving for the corresponding stationary distribution. For the case of nearly uncoupled chains, under some mild technical assumptions, we find the best augmentation in the sense of minimizing the resulting error-bound, out of all augmentations which are based solely on the data given by T. Doing that for all submatrices representing transition probabilities inside all the subsets of a nearly uncoupled Markov chain constitutes part of the aggregation step in a standard aggregation/disaggregation procedure for approximating the stationary distribution of the original process. Scope and purpose The model of nearly uncoupled Markov chains concerns a probabilistic model with a large state space. Moreover, the states can be decomposed into a number of subsets such that transitions are more frequent within subsets than between subsets. Such models call for aggregation/disaggregation procedures in which behavior inside a subset is well approximated with data ignoring out-of-subset information. These approximations lead to errors in computing various parameters. This paper suggests an approximation mechanism such that the resulting bounds on the errors are minimal.