Detection of Markov Chain Models from Observed Data using Binary Hypothesis Testing and Consensus

Abstract

In this paper we consider the problem of detecting which Markov chain model generates observed time series data. We consider two Markov chains. The state of the Markov chain cannot be observed directly, only a function of the state can be observed. Using these observations, the aim is to find which of the two Markov chains has generated the observations. We consider two observers. Each observer observes a function of the state of the Markov chains. We formulate a binary hypothesis testing problem for each observer. Each observer makes a decision on the hypothesis based on its observations. Then Observer 1 communicates its decision to Observer 2 and vice-versa. If the decisions are the same, then a consensus has been achieved. If their decisions are different then the binary hypothesis testing problem is continued. This process is repeated until consensus has been achieved. We solve the binary hypothesis testing problem and prove the convergence of the consensus algorithm. The “value” of the information gained through 1-bit communication is discussed along with simulation results.