Identifying the generator matrix of a stationary Markov chain using partially observable data.

Abstract

Given that most states in real-world systems are inaccessible, it is critical to study the inverse problem of an irreversibly stationary Markov chain regarding how a generator matrix can be identified using minimal observations. The hitting-time distribution of an irreversibly stationary Markov chain is first generalized from a reversible case. The hitting-time distribution is then decoded via the taboo rate, and the results show remarkably that under mild conditions, the generator matrix of a reversible Markov chain or a specific case of irreversibly stationary ones can be identified by utilizing observations from all leaves and two adjacent states in each cycle. Several algorithms are proposed for calculating the generator matrix accurately, and numerical examples are presented to confirm their validity and efficiency. An application to neurophysiology is provided to demonstrate the applicability of such statistics to real-world data. This means that partially observable data can be used to identify the generator matrix of a stationary Markov chain.