Markov Chain Tree Theorem Research Articles

Generalized Markov chain tree theorem and Kemeny’s constant for a class of non-Markovian matrices

Given an ergodic Markov chain with transition matrix P and stationary distribution π, the classical Markov chain tree theorem expresses π in terms of graph-theoretic parameters associated with the graph of P. For a class of non-stochastic matrices M2 associated with P, recently introduced by the first author in Choi (2020) and Choi and Huang (2020), we prove a generalized version of Markov chain tree theorem in terms of graph-theoretic quantities of M2. This motivates us to define generalized version of mean hitting time, fundamental matrix and Kemeny’s constant associated with M2, and we show that they enjoy similar properties as their counterparts of P even though M2 is non-stochastic. We hope to shed lights on how concepts and results originated from the Markov chain literature, such as the Markov chain tree theorem, Kemeny’s constant or the notion of hitting time, can possibly be extended and generalized to a broader class of non-stochastic matrices via introducing appropriate graph-theoretic parameters. In particular, when P is reversible, the results of this paper reduce to the results of P.

Tree formulas, mean first passage times and Kemeny’s constant of a Markov chain

In this paper, we aim to provide probabilistic and combinatorial insights into tree formulas for the Green function and hitting probabilities of Markov chains on a finite state space. These tree formulas are closely related to loop-erased random walks by Wilson's algorithm for random spanning trees, and to mixing times by the Markov chain tree theorem. Let $m_{ij}$ be the mean first passage time from $i$ to $j$ for an irreducible chain with finite state space $S$ and transition matrix $(p_{ij}; i, j \in S)$. It is well-known that $m_{jj} = 1/\pi_j = \Sigma^{(1)}/\Sigma_j$, where $\pi$ is the stationary distribution for the chain, $\Sigma_j$ is the tree sum, over $n^{n-2}$ trees $\textbf{t}$ spanning $S$ with root $j$ and edges $i \rightarrow k$ directed to $j$, of the tree product $\prod_{i \rightarrow k \in \textbf{t} }p_{ik}$, and $\Sigma^{(1)}:= \sum_{j \in S} \Sigma_j$. Chebotarev and Agaev derived further results from {\em Kirchhoff's matrix tree theorem}. We deduce that for $i \ne j$, $m_{ij} = \Sigma_{ij}/\Sigma_j$, where $\Sigma_{ij}$ is the sum over the same set of $n^{n-2}$ spanning trees of the same tree product as for $\Sigma_j$, except that in each product the factor $p_{kj}$ is omitted where $k = k(i,j,\textbf{t})$ is the last state before $j$ in the path from $i$ to $j$ in $\textbf{t}$. It follows that Kemeny's constant $\sum_{j \in S} m_{ij}/m_{jj}$ equals to $ \Sigma^{(2)}/\Sigma^{(1)}$, where $\Sigma^{(r)}$ is the sum, over all forests $\textbf{f}$ labeled by $S$ with $r$ trees, of the product of $p_{ij}$ over edges $i \rightarrow j$ of $\textbf{t}$. We show that these results can be derived without appeal to the matrix tree theorem. A list of relevant literature is also reviewed.

Counting trees with random walks

We give a simple proof of Tutte’s matrix-tree theorem, a well-known result providing a closed-form expression for the number of rooted spanning trees in a directed graph. Our proof stems from placing a random walk on a directed graph and then applying the Markov chain tree theorem to count trees. The connection between the two theorems is not new, but it appears that only one direction of the formal equivalence between them is readily available in the literature. The proof we now provide establishes the other direction. More generally, our approach is another example showing that random walks can serve as a powerful glue between graph theory and Markov chain theory, allowing formal statements from one side to be carried over to the other.

Temporally Agnostic Rumor-Source Detection

We revisit the problem of inferring the source of a rumor on a network, given a snapshot of the extent of its spread. We differ from prior work in two aspects: We consider settings where additional relative information about the infection times of a fraction of node pairs is also available to the estimator and instead of only considering the most likely spreading pattern, we take a complementary approach where our estimator for general networks ranks each node based on counting the number of possible spreading patterns with a given node as root that are compatible with the observations. We first consider the case where additional information is available about infection incidents in the form of a set of directed pairs of neighboring nodes with the implication that the first is known to have infected the second. Under this hypothesis, we derive an estimator for the most likely rumor source based on the Markov chain tree theorem and propose a Markov Chain Monte Carlo scheme to find it. Empirical studies of various performance measures are provided, along with comparisons with the popular scheme of Shah and Zaman adapted to this framework. A further variant of the problem considered is when such pairwise temporal precedence is known for a fraction of pairs of nodes which, however, are not necessarily neighbors. For this case, we again propose a Markov Chain Monte Carlo based scheme for rumor source detection, which combines Aldous's algorithm for uniform sampling of arborescences with rejection sampling.

The Markov Chain Tree Theorem in commutative semirings and the State Reduction Algorithm in commutative semifields

We extend the Markov Chain Tree Theorem to general commutative semirings, and we generalize the State Reduction Algorithm to general commutative semifields. This leads to a new universal algorithm, whose prototype is the State Reduction Algorithm which computes the Markov chain tree vector of a stochastic matrix.

On the Markov chain tree theorem in the Max algebra

A max-algebraic analogue of the Markov Chain Tree Theorem is presented, and its connections with the classical Markov Chain Tree Theorem and the max-algebraic spectral theory are investigated.

Forest matrices around the Laplacian matrix

We study the matrices Q k of in-forests of a weighted digraph Γ and their connections with the Laplacian matrix L of Γ. The ( i, j) entry of Q k is the total weight of spanning converging forests ( in-forests) with k arcs such that i belongs to a tree rooted at j. The forest matrices, Q k , can be calculated recursively and expressed by polynomials in the Laplacian matrix; they provide representations for the generalized inverses, the powers, and some eigenvectors of L. The normalized in-forest matrices are row stochastic; the normalized matrix of maximum in-forests is the eigenprojection of the Laplacian matrix, which provides an immediate proof of the Markov chain tree theorem. A source of these results is the fact that matrices Q k are the matrix coefficients in the polynomial expansion of adj( λI+ L). Thereby they are precisely Faddeev’s matrices for − L.

A proof of the Markov chain tree theorem

Let X be a finite set, P be a stochastic matrix on X, and P̄ = lim n → ∞ (1/ n)∑ n−1 k=0 P k . Let G = ( X, E) be the weighted directed graph on X associated to P, with weights p ij . An arborescence is a subset a ⊆ E which has at most one edge out of every node, contains no cycles, and has maximum possible cardinality. The weight of an arborescence is the product of its edge weights. Let A denote the set of all arborescences. Let A ij denote the set of all arborescences which have j as a root and in which there is a directed path from i to j. Let ∥ A ∥, resp. ∥ A ij ∥, be the sum of the weights of the arborescences in A , resp. A ij . The Markov chain tree theorem states that p̄ ij = ∥ A ij ∥/∥ A ∥. We give a proof of this theorem which is probabilistic in nature.

Estimating a probability using finite memory

Let <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\{X_{i}\}_{i=1}^{\infty}</tex> be a sequence of independent Bernoulli random variables with probability <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</tex> that <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X_{i} = 1</tex> and probability <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">q=1-p</tex> that <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X_{i} = 0</tex> for all <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i \geq 1</tex> . Time-invariant finite-memory (i.e., finite-state) estimation procedures for the parameter p are considered which take <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X_{1}, \cdots</tex> as an input sequence. In particular, an n-state deterministic estimation procedure is described which can estimate p with mean-square error <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O(\log n/n)</tex> and an <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</tex> -state probabilistic estimation procedure which can estimate <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</tex> with mean-square error <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O(1/n)</tex> . It is proved that the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O(1/n)</tex> bound is optimal to within a constant factor. In addition, it is shown that linear estimation procedures are just as powerful (up to the measure of mean-square error) as arbitrary estimation procedures. The proofs are based on an analog of the well-known matrix tree theorem that is called the Markov chain tree theorem.