Information geometry of the family of Markov kernels defined by a context tree

Abstract

We prove that a tree model is an exponential family (e-family) of Markov kernels, if and only if it is an FSMX model. The notion of e-family of Markov kernels was first introduced by Nakagawa and Kanaya ('93) in the one-dimensional case. Then, Nagaoka ('05) gave its established form, and Hayashi & Watanabe ('16) discussed it. A tree model is the Markov model defined by a context tree. It is noted by Weinberger et al., ('95) that tree models are classified into two classes; FSMX models and non-FSMX models, depending on the shape of their context trees. The FSMX model is a tree model and a finite state machine. We further show that, for Markov models, the e-family of Markov kernels is equivalent to the asymptotic e-family, which was introduced by Takeuchi & Barron ('98). Note that Takeuchi & Kawabata ('07) proved that non-FSMX tree models are not asymptotic e-families for the binary alphabet case. This paper enhances their result and reveals the information geometrical properties of tree models.