Higher-order multivariate Markov chains and their applications

Abstract

Markov chains are commonly used in modeling many practical systems such as queuing systems, manufacturing systems and inventory systems. They are also effective in modeling categorical data sequences. In a conventional n th order multivariate Markov chain model of s chains, and each chain has the same set of m states, the total number of parameters required to set up the model is O ( m ns ) . Such huge number of states discourages researchers or practitioners from using them directly. In this paper, we propose an n th-order multivariate Markov chain model for modeling multiple categorical data sequences such that the total number of parameters are of O ( ns 2 m 2 ) . The proposed model requires significantly less parameters than the conventional one. We develop efficient estimation methods for the model parameters. An application to demand predictions in inventory control is also discussed.