A Hierarchical Bayesian Hidden Markov Model for Multi-Dimensional Discrete Data

Abstract

1.1 Motivation A fundamental problem encountered in many fields is to model data t o given a discrete time-series data sequence ( ) T o o : y , ,L 1 = . This problem is found in diverse fields, such as control systems, robotics, event detection (Motoi et al., 2007), handwriting recognition (Yasuda et al., 2000 ; Funada et al., 2005), and protein structure prediction (Krogh et al., 2001 ; Tusnady & Simon, 1998 ; Kaburagi et al., 2007). The data t o can often be a multidimensional variable exhibiting stochastic activity. A powerful tool for solving such problems is multi-dimensional discrete Hidden Markov Models (HMMs), and the effectiveness of this approach has been demonstrated in numerous studies (Motoi et al., 2007 ; Yasuda et al., 2000 ; Funada et al., 2005 ; Kaburagi et al., 2007). The hidden states of the HMMs are treated as hidden factors for emission of the observed data t o . However, if redundant components having low dependencies on the hidden states are contained in the data t o , these components often have a negative impact on the HMM performance. Overcoming this problem requires a method of quantifying the redundancy (state independence) of these components and/or reducing their influence. In this chapter, we describe an extension of the HMM for these kinds of data sequences within the framework of a hierarchical Bayesian scheme. In this extended model, we introduce commonality hyperparameters to describe the degree of commonality of the emission probabilities among different hidden states (that is, hidden factors of the data t o ). Additionally, there is a one-to-one relationship between each hyperparameter and a component of the data t o . This allows us to identify low-dependency components and to minimize their negative impact. Like other Bayesian HMMs, the extended model requires complicated integrations in the learning and prediction processes, usually involving a posterior distribution. Analytic solutions of these integrations are often intractable or non-trivial due to their inherent