Variational hidden conditional random fields with beta processes

Abstract

Hidden conditional random fields (HCRFs) are an effective method for sequential classification. It extends the conditional random fields (CRFs) by introducing latent variables to represent the hidden states, which helps to learn the hidden structures in the sequential data. In order to enhance the flexibility of the HCRF, Dirichlet processes (DPs) are employed as priors of the state transition probabilities, which allows the model to have countable infinite hidden states. Besides DPs, Beta processes (BPs) are another kinds of prior models for Bayesian nonparametric modeling, which are more suitable for latent feature models. In this paper, we propose a novel Bayesian nonparametric version of the HCRF referred as BP-HCRF, which takes the advantages of the BPs on modeling hidden states. In the BP-HCRF, BPs are employed as priors for the state indicator variables for each sequence, and the modeled sequences can have different state spaces with infinite hidden states. We develop a variational inference approach for the BP-HCRF using the stick-breaking construction of BPs. We conduct experiments on synthetic dataset to demonstrate the effectiveness of our proposed model.