Factored Latent-Dynamic Conditional Random Fields for single and multi-label sequence modeling

Abstract

Conditional Random Fields (CRF) are frequently applied for labeling and segmenting sequence data. Morency et al. (2007) introduced hidden state variables in a labeled CRF structure in order to model the latent dynamics within class labels, thus improving the labeling performance. Such a model is known as Latent-Dynamic CRF (LDCRF). We present Factored LDCRF (FLDCRF), a structure that allows multiple latent dynamics of the class labels to interact with each other. Including such latent-dynamic interactions leads to improved labeling performance on single-label and multi-label sequence modeling experiments across two different datasets, viz., UCI gesture phase data and UCI opportunity data. FLDCRF outperforms all state-of-the-art sequence models, viz., CRF, LDCRF, LSTM, LSTM-CRF, Factorial CRF, Coupled CRF and a multi-label LSTM model across experiments in this paper. In addition, FLDCRF offers easier model selection and is more consistent across validation and test data than LSTM models. FLDCRF is also much faster to train compared to LSTM, even without a GPU. FLDCRF outshines the best LSTM model by ∼4% on a single-label task on the UCI gesture phase data and outperforms LSTM models by ∼2% on average on the multi-label sequence tagging experiment on the UCI opportunity data.