Joint semi-supervised learning of Hidden Conditional Random Fields and Hidden Markov Models

Abstract

Although semi-supervised learning has generated great interest for designing classifiers on static patterns, there has been comparatively fewer works on semi-supervised learning for structured outputs and in particular for sequences. We investigate semi-supervised approaches for learning hidden state conditional random fields for sequence classification. We propose a new approach that iteratively learns a pair of discriminative-generative models, namely Hidden Markov Models (HMMs) and Hidden Conditional Random Fields (HCRFs). Our method builds on simple strategies for semi-supervised learning of HMMs and on strategies for initializing HCRFs from HMMs. We investigate the behavior of the method on artificial data and provide experimental results for two real problems, handwritten character recognition and financial chart pattern recognition. We compare our approach with state of the art semi-supervised methods.