Self-attention-based conditional random fields latent variables model for sequence labeling

Abstract

• CRFLV-I and CRFLV-II are proposed for sequence labeling. • Encoding schema is administered in the form of latent variables able to capture structures of hidden variables. • The proposed models make use of different schemas in different scenarios. • We show that our schemas holds strong performance against BIO or BILOU encoding schemas. To process data like text and speech, Natural Language Processing (NLP) is a valuable tool. As on of NLP’s upstream tasks, sequence labeling is a vital part of NLP through techniques like text classification, machine translation, and sentiment analysis. In this paper, our focus is on sequence labeling where we assign semantic labels within input sequences. We present two novel frameworks, namely SA-CRFLV-I and SA-CRFLV-II, that use latent variables within random fields. These frameworks make use of an encoding schema in the form of a latent variable to be able to capture the latent structure in the observed data. SA-CRFLV-I shows the best performance at the sentence level whereas SA-CRFLV-II works best at the word level. In our in-depth experimental results, we compare our frameworks with 4 well-known sequence prediction methodologies which include NER, reference parsing, chunking as well as POS tagging. The proposed frameworks are shown to have better performance in terms of many well-known metrics.