Connecting Distant Entities with Induction through Conditional Random Fields for Named Entity Recognition: Precursor-Induced CRF

Wangjin Lee,Jinwook Choi

doi:10.18653/v1/w18-2402

Abstract

This paper presents a method of designing specific high-order dependency factor on the linear chain conditional random fields (CRFs) for named entity recognition (NER). Named entities tend to be separated from each other by multiple outside tokens in a text, and thus the first-order CRF, as well as the second-order CRF, may innately lose transition information between distant named entities. The proposed design uses outside label in NER as a transmission medium of precedent entity information on the CRF. Then, empirical results apparently demonstrate that it is possible to exploit long-distance label dependency in the original first-order linear chain CRF structure upon NER while reducing computational loss rather than in the second-order CRF.

Highlights

The concept of conditional random fields (CRFs) (John Lafferty, Andrew McCallum, & Fernando Pereira, 2001) has been successfully adapted in many sequence labeling problems (Andrew McCallum & Wei Li, 2003; Fei Sha & Fernando Pereira, 2003; John Lafferty et al, 2001; McDonald & Pereira, 2005)
Even in deep-learning architecture, CRF has been used as a fundamental element in named entity recognition (Lample, Ballesteros, Subramanian, Kawakami, & Dyer, 2016; Liu, Tang, Wang, & Chen, 2017)
The pre-induced CRF takes significantly less time than the second-order CRF while the preinduced CRF exploits longer label transition dependency than the second-order CRF. These results indicate that the precursor-induced CRF, where long-distance dependency is introduced in CRF by label induction, slightly improves the effectiveness in clinical and biomedical named entity recognition (NER) while significantly reducing computational cost rather than building second- or higher-order CRFs

Summary

Introduction

The concept of conditional random fields (CRFs) (John Lafferty, Andrew McCallum, & Fernando Pereira, 2001) has been successfully adapted in many sequence labeling problems (Andrew McCallum & Wei Li, 2003; Fei Sha & Fernando Pereira, 2003; John Lafferty et al, 2001; McDonald & Pereira, 2005). A CRF in named entity recognition (NER) cannot fully capture dependencies between named entity (NE) labels. High-order interdependencies of named entities between successive outside tokens are not captured by first-order or second-order transition factors. Dependencies between neighbor labels are generally used in practice because conventional high-order CRFs are known to be intractable in NER (Ye, Lee, Chieu, & Wu, 2009). C 2018 Association for Computational Linguistics kens, this study explores the method which modifies the first-order linear-chain CRF by using the induction method.

Precursor-induced CRF

Experiments