Human-in-the-Loop Based Named Entity Recognition

Abstract

Named Entity Recognition (NER) is a challenging issue in Natural Language Processing (NLP) tasks, and has drawn much attention from industry and academia. At present, with the incessant evolution of Deep Neural Network (DNN) model, it has been widely used in NER tasks. However, DNN models heavily depend on a large amount of annotated training samples, and these models will show certain limitations when applied to domain-specific tasks. In this paper, a Human-in-the-loop based NER (H-NER) approach is proposed from the perspective of human-machine collaboration. In particular, interactive operations allow users to quickly annotate samples and verify the accuracy of annotated samples, and NER model will be updated iteratively based on progressively increased training samples. Experimental results show that this approach can effectively reduce the task load of annotation, and reach or even exceed the existing sample selection strategies in performance indicators such as F1-score (entity-level) and Accuracy (sentence-level).Furthermore, this approach will expand the serviceable range of NER and greatly improve its applicability.