Named Entity Recognition over Dialog Dataset Using Pre-trained Transformers

Abstract

Need of natural language processing (NLP) applications and advancement in deep learning (DL) techniques have increased the need of large amount of human readable data leading to interesting research area named entity recognition (NER), which is a sub-task of natural language processing. NER identifies and tags different real-life entities in their pre-defined categories. Pre-defined categories include person, locations, times, organizations, events, etc., depending upon dataset in hand. For different natural language processing applications such as information retrieval (IR), question answering system (QAS), text summarization (TS), and machine translation (MT), NER forms a base work. Performance of earlier NER techniques is good but requires human intervention for forming domain-specific features or rules. Performance of NER system is further improved by application of emerging deep learning models, and also use of NER in dialog system is less explored area. So, in this paper, our aim is to mention different techniques which can be applied to do NER task and fine-tune the prê-trained transformers to work on in-car dialog dataset for NER task and evaluate the performance of system.