Extending hybrid Conditional Random Fields approach of Named Entity Recognition for Marathi tweets

Abstract

Named Entity Recognition (NER) is gaining popularity in multiple Information Retrieval applications as it facilitates information extraction. Main goal of NER is to obtain named entities which are usually proper nouns, temporal entities and numerical values. Initial Named Entity Recognizers were designed to deal with formal English text. With increased use of social media, many IR and Natural Language Processing based applications designed to get information from short text like tweets. Formal text based standard NER systems fail to deal with such short text due to limited information and presence of noise. Ample work has been done for NER systems which handle English short text. Relatively Indian languages, especially Marathi, need to build dedicated NER systems to extract named entities from tweets. From multiple approaches of NER, a Conditional Random Fields (CRFs) has shown good accuracy for some Indian languages like Hindi. So here we have proposed a dedicated NER system with CRF based hybrid approach to identify NEs from Marathi tweets. Multiple linguistic features are used in addition to Marathi gazetteers to facilitate the task.