Name Entity Recognition for Malay Texts Using Cross-Lingual Annotation Projection Approach

Abstract

AbstractCross-lingual annotation projection methods can benefit from rich-resourced languages to improve the performance of Natural Language Processing (NLP) tasks in less-resourced languages. In this research, Malay is experimented as the less-resourced language and English is experimented as the rich-resourced language. The research is proposed to reduce the deadlock in Malay computational linguistic research due to the shortage of Malay tools and annotated corpus by exploiting state-of-the-art English tools. This paper proposes an alignment method known as MEWA (Malay-English Word Aligner) that integrates a Dice Coefficient and bigram string similarity measure with little supervision to automatically recognize three common named entities – person (PER), organization (ORG) and location (LOC). Firstly, the test collection of Malay journalistic articles describing on Indonesian terrorism is established in three volumes – 646, 5413 and 10002 words. Secondly, a comparative study between selected state-of-the-art tools is conducted to evaluate the performance of the tools against the test collection. Thirdly, MEWA is experimented to automatically induced annotations using the test collection and the identified English tool. A total of 93% accuracy rate is achieved in a series of NE annotation projection experiment.KeywordsNatural Language ProcessingNews ArticleName Entity RecognitionTest CollectionEntity RecognitionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.