Malay Word Stemmer to Stem Standard and Slang Word Patterns on Social Media

Abstract

Word stemmer is a text preprocessing tool used in many artificial intelligence applications such as text mining, text categorization and information retrieval. It is used to stem derived words into their respective root words. Many researchers have proposed word stemmers for Malay language using various stemming approaches. Since the proliferation of social media, there are various word patterns have been used by social media users in which the existing word stemmers do not support in their stemming rules. These word patterns are slang words or informal conversation words which are used in daily conversation. Therefore, this paper proposes the new word stemmer for Malay language that able to stem standard and slang words. This paper also examines the differences between standard words and slang words. The experimental results show that the proposed word stemmer able to stem standard affixation and reduplication words and also stem slang affixation and reduplication words with better stemming accuracy.