(LISACMT) Language Identification and Sentiment analysis of English-Urdu ‘code-mixed’ text using LSTM

Abstract

Sentiment analysis is the method of identifying and extracting the opinions, attitudes, and polarity indicated in a text. Individuals, organizations and companies make decisions based on the sentiments expressed by users toward products, services, social or cultural issues, and government policies. Advancements in Natural Language Processing and Artificial Intelligence makes text analysis easier. However, text classification is more focused on the English language. The increase in social-media content available on a number of platforms is predominantly informal and noisy. Social-media content influenced by the regional languages makes the text analysis more challenging. A new phenomenon called code-mixing which involves mixing of linguistic units of one language into the utterances of another language is nowadays exhibited on social-media textual content. To deduce useful information from this type of text is a challenging task. This paper presents a language identification and sentiment analysis method for English-Urdu code-mixed text using deep learning approaches. For language identification Artificial Neural network along with character based embedding has been used. A baseline Long Short-Term Memory approach using word based embeddings has been employed for sentiment classification. Both language identification and sentiment classification approaches showed promising results.