Enabling an On-demand Access to Community Sentiments using LSTM RNNs Web Service Architecture

Abstract

Analyzing community response has always played an important role in marketing and development of various products ranging from services to manufacturing. Analyzing the responses and interpreting them helps improve the quality of the product. This task is traditionally done by community managers using surveys and focus groups. One other hands-off solution would be using the technology to analyze sentiments from users’ digital interactions in various online communities. Sentiment Analysis is the use of Natural Language Processing to process text to identify and retrieve sentiments behind the textual data. Sentiment Analysis can be approached either by supervised classification where a machine learning model is trained using labeled data or lexicon-based unsupervised analysis where a sentiment dictionary is used to find the overall sentiment. The idea is to build a system which presents community sentiment results on discussion threads from reddit on demand with the end user not to concern about data processing. We propose a streamlined pipeline architecture for a web application with several closely connected components in the backend that allows users to simply send the reddit discussion thread's URL, which the backend receives and scrapes the comments using the Reddit API, creates a dataset, cleans it and performs sentiment analysis using a RNNs. We used a Long Short-Term Memory (LSTM) a variant of RNN like Bidirectional RNNs and Multi-layered RNNs are also used to get better results.