An efficient preprocessing method for supervised sentiment analysis by converting sentences to numerical vectors: a twitter case study

Abstract

Along with the significant growth of social media, individuals and companies are increasingly receiving public opinions which direct their decisions. Opinion mining, which is considered as a sub-field of natural language processing, information retrieval, and text mining, is the process of understanding the users’ views from their comment, which have been represented as unstructured texts. Emergence of online social media has led to the production of a huge amount of user comments on websites, and thus, has raised opinion mining as a very useful and challenging problem. In this paper, an efficient preprocessing method for opinion mining is presented and will be used for analyzing users’ comments on Twitter social network. For this purpose, different text preprocessing techniques have been used on the dataset to achieve an acceptable standard text. Word2vec method which is a fast and accurate method have been also exploited to convert the words’ arrays to numerical vectors. Machine learning methods, with supervised learning approach, have been applied on the obtained data after this fast and accurate preprocessing phase. Python and RapidMiner have been used to implement different opinion mining methods and the results of these implementations have been compared and evaluated. The experimental results show that the combined use of the preprocessing method of this paper and support vector machine and artificial neural network have the highest accuracy compared to other methods.