Abstract
A public dataset, with a variety of properties suitable for sentiment analysis [1], event prediction, trend detection and other text mining applications, is needed in order to be able to successfully perform analysis studies. The vast majority of data on social media is text-based and it is not possible to directly apply machine learning processes into these raw data, since several different processes are required to prepare the data before the implementation of the algorithms. For example, different misspellings of same word enlarge the word vector space unnecessarily, thereby it leads to reduce the success of the algorithm and increase the computational power requirement. This paper presents an improved Turkish dataset with an effective spelling correction algorithm based on Hadoop [2]. The collected data is recorded on the Hadoop Distributed File System and the text based data is processed by MapReduce programming model. This method is suitable for the storage and processing of large sized text based social media data. In this study, movie reviews have been automatically recorded with Apache ManifoldCF (MCF) [3] and data clusters have been created. Various methods compared such as Levenshtein and Fuzzy String Matching have been proposed to create a public dataset from collected data. Experimental results show that the proposed algorithm, which can be used as an open source dataset in sentiment analysis studies, have been performed successfully to the detection and correction of spelling errors.
Highlights
Today, most of the social media sources we meet frequently encounter in a very wide range of fields are generate by video sharing (YouTube), photo sharing (Instagram), and location based applications (Foursquare), blogs, microblogs (Twitter), social networks (Facebook)
In this study we propose a method for creating a data set in Turkish to improve the performance of sentiment analysis studies which are based on textual data
Data warehouses and open-source libraries for many languages are included in the literature, a full Turkish data set and library are not open source
Summary
Most of the social media sources we meet frequently encounter in a very wide range of fields are generate by video sharing (YouTube), photo sharing (Instagram), and location based applications (Foursquare), blogs, microblogs (Twitter), social networks (Facebook). A company may collect data from social media for analyzing the opinions of their customers on their products. One of the most important reasons for this is that Twitter users share short and many messages By benefitting from these social media posts one can come up with the distribution of positive and negative thoughts, the tendencies of the targeted customer groups, the reputation and influence status on social media for people or companies. These type of analyses are mainly carried out by applying machine learning techniques on large volumes of data. Different methods are used during the collection of data such as data can be collected by using the application programming interface
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
