Abstract
Sentiment lexicons are useful to automatically extract sentiment from text. In this paper, we generate several Norwegian sentiment lexicons by extracting sentiment information from two different types of Norwegian text corpus, namely, news corpus and discussions forums. The methodology is based on the Point wise Mutual Information (PMI). We introduce a modification of the PMI that considers small blocks of the text instead of the text as a whole. The rational of this modification is to counter the detrimental effect of the length of the text on the PMI and on the quality of the lexicon. The high computational cost due to the huge amount of textual information to be processed in addition to our modified PMI formula is tackled efficiently by relying heavily on parallelization using Map-Reduce and Mongo DB shards. Movie and product reviews are used to evaluate the generated sentiment lexicons to correctly classify review ratings. The lexicon exhibits a satisfactory performance when evaluated, in particular when considering the context change in the corpuses. In fact, surprisingly enough, sentiment lexicon generated from large News corpus exhibits a satisfactory performance when tested on annotated product and movie reviews despite that the later two have different contexts, bearing similarity to the notion of Transfer Learning [1] reported in the literature. Some suggestions on how to increase the performance are proposed. All the sentiment lexicons are publicly available for those that are interested.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.