Self-Attention Networks and Adaptive Support Vector Machine for aspect-level sentiment classification

Meizhen Liu,Yang Zhao,Jiakai He,Fengyu Zhou,Hongchang Sun,Ke Chen

doi:10.1007/s00500-022-06793-7

Abstract

Aspect-level sentiment classification aims to integrating the context to predict the sentiment polarity of aspect-specific in a text, which has been quite useful and popular, e.g., opinion survey and products’ recommending in e-commerce. Many recent studies exploit a Long Short-Term Memory (LSTM) networks to perform aspect-level sentiment classification, but the limitation of long-term dependencies is not solved well, so that the semantic correlations between each two words of the text are ignored. In addition, traditional classification model adopts SoftMax function based on probability statistics as classifier, but ignores the words’ features in the semantic space. Support Vector Machine (SVM) can fully use the information of characteristics, and it is appropriate to make classification in the high-dimensional space, however, which just considers the maximum distance between different classes and ignores the similarities between different features of the same classes. To address these defects, we propose the two-stage novel architecture named Self Attention Networks and Adaptive SVM (SAN-ASVM) for aspect-level sentiment classification. In the first stage, in order to overcome the long-term dependencies, Multi-Heads Self Attention (MHSA) mechanism is applied to extract the semantic relationships between each two words; furthermore, 1-hop attention mechanism is designed to pay more attention on some important words related to aspect-specific. In the second stage, ASVM is designed to substitute the SoftMax function to perform sentiment classification, which can effectively make multi-classifications in high-dimensional space. Extensive experiments on SemEval2014, SemEval2016 and Twitter datasets are conducted, and compared experiments prove that SAN-ASVM model can obtain better performance.

Highlights

With the advancement of electronic informational technology, more and more people share their comments or experiences on the internet, which can reflect their attitudes on a product or service, so how to mine these contexts and extract the main information has been one of hot researching topics in the field of Natural Language Process (NLP)
We introduce the two-stages novel architecture ASN-Adaptive Support Vector Machine (ASVM) to solve the problem of aspect-level sentiment analysis, in which the sentence’s representation integrating with semantic correlations is generated by ASN and sentiment classification is conducted by ASVM
The goal has been successfully achieved by proposing a novelty two-stages architecture namely GloVe-Self-Attention Network (SAN)-ASVM for sentiment classification that could effectively extract the context’s feature related to aspectspecific and the performance of sentiment classification was improved by ASVM

Summary

Introduction

With the advancement of electronic informational technology, more and more people share their comments or experiences on the internet, which can reflect their attitudes on a product or service, so how to mine these contexts and extract the main information has been one of hot researching topics in the field of Natural Language Process (NLP). Sentiment analysis of aspect-level can excavate semantic correlation with context, but a text consists of several words and the same word may have different semantics in different context. In this text “The speed of computer is so fast that a lot of time is saved”, “fast” is attribute word which means the positive polarity. Another text, “Time has passed so fast, I haven't had time to read this book.”, “fast” implies regret. A method based on aspect-level sentiment analysis should be proposed

Objectives

Methods

Findings

Conclusion