Word Embeddings with Fuzzy Ontology Reasoning for Feature Learning in Aspect Sentiment Analysis

Abstract

AbstractIn this paper, a hybrid feature learning approach is proposed to employ aspect sentiment analysis on sentence level through contextual discovery of unstructured text data. The proposed approach joins sentiment lexicon with pre-trained BERT (Bidirectional Encoder Representations from Transformers) word embeddings model for feature deep learning and prediction of context words. In addition, fuzzy ontology reasoning is employed for supporting more in-depth feature extraction through representing semantic knowledge by forming relationships between aspects. Subsequently, the extracted sentiment indicators in online user reviews are classified using Bi-LSTM (Bi-directional Long Short-Term Memory) deep learning model so that the context around words are learned and the corresponding meanings are captured both syntactically and semantically. According to the obtained results, the proposed approach outperforms other related feature learning approaches through improving sentence aspect sentiment analysis and accordingly boosting the overall accuracy of sentiment classification. An average accuracy of 96%, AUC score of 94.5%, and F-score of 95% are achieved by the proposed approach considering five public social media datasets of online reviews. The significance of this study is investigating enrichment of extracted features through using BERT transformer with fuzzy ontology in order to improve the performance of aspect-based sentiment analysis while adding the contextual meanings to the prediction task, and extracting the indirect relationships embedded in social data of user reviews.KeywordsSentiment analysisFeature learningBERTNatural Language ProcessingFuzzy ontology