Cross-lingual Sentiment Analysis Research Articles

Improving Cross-lingual Aspect-based Sentiment Analysis with Sememe Bridge

Improving Cross-lingual Aspect-based Sentiment Analysis with Sememe Bridge

A multimodal approach to cross-lingual sentiment analysis with ensemble of transformer and LLM

Sentiment analysis is an essential task in natural language processing that involves identifying a text’s polarity, whether it expresses positive, negative, or neutral sentiments. With the growth of social media and the Internet, sentiment analysis has become increasingly important in various fields, such as marketing, politics, and customer service. However, sentiment analysis becomes challenging when dealing with foreign languages, particularly without labelled data for training models. In this study, we propose an ensemble model of transformers and a large language model (LLM) that leverages sentiment analysis of foreign languages by translating them into a base language, English. We used four languages, Arabic, Chinese, French, and Italian, and translated them using two neural machine translation models: LibreTranslate and Google Translate. Sentences were then analyzed for sentiment using an ensemble of pre-trained sentiment analysis models: Twitter-Roberta-Base-Sentiment-Latest, bert-base-multilingual-uncased-sentiment, and GPT-3, which is an LLM from OpenAI. Our experimental results showed that the accuracy of sentiment analysis on translated sentences was over 86% using the proposed model, indicating that foreign language sentiment analysis is possible through translation to English, and the proposed ensemble model works better than the independent pre-trained models and LLM.

A Systematic Review of Cross-Lingual Sentiment Analysis: Tasks, Strategies, and Prospects

Traditional methods for sentiment analysis, when applied in a monolingual context, often yield less than optimal results in multilingual settings. This underscores the need for a more thorough exploration of cross-lingual sentiment analysis (CLSA) methodologies to improve analytical effectiveness. CLSA, confronted with obstacles such as linguistic disparities and a lack of resources, seeks to evaluate sentiments across a range of languages. First, the research background, challenges, existing solution ideas and evaluation tasks of CLSA are summarized. Subsequently, new perspectives including different granularity levels, machine translation support, and sentiment transfer strategies perspectives are highlighted. Finally, potential avenues for future research are discussed.

KNetwork: advancing cross-lingual sentiment analysis for enhanced decision-making in linguistically diverse environments

KNetwork: advancing cross-lingual sentiment analysis for enhanced decision-making in linguistically diverse environments

A comparative study of cross-lingual sentiment analysis

This paper presents a detailed comparative study of the zero-shot cross-lingual sentiment analysis. Namely, we use modern multilingual Transformer-based models and linear transformations combined with CNN and LSTM neural networks. We evaluate their performance in Czech, French, and English. We aim to compare and assess the models’ ability to transfer knowledge across languages and discuss the trade-off between their performance and training/inference speed. We build strong monolingual baselines comparable with the current SotA approaches, achieving state-of-the-art results in Czech (96.0% accuracy) and French (97.6% accuracy). Next, we compare our results with the latest large language models (LLMs), i.e., Llama 2 and ChatGPT.We show that the large multilingual Transformer-based XLM-R model consistently outperforms all other cross-lingual approaches in zero-shot cross-lingual sentiment classification, surpassing them by at least 3%. Next, we show that the smaller Transformer-based models are comparable in performance to older but much faster methods with linear transformations. The best-performing model with linear transformation achieved an accuracy of 92.1% on the French dataset, compared to 90.3% received by the smaller XLM-R model. Notably, this performance is achieved with just approximately 0.01 of the training time required for the XLM-R model. It underscores the potential of linear transformations as a pragmatic alternative to resource-intensive and slower Transformer-based models in real-world applications. The LLMs achieved impressive results that are on par or better, at least by 1%–3%, but with additional hardware requirements and limitations. Overall, this study contributes to understanding cross-lingual sentiment analysis and provides valuable insights into the strengths and limitations of cross-lingual approaches for sentiment analysis.

Cross-Lingual Sentiment Analysis: Comparative Study of Opinion Expression Across Different Languages

Understanding the interconnectedness of the Jordanian language, culture, and emotion is the purpose of this research. To accomplish this objective, the study employs cross-lingual sentiment analysis, a method that permits a comprehensive examination of the subject. By employing this analysis method, the study aims to offer a deeper understanding of the intricate relationship between culture, language, and emotion in Jordan. Comparing the manners in which speakers of Arabic and English express emotion in their respective languages was the primary objective of this study. The findings of this research shed light on the intricate relationship between the selection of a language and one's cultural identity, illustrating how this dynamic substantially influences the emotional landscape. This inquiry showcases the substantial impact of linguistic relativity and the inherent emotional intensity inscribed in language through the disclosure of a substantial increase in sentiment ratings for posts written in the English language. The results of this research further demonstrate the significance of the influence of linguistic imperialism on the social classification of feelings. The findings above offer significant insights into the complex dynamics between language, culture, and emotion within a heterogeneous and multicultural society.

Cross-lingual Sentiment Analysis of Tamil Language Using a Multi-stage Deep Learning Architecture

In recent years, sentiment analysis has become a focal point in natural language processing. Cross-lingual sentiment analysis is a particularly demanding yet essential task that seeks to construct models capable of effectively analyzing sentiments across a variety of languages. The primary motivation behind this research is to bridge the gap in current techniques that often struggle to perform well with low-resource languages, due to the scarcity of large, annotated datasets, and their unique linguistic characteristics. In light of these challenges, we propose a novel Multi-Stage Deep Learning Architecture (MSDLA) for cross-lingual sentiment analysis of the Tamil language, a low-resource language. Our approach utilizes transfer learning from a source language with abundant resources to overcome data limitations. Our proposed model significantly outperforms existing methods on the Tamil Movie Review dataset, achieving an accuracy, precision, recall, and F1-score of 0.8772, 0.8614, 0.8825, and 0.8718, respectively. ANOVA statistical comparison demonstrates that the MSDLA’s improvements over other models, including mT5, XLM, mBERT, ULMFiT, BiLSTM, LSTM with Attention, and ALBERT with Hugging Face English Embedding are significant, with p-values all less than 0.005. Ablation studies confirm the importance of both cross-lingual semantic attention and domain adaptation in our architecture. Without these components, the model’s performance drops to 0.8342 and 0.8043 in accuracy, respectively. Furthermore, MSDLA demonstrates robust cross-domain performance on the Tamil News Classification and Thirukkural datasets, achieving an accuracy of 0.8551 and 0.8624, respectively, significantly outperforming the baseline models. These findings illustrate the robustness and efficacy of our approach, making a significant contribution to cross-lingual sentiment analysis techniques, especially for low-resource languages.

How a Deep Contextualized Representation and Attention Mechanism Justifies Explainable Cross-Lingual Sentiment Analysis

The number of applications in sentiment analysis is growing daily, and research in this field is increasing. Despite the rapid growth of data sources in English, low-resource languages suffer from a lack of data for accurate training models. Moreover, users cannot trust such systems without explaining the output. In this study, we propose a cross-lingual deep neural model to improve the accuracy of sentiment analysis for low-resource languages while providing an explainable description of the predictions. The proposed model contains a word representation model where we use XLM-RoBERTa, a pre-trained contextualized transformer-based cross-lingual language model, and a long short-term memory network together with an attention mechanism that helps improve the explainability of the model and detect the informative words that impact text polarity. Our experiments show the superiority of the proposed model compared to the state-of-the-art mono-lingual techniques and cross-lingual models. The results show 0.55% improvement compared to the cross-lingual sentiment analysis proposed by Ghasemi et al. and 15.08% improvement compared to the mono-lingual contextualized sentiment analysis. Moreover, we achieve 0.54% further improvement when using attention mechanisms for enhancing the model with explainability.

Advances in Sentiment Analysis in Deep Learning Models and Techniques

The article investigates the advantages, disadvantages, and areas of research that need more exploration regarding deep learning architectures used in sentiment analysis. These architectures let models learn complex language features from data without explicit feature engineering, changing sentiment analysis. The models' capacity to capture long-range dependencies has improved their context and nuanced expression interpretation, especially in long or metaphorical texts. Deep learning sentiment analysis algorithms have improved, yet they still face obstacles. The complexity of these models raises ethical questions about bias and transparency. They also require huge, annotated datasets and computational resources, which limits their use in resource-constrained contexts. Adopting deep learning models requires balancing performance and practicality. Explore critical deep learning sentiment analysis research gaps. Cross-domain and cross-lingual sentiment analysis requires context- and language-specific models. Textual and non-textual multimodal sentiment analysis offers untapped potential for complex sentiment interpretation. Responsible AI deployment requires model interpretability, robustness against adversarial assaults, and domain consistency. Finally, deep learning and sentiment analysis have changed our knowledge of human emotion. Accuracy and contextual comprehension have improved, but model transparency, data prerequisites, and practical applicability remain issues. Overcoming these restrictions and exploring research gaps will enable responsible sentiment analysis AI innovation.

Semantic Orientation of Crosslingual Sentiments: Employment of Lexicon and Dictionaries

Sentiment Analysis is a modern discipline at the crossroads of data mining and natural language processing. It is concerned with the computational treatment of public moods shared in the form of text over social networking websites. Social media users express their feelings in conversations through cross-lingual terms, intensifiers, enhancers, reducers, symbols, and Net Lingo. However, the generic Sentiment Analysis (SA) research lacks comprehensive coverage about such abstruseness. In particular, they are inapt in the semantic orientation of Crosslingual based code switching, capitalization and accentuation of opinionative text due to the lack of annotated corpora, computational resources, linguistic processing and inefficient machine translation. This study proposes a Heuristic Framework for Crosslingual Sentiment Analysis (HF-CSA) and takes into consideration the NetLingua, code switching, opinion intensifiers, enhancers and reducers in order to cope with intrinsic linguistic peculiarities. The performance of proposed HF-CSA is examined on the Twitter dataset and the robustness of system is assessed on SemEval-2020 task9. The results show that HF-CSA outperformed the existing systems and reached to 71.6% and 76.18% of average accuracy on Clift and SemEval-2020 datasets respectively.

Multi-Label Sentiment Analysis on 100 Languages With Dynamic Weighting for Label Imbalance.

We investigate cross-lingual sentiment analysis, which has attracted significant attention due to its applications in various areas including market research, politics, and social sciences. In particular, we introduce a sentiment analysis framework in multi-label setting as it obeys Plutchik's wheel of emotions. We introduce a novel dynamic weighting method that balances the contribution from each class during training, unlike previous static weighting methods that assign non-changing weights based on their class frequency. Moreover, we adapt the focal loss that favors harder instances from single-label object recognition literature to our multi-label setting. Furthermore, we derive a method to choose optimal class-specific thresholds that maximize the macro-f1 score in linear time complexity. Through an extensive set of experiments, we show that our method obtains the state-of-the-art performance in seven of nine metrics in three different languages using a single model compared with the common baselines and the best performing methods in the SemEval competition. We publicly share our code for our model, which can perform sentiment analysis in 100 languages, to facilitate further research.

Zero-shot learning based cross-lingual sentiment analysis for sanskrit text with insufficient labeled data

Zero-shot learning based cross-lingual sentiment analysis for sanskrit text with insufficient labeled data

Cross-Lingual Knowledge Transferring by Structural Correspondence and Space Transfer.

The cross-lingual sentiment analysis (CLSA) aims to leverage label-rich resources in the source language to improve the models of a resource-scarce domain in the target language, where monolingual approaches based on machine learning usually suffer from the unavailability of sentiment knowledge. Recently, the transfer learning paradigm that can transfer sentiment knowledge from resource-rich languages, for example, English, to resource-poor languages, for example, Chinese, has gained particular interest. Along this line, in this article, we propose semisupervised learning with SCL and space transfer (ssSCL-ST), a semisupervised transfer learning approach that makes use of structural correspondence learning as well as space transfer for cross-lingual sentiment analysis. The key idea behind ssSCL-ST, at a high level, is to explore the intrinsic sentiment knowledge in the target-lingual domain and to reduce the loss of valuable knowledge due to the knowledge transfer via semisupervised learning. ssSCL-ST also features in pivot set extension and space transfer, which helps to enhance the efficiency of knowledge transfer and improve the classification accuracy in the target language domain. Extensive experimental results demonstrate the superiority of ssSCL-ST to the state-of-the-art approaches without using any parallel corpora.

Bangla-BERT: Transformer-Based Efficient Model for Transfer Learning and Language Understanding

The advent of pre-trained language models has directed a new era of Natural Language Processing (NLP), enabling us to create powerful language models. Among these models, Transformer-based models like BERT have grown in popularity due to their cutting-edge effectiveness. However, these models heavily rely on resource-intensive languages, forcing other languages into multilingual models(mBERT). The two fundamental challenges with mBERT become significantly more challenging in a resource-constrained language like Bangla. It was trained on a limited and organized dataset and contained weights for all other languages. Besides, current research on other languages suggests that a language-specific BERT model will exceed multilingual ones. This paper introduces Bangla-BERT^a, a monolingual BERT model for the Bangla language. Despite the limited data available for NLP tasks in Bangla, we perform pre-training on the largest Bangla language model dataset, BanglaLM, which we constructed using 40 GB of text data. Bangla-BERT achieves the highest results in all datasets and vastly improves the state-of-the-art performance in binary linguistic classification, multilabel extraction, and named entity recognition, outperforming multilingual BERT and other previous research. The pre-trained model is assessed against several non-contextual models such as Bangla fasttext and word2vec the downstream tasks. Finally, this model is evaluated by transfer learning based on hybrid deep learning models such as LSTM, CNN, and CRF in NER, and it is observed that Bangla-BERT outperforms state-of-the-art methods. The proposed Bangla-BERT model is assessed by using benchmark datasets, including Banfakenews, Sentiment Analysis on Bengali News Comments, and Cross-lingual Sentiment Analysis in Bengali. Finally, it is concluded that Bangla-BERT surpasses all prior state-of-the-art results by 3.52%, 2.2%, and 5.3%.

English–Welsh Cross-Lingual Embeddings

Cross-lingual embeddings are vector space representations where word translations tend to be co-located. These representations enable learning transfer across languages, thus bridging the gap between data-rich languages such as English and others. In this paper, we present and evaluate a suite of cross-lingual embeddings for the English–Welsh language pair. To train the bilingual embeddings, a Welsh corpus of approximately 145 M words was combined with an English Wikipedia corpus. We used a bilingual dictionary to frame the problem of learning bilingual mappings as a supervised machine learning task, where a word vector space is first learned independently on a monolingual corpus, after which a linear alignment strategy is applied to map the monolingual embeddings to a common bilingual vector space. Two approaches were used to learn monolingual embeddings, including word2vec and fastText. Three cross-language alignment strategies were explored, including cosine similarity, inverted softmax and cross-domain similarity local scaling (CSLS). We evaluated different combinations of these approaches using two tasks, bilingual dictionary induction, and cross-lingual sentiment analysis. The best results were achieved using monolingual fastText embeddings and the CSLS metric. We also demonstrated that by including a few automatically translated training documents, the performance of a cross-lingual text classifier for Welsh can increase by approximately 20 percent points.

Cross Lingual Sentiment Analysis: A Clustering-Based Bee Colony Instance Selection and Target-Based Feature Weighting Approach

The lack of sentiment resources in poor resource languages poses challenges for the sentiment analysis in which machine learning is involved. Cross-lingual and semi-supervised learning approaches have been deployed to represent the most common ways that can overcome this issue. However, performance of the existing methods degrades due to the poor quality of translated resources, data sparseness and more specifically, language divergence. An integrated learning model that uses a semi-supervised and an ensembled model while utilizing the available sentiment resources to tackle language divergence related issues is proposed. Additionally, to reduce the impact of translation errors and handle instance selection problem, we propose a clustering-based bee-colony-sample selection method for the optimal selection of most distinguishing features representing the target data. To evaluate the proposed model, various experiments are conducted employing an English-Arabic cross-lingual data set. Simulations results demonstrate that the proposed model outperforms the baseline approaches in terms of classification performances. Furthermore, the statistical outcomes indicate the advantages of the proposed training data sampling and target-based feature selection to reduce the negative effect of translation errors. These results highlight the fact that the proposed approach achieves a performance that is close to in-language supervised models.

On the Effect of Word Order on Cross-lingual Sentiment Analysis

Current state-of-the-art models for sentiment analysis make use of word order either explicitly by pre-training on a language modeling objective or implicitly by using recurrent neural networks (Rnns) or convolutional networks (Cnns). This is a problem for cross-lingual models that use bilingual embeddings as features, as the difference in word order between source and target languages is not resolved. In this work, we explore reordering as a pre-processing step for sentence-level crosslingual sentiment classification with two language combinations (English-Spanish, English-Catalan). We find that while reordering helps both models, Cnns are more sensitive to local reorderings, while global reordering benefits Rnns.

Developing Cross-lingual Sentiment Analysis of Malay Twitter Data Using Lexicon-based Approach

Sentiment analysis is a process of detecting and classifying sentiments into positive, negative or neutral. Most sentiment analysis research focus on English lexicon vocabularies. However, Malay is still under-resourced. Research of sentiment analysis in Malaysia social media is challenging due to mixed language usage of English and Malay. The objective of this study was to develop a cross-lingual sentiment analysis using lexicon based approach. Two lexicons of languages are combined in the system, then, the Twitter data were collected and the results were determined using graph. The results showed that the classifier was able to determine the sentiments. This study is significant for companies and governments to understand people’s opinion on social network especially in Malay speaking regions.

Cross-lingual sentiment transfer with limited resources

We describe two transfer approaches for building sentiment analysis systems without having gold labeled data in the target language. Unlike previous work that is focused on using only English as the source language and a small number of target languages, we use multiple source languages to learn a more robust sentiment transfer model for 16 languages from different language families. Our approaches explore the potential of using an annotation projection approach and a direct transfer approach using cross-lingual word representations and neural networks. Whereas most previous work relies on machine translation, we show that we can build cross-lingual sentiment analysis systems without machine translation or even high quality parallel data. We have conducted experiments assessing the availability of different resources such as in-domain parallel data, out-of-domain parallel data, and in-domain comparable data. Our experiments show that we can build a robust transfer system whose performance can in some cases approach that of a supervised system.

Structural Correspondence Learning for Cross-Lingual Sentiment Classification with One-to-Many Mappings

Structural correspondence learning (SCL) is an effective method for cross-lingual sentiment classification. This approach uses unlabeled documents along with a word translation oracle to automatically induce task specific, cross-lingual correspondences. It transfers knowledge through identifying important features, i.e., pivot features. For simplicity, however, it assumes that the word translation oracle maps each pivot feature in source language to exactly only one word in target language. This one-to-one mapping between words in different languages is too strict. Also the context is not considered at all. In this paper, we propose a cross-lingual SCL based on distributed representation of words; it can learn meaningful one-to-many mappings for pivot words using large amounts of monolingual data and a small dictionary. We conduct experiments on NLP&CC 2013 cross-lingual sentiment analysis dataset, employing English as source language, and Chinese as target language. Our method does not rely on the parallel corpora and the experimental results show that our approach is more competitive than the state-of-the-art methods in cross-lingual sentiment classification.