An empirical study of Multimodal Entity-Based Sentiment Analysis with ChatGPT: Improving in-context learning via entity-aware contrastive learning

Abstract

Multimodal Entity-Based Sentiment Analysis (MEBSA) is an emerging task within sentiment analysis, with the objective of simultaneously detecting entity, sentiment, and entity category from multimodal inputs. Despite achieving promising results, most existing MEBSA studies requires a substantial quantity of annotated data. The acquisition of such data is both costly and time-intensive in practical applications. To alleviate the reliance on annotated data, this work explores the potential of in-context learning (ICL) with a representative large language model, ChatGPT, for the MEBSA task. Specifically, we develop a general ICL framework with task instructions for zero-shot learning, followed by extending it to few-shot learning by incorporating a few demonstration samples in the prompt. To enhance the performance of the ICL framework in the few-shot learning setting, we further develop an Entity-Aware Contrastive Learning model to effectively retrieve demonstration samples that are similar to each test sample. Experiments demonstrate that our developed ICL framework exhibits superior performance over other baseline ICL methods, and is comparable to or even outperforms many existing fine-tuned methods on four MEBSA subtasks.