A Comparative Analysis of ChatGPT-4, ChatGPT-3.5, and Bard (Gemini Pro) in Sarcasm Detection

Abstract

Understanding nuanced human communication like sarcasm is a significant challenge in the rapidly evolving domains of artificial intelligence (AI) and natural language processing (NLP). This study aims to comparatively analyze the sarcasm detection capabilities of three advanced AI models: ChatGPT-4, ChatGPT-3.5, and Bard (Gemini Pro). Utilizing the Sarcasm Corpus V2, focused on general sarcasm, the research involved testing 100 sentences (50 sarcastic and 50 non-sarcastic) with each model to assess their detection accuracy. The results indicated distinct performance variations among the models. ChatGPT-4 and Bard showed a relatively balanced ability in identifying both sarcastic and non-sarcastic sentences, whereas ChatGPT-3.5 exhibited a stronger accuracy in detecting non-sarcastic sentences but struggled with sarcastic ones. Statistical analysis confirmed that the differences in performance were significant (p < 0.05). These findings have critical implications for the development and application of AI in fields requiring nuanced language understanding, such as social media analysis, customer service, and sentiment analysis. The study highlights the varying strengths and weaknesses of current AI models in processing complex linguistic constructs like sarcasm and underscores the need for continued advancements in this area. Conclusively, this research provides valuable insights into the current state of sarcasm detection in AI, contributing to the broader understanding of AI's language processing capabilities. It also opens avenues for future research, particularly in enhancing AI algorithms for improved sarcasm detection across diverse contexts and languages