Movie Review Sentiment Analysis: Supervised Learning versus Large Language Model

Abstract

Sentiment analysis is frequently used to derive insights from natural language. Examples include analysis of textual data to measure brand perception, social media trends, or customer opinion about products. This paper evaluates the performance of three supervised machine learning methods and compares them with the next-generation large language model (LLM), which recently gained popularity with the release of OpenAI ChatGPT. Specifically, we explore the application of Decision Tree, Random Forest, and Support Vector Machine classifiers to a representative sample of 100K movie reviews collected by a well-known website, IMDb.com. Reviews are tagged with numeric ratings, allowing the formulation of a supervised learning problem and exploring the ability to differentiate sentiment between strongly opinionated positive and negative reviews and also, a more challenging problem of differentiating between weakly opinionated positive and negative reviews. Models are tuned to optimize recall and precision in this application, achieving an accuracy score of 0.89 for strong reviews and 0.63 for weak reviews. We then compare the results with ChatGPT, without specialized training, which reaches a perfect accuracy score of 1.00 for strongly opinionated reviews and 0.75 for weakly opinionated reviews, concluding that it outperforms supervised learning approaches but is also imperfect in distinguishing more subtle sentiment in weakly opinionated reviews.