Content-Rating Consistency of Online Product Review and Its Impact on Helpfulness: A Fine-Grained Level Sentiment Analysis

Abstract

Aim/Purpose: The objective of this research is to investigate the effect of review consistency between textual content and rating on review helpfulness. A measure of review consistency is introduced to determine the degree to which the review sentiment of textual content conforms with the review rating score. A theoretical model grounded in signaling theory is adopted to explore how different variables (review sentiment, review rating, review length, and review rating variance) affect review consistency and the relationship between review consistency and review helpfulness. Background: Online reviews vary in their characteristics and hence their different quality features and degrees of helpfulness. High-quality online reviews offer consumers the ability to make informed purchase decisions and improve trust in e-commerce websites. The helpfulness of online reviews continues to be a focal research issue regardless of the independent or joint effects of different factors. This research posits that the consistency between review content and review rating is an important quality indicator affecting the helpfulness of online reviews. The review consistency of online reviews is another important requirement for maintaining the significance and perceived value of online reviews. Incidentally, this parameter is inadequately discussed in the literature. A possible reason is that review consistency is not a review feature that can be readily monitored on e-commerce websites. Methodology: More than 100,000 product reviews were collected from Amazon.com and preprocessed using natural language processing tools. Then, the quality reviews were identified, and relevant features were extracted for model training. Machine learning and sentiment analysis techniques were implemented, and each review was assigned a consistency score between 0 (not consistent) and 1 (fully consistent). Finally, signaling theory was employed, and the derived data were analyzed to determine the effect of review consistency on review helpfulness, the effect of several factors on review consistency, and their relationship with review helpfulness. Contribution: This research contributes to the literature by introducing a mathematical measure to determine the consistency between the textual content of online reviews and their associated ratings. Furthermore, a theoretical model grounded in signaling theory was developed to investigate the effect on review helpfulness. This work can considerably extend the body of knowledge on the helpfulness of online reviews, with notable implications for research and practice. Findings: Empirical results have shown that review consistency significantly affects the perceived helpfulness of online reviews. The study similarly finds that review rating is an important factor affecting review consistency; it also confirms a moderating effect of review sentiment, review rating, review length, and review rating variance on the relationship between review consistency and review helpfulness. Overall, the findings reveal the following: (1) online reviews with textual content that correctly explains the associated rating tend to be more helpful; (2) reviews with extreme ratings are more likely to be consistent with their textual content; and (3) comparatively, review consistency more strongly affects the helpfulness of reviews with short textual content, positive polarity textual content, and lower rating scores and variance. Recommendations for Practitioners: E-commerce systems should incorporate a review consistency measure to rank consumer reviews and provide customers with quick and accurate access to the most helpful reviews. Impact on Society: Incorporating a score of review consistency for online reviews can help consumers access the best reviews and make better purchase decisions, and e-commerce systems improve their business, ultimately leading to more effective e-commerce. Future Research: Additional research should be conducted to test the impact of review consistency on helpfulness in different datasets, product types, and different moderating variables.