Predicting long-term product ratings based on few early ratings and user base analysis

Abstract

Product reviews and ratings are major quality indicators in online shopping systems. In making the decision about carrying a product as part of their inventory, an online seller often pays attention to the product’s rating. However, when the observed average is based on a small number of individual user-submitted ratings, the decision-maker may not feel as confident about the product, even when the average is high. The long-term average rating predictions can help online retailers to identify products to promote on their websites as “top picks”. The paper proposes a Bayesian Network model to predict the long-term average product ratings based on a (limited) number of early submitted ratings. Performance of the proposed model is compared with the performance of five other prediction methods/models; a Linear Regression model, two variations of a Running Average predictor, an Ordered Logistic Regression model and a Confirmatory Factor Analysis model. Each model’s performance is evaluated using the “MovieLens” dataset (GroupLensResearch, 2012). The training is done on 56,590 data points, the ratings submitted for 1155 movies, and the prediction results are reported for 495 movies. It is demonstrated that the proposed Bayesian Network and the Linear Regression models are particularly effective in making accurate predictions around the time of product introduction when the information about the prospective, future ratings is especially valuable. The Bayesian Network model performs better in the very early stage of feedback collection, and in the later stages, as more feedback is received, the Linear Regression model emerges as the best predictor. The strengths and weaknesses of all the assessed prediction methods are discussed.