The effectiveness of online reviews in the presence of self-selection bias

Abstract

Online reviews suffer from self-selection biases. One of these is under-reporting bias: those who have an extreme experience, either positive or negative, are more likely to review a product than those who have a moderate experience, and consequently are overrepresented in the sample of reviews. In this paper we study whether under-reporting bias decreases the effectiveness of the mean star-rating as a criterion for choosing between competing products of different but unobservable qualities. We formulate a model of consumer choice and the decision to review, and simulate this model in a variety of scenarios. We find that if under-reporting bias decreases the effectiveness of the mean star-rating, this is usually only for products that are of either extremely poor or extremely good quality. Even in these cases, the effectiveness is reduced only when the extent of bias is extreme and the variance in quality is low. Otherwise, the presence of under-reporting bias generally does not decrease, and in fact, often enhances the effectiveness of the mean star-rating as a measure of relative quality.