How rating mechanisms shape user search, quality inference and engagement in online platforms: Experimental evidence

Abstract

This paper experimentally tests three rating mechanisms on their ability to reduce the asymmetric information between users on the quality of content they evaluate in online platforms. Using a controlled laboratory environment, exogenously imposed variation in content quality and user preferences allows for the construction of a structural model over user search, inference, selection and engagement in the presence of a given rating mechanism. The results of this study suggest that increasing the granularity of the signals available to users increases their ability to make inferences into the quality of content, which causes improvements in user welfare. The experimental data directly revealed that a five-point likert scale rating mechanism improved user welfare by 10.8% over a single ‘like’ button, and counterfactual analysis from the structural model indirectly revealed it improved welfare by 16.2% over providing no rating mechanism at all. User engagement also increased with the granularity of the rating mechanism, which further aided in the reduction of the asymmetric information.