Do Noisy Customer Reviews Discourage Platform Sellers? Empirical Analysis of an Online Solar Marketplace

Abstract

Problem definition: Customer reviews are essential to online marketplaces. However, reviews typically vary; ratings of a product or service are rarely the same. In many service marketplaces, especially the ones for solar panel installations, supply-side participants are active. That is, a seller must make a proposal to serve each customer. In such marketplaces, it is not clear how (or if) the dispersion in customer reviews affects the seller activity level and number of matches in the marketplace. Our paper examines this by considering both ratings and text reviews. Academic/Practical Relevance: To our knowledge, this is the first paper that empirically studies how the review dispersion affects a seller's activity level and the number of matches in an online marketplace with active sellers. Distinct from literature, we examine the relationship between the review dispersion and supply-side activities in an online service marketplace. Methodology: We collaborated with one of the largest online solar marketplaces in the U.S. that connects potential solar panel adopters with installers. We obtained a unique dataset from the marketplace for 2013 - 2018. We complement this with public datasets. Our analysis uses traditional econometrics methods, a clustering method, and the deep-learning-based natural-language-processing model BERT developed by Google AI. Results: We find that the dispersion in customer reviews has a significant and inverted U-shaped effect on an installer's marketplace activity level. Specifically, the installer's activity level increases with the review dispersion if and only if that dispersion is below a certain threshold. Above that threshold, more dispersion in reviews lowers the installer's activity level. Intuitively, a marketplace operator would favor having more sellers with perfect ratings. In contrast, we identify a significant and inverted U-shaped relationship between the market-level review dispersion and transactions. Managerial Implications: Our findings provide valuable insights to marketplace operators about the implications of review dispersions, and call for innovative platform design.