Do Noisy Customer Reviews Discourage Platform Sellers? Empirical Analysis of an Online Solar Marketplace

Abstract

Problem definition: Customer reviews are essential to online marketplaces. However, reviews typically vary; ratings of a product or service are rarely the same. In many service marketplaces, including the ones for solar panel installations, supply-side participants are active. That is, a seller must make a proposal to serve each customer. In such marketplaces, it is not clear how (or if) the dispersion in customer reviews affects the seller activity level and number of matches in the marketplace. Our paper examines this by considering both ratings and text reviews. To our knowledge, this is the first paper that empirically studies how the review dispersion affects a seller’s activity level and the number of matches in an online marketplace with active sellers. Distinct from literature, we examine the relationship between the review dispersion and supply-side activities in an online service marketplace. Methodology/results: We collaborated with one of the largest online solar marketplaces in the United States that connects potential solar panel adopters with installers. We obtained a unique data set from the marketplace for 2013 − 2018. We complement this with public data sets. Our analysis uses traditional econometrics methods, a clustering method, and the deep-learning-based natural-language-processing model BERT developed by Google AI. We find that the dispersion in customer reviews has a significant and inverted U-shaped relationship with an installer’s marketplace activity level. Intuitively, a marketplace operator would favor having more sellers with perfect ratings. In contrast, we identify a significant and inverted U-shaped relationship between the market-level review dispersion and transactions. Managerial implications: Our paper provides key insights to marketplace operators and sellers. We find that in contrast to general belief, an operator can improve its market transactions by keeping/promoting sellers with low ratings or avoiding (negative) review filtering. Furthermore, sellers’ implementation of “rating gating” to avoid negative reviews may backfire for them by reducing their matches. Supplemental Material: The e-companion is available at https://doi.org/10.1287/msom.2021.0104 .