Reciprocity through ratings: An experimental study of bias in evaluations

Simon D Halliday,Jonathan Lafky

doi:10.1016/j.socec.2019.101480

Simon D Halliday, Jonathan Lafky

Open Access

https://doi.org/10.1016/j.socec.2019.101480

Copy DOI

Abstract

This paper studies the potential for ratings of seller quality to be influenced by side payments to raters. In a laboratory setting, we find that even modest side payments from sellers to raters have large effects, with the type of rating (favorable or unfavorable) given to a seller determined primarily by how large a monetary transfer the seller makes to the rater. Our results demonstrate that side payments can crowd out a rater’s concern for buyers, even in situations where there is no potential for long-term relationship building.

Highlights

Online commerce is a major component of the economy, accounting for $389 billion in sales in 2016, or 8.0% of all US retail.1 This is an increase of 14.4% from 2015, compared to only 2.8% growth in total retail
We show that such payments can heavily influence rater behavior, decreasing the usefulness of ratings for future consumers
Due to the lower frequency of rating and resultant decrease in the number of ratings observed, a larger sample size was necessary in the costly treatment relative to the baseline free rating treatment in order to observe a similar number of ratings in each treatment

Summary

Introduction

Online commerce is a major component of the economy, accounting for $389 billion in sales in 2016, or 8.0% of all US retail. This is an increase of 14.4% from 2015, compared to only 2.8% growth in total retail. A rater who desires to reciprocate a seller’s sidepayment (or lack thereof) may be tempted to do so at the expense of accurate ratings, resulting in two competing objectives: reciprocity toward the seller in response to a side-payment, and a desire to accurately inform future buyers. We are primarily interested in the effect of side-payments on ratings, the difference in costs between treatments allows us to examine the opposite direction, asking whether the ease with which consumers can punish or reward via ratings influences how generous sellers are with side-payments This is similar to Xiao (2013) who studies the effectiveness of punishment when the decision to punish is either free or beneficial to the punisher. We first develop of simple model of rating behavior, initially for strictly-self interested agents, and when raters have social preferences

Theory

Experimental Design and Procedures

Results

Result

Conclusion