The Role of Feedback in Dynamic Crowdsourcing Contests: A Structural Empirical Analysis

Abstract

In this paper, we empirically examine the impact of performance feedback on the outcome of crowdsourcing contests. We develop a dynamic structural model to capture the economic processes that drive contest participants’ behavior, and estimate the model using a rich data set collected from a major online crowdsourcing design platform. The model captures key features of the crowdsourcing context, including a large participant pool, entries by new participants throughout the contest, exploitation (revision of previous submissions) and exploration (radically novel submissions) behaviors by contest incumbents, and the participants’ strategic choice among these entry, exploration, and exploitation decisions in a dynamic game. We find that the cost associated with exploratory actions is higher than the cost associated with exploitative actions. High-performers prefer the exploitative strategy, while low-performers tend to make fewer follow-up submissions and prefer the exploratory strategy. Using counter-factual simulations, we compare the outcome of crowdsourcing contests under alternative feedback disclosure policies and award levels. Our simulation results suggest that the full feedback policy (providing feedback throughout the contest) may not be optimal. The late feedback policy (providing feedback only in the second half of the contest) leads to a better overall contest outcome.