"But You Promised": Methods to Improve Crowd Engagement In Non-Ground Truth Tasks

Abstract

Crowdsourcing platforms were initially designed to recruit people to perform tasks that were simple cognitively but difficult for computers. One challenge in these settings is to identify an incentive mechanism for motivating workers to complete tasks and do high-quality work. Previous research has studied the use of financial incentive mechanisms and social comparison as motivators. These mechanisms can only be applied to ground truth tasks, tasks for which there is an objective performance scale. In this paper, we define and compare three innovative methods for improving worker engagement on non-ground truth tasks drawing on a psychological theory of commitment. The three methods are similar in asking participants to promise they will complete a task, but they differ in terms of how the commitment is made. In the first method, participants commit by signing a contract; in the second, by listening to a recording; in the third, by recording a personal commitment. The last two methods significantly improved the task completion rate when compared to two baseline conditions. The methods we propose can be implemented simply, can be used for any task, and do not affect participants' behavior other than by improving their engagement.