Exploring Real-Time Collaboration in Crowd-Powered Systems Through a UI Design Tool

Abstract

Real-time collaboration between a requester and crowd workers expands the scope of tasks that crowdsourcing can be used for by letting requesters and crowd workers interactively create various artifacts (e.g., a sketch prototype, writing, or program code). In such systems, it is increasingly common to allow requesters to verbally describe their requests, receive responses from workers, and provide immediate and continuous feedback to enhance the overall outcome of the two groups' real-time collaboration. This work is motivated by the lack of a deep understanding of the challenges that end users of such systems face in their communication with workers and the need of design implications that can address such challenges for other similar systems. In this paper, we investigate how requesters verbally communicate and collaborate with crowd workers to solve a complex task. Using a crowd-powered UI design tool, we conducted a qualitative user study to explore how requesters with varying expertise communicate and collaborate with crowd workers. Our work also identifies the unique challenges that collaborative crowdsourcing systems pose: potential expertise differences between requesters and crowd workers, the asymmetry of two-way communication (e.g., speech versus text), and the shared artifact's concurrent modification by two disparate groups. Finally, we make design recommendations that can inform the design of future real-time collaboration processes in crowdsourcing systems.