Quality-Assured Synchronized Task Assignment in Crowdsourcing

Abstract

With the rapid development of crowdsourcing platforms that aggregate the intelligence of Internet workers, crowdsourcing has been widely utilized to address problems that require human cognitive abilities. Considering great dynamics of worker arrival and departure, it is of vital importance to design a task assignment scheme to adaptively select the most beneficial tasks for the available workers. In this paper, in order to make the most efficient utilization of the worker labor and balance the accuracy of answers and the overall latency, we a) develop a parameter estimation model that assists in estimating worker expertise, question easiness, and answer confidence; b) propose a quality-assured synchronized task assignment scheme that executes in batches and maximizes the number of potentially completed questions (MCQ) within each batch. We prove that MCQ problem is NP-hard and present two greedy approximation solutions to address the problem. The effectiveness and efficiency of the approximation solutions are further evaluated through extensive experiments on synthetic and real datasets. The experimental results show that the accuracy and the overall latency of the MCQ approaches outperform the existing online task assignment algorithms in the synchronized task assignment scenario.