Adaptive Task Replication Strategy for Human Computation

Abstract

Human computation systems are distributed systems in which the processors are human beings, called workers. In such systems, task replication has been used as a way to obtain results redundancy and quality. The level of replication is usually defined before the tasks start executing. This approach, however, generates the problem of defining the suitable task replication level. If the level of replication is overestimated, it is used an excessive amount of workers and, therefore, there is an increase in the cost of executing all tasks. On the other hand, if the level of replication is underestimated, a desired level of quality cannot be achieved. This work proposes an adaptive replication strategy that defines the level of replication for each task during execution time. The strategy is based on estimations of the degree of difficulty of tasks and the degree of credibility of workers. Results from simulations using data from two real human computation applications show that, compared to non-adaptive task replication, the proposed strategy reduces the number of replicas substantially, without compromising the accuracy of the obtained answers.