Abstract
Crowdsourcing integrates human wisdom to solve problems. Tremendous research efforts have been made in this area. However, most of them assume that workers have the same credibility in different domains and workers complete tasks independently. This leads to an inaccurate evaluation of worker credibility, hampering crowdsourcing results. To consider the impact of worker domain expertise, we adopted a vector to more accurately measure the credibility of each worker. Based on this measurement and prior task domain knowledge, we calculated fine-grained worker credibility on each given task. To avoid tasks being assigned to dependent workers who copy answers from others, we conducted copier detection via Bayesian analysis. We designed a crowdsourcing system called SWWC composed of a task assignment stage and a truth discovery stage. In the task assignment stage, we assigned tasks wisely to workers based on worker domain expertise calculation and copier removal. In the truth discovery stage, we computed the estimated truth and worker credibility by an iterative method. Then, we updated the domain expertise of workers to facilitate the upcoming task assignment. We also designed initialization algorithms to better initialize the accuracy of new workers. Theoretical analysis and experimental results showed that our method had a prominent advantage, especially under a copying situation.
Highlights
With the rapid development of mobile Internet and the popularity of intelligent terminal devices, the wide use of crowdsourcing is gradually being integrated into people’s lives [1]
We propose a system, selecting workers wisely for crowdsourcing (SWWC), as an overall solution, which consists of two stages, i.e., the task assignment stage and the truth discovery stage
These observations confirmed that it is inaccurate to quantify worker reliability by using one single value. These results indicate the necessity of considering the domain in task assignment, because workers may be good at one domain, but know nothing about another domain
Summary
With the rapid development of mobile Internet and the popularity of intelligent terminal devices, the wide use of crowdsourcing is gradually being integrated into people’s lives [1]. ARE [14] considers the domain in task assignment to select one expert to complete each task In crowdsourcing, it often needs not a worker, but a large number of high-quality workers. The former stage determines how to assign tasks to domain experts wisely and efficiently The latter stage adopts an iterative method to calculate worker credibility and the truth from each other. To the best of our knowledge, we are the first to propose a crowdsourcing system that comprehensively considers the worker domain expertise and copier detection; We used a greedy strategy to select experts in task assignment and updated worker domain expertise vectors in truth discovery for more precise quantification.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have