Abstract
Crowdsourcing provides a distributed method to solve the tasks that are difficult to complete using computers and require the wisdom of human beings. Due to its fast and inexpensive nature, crowdsourcing is widely used to collect metadata and data annotation in many fields, such as information retrieval, machine learning, recommendation system, and natural language processing. Crowdsourcing helps enable the collection of rich and large‐scale data, which promotes the development of researches driven by data. In recent years, a large amount of effort has been spent on crowdsourcing in data collection, to address the challenges, including quality control, cost control, efficiency, and privacy protection. In this paper, we introduce the concept and workflow of crowdsourcing data collection. Furthermore, we review the key research topics and related technologies in its workflow, including task design, task‐worker matching, response aggregation, incentive mechanism, and privacy protection. Then, the limitations of the existing work are discussed, and the future development directions are identified.
Highlights
Machine learning and deep learning technologies have increasingly become a research topic in many fields, including computer vision, natural language processing, and other fields related to artificial intelligence
The data collected through crowdsourcing may contain a large amount of sensitive information, which is directly related to user privacy, such as the user’s geographical location, travel trajectory, and personal preferences
Platform-assigned tasks (PAT) involves assigning a given task to suitable workers based on various conditions, aimed at achieving optimization goals benefitting the requester, such as maximizing the number of tasks assigned, minimizing the cost currently, and improving the quality of task responses, or goals benefitting the worker, such as maximizing the reward received by the worker
Summary
Machine learning and deep learning technologies have increasingly become a research topic in many fields, including computer vision, natural language processing, and other fields related to artificial intelligence. Some technologies are applied to the crowdsourcing process to control the quality, cost, efficiency, and preserving privacy These techniques focus on solving the following key issues: how to design a Wireless Communications and Mobile Computing task, how to select a worker (i.e., people who perform tasks), how to aggregate workers’ responses, how to design an incentive mechanism, and how to protect privacy from disclosure. This survey describes the process of crowdsourcing data collection, reviews the key research topics and related technologies in its workflow, and discusses the limitations of the existing work and open problems.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
