Abstract
Purpose Allocation of complex crowdsourcing tasks, which typically include heterogeneous attributes such as value, difficulty, skill required, effort required and deadline, is still a challenging open problem. In recent years, agent-based crowdsourcing approaches focusing on recommendations or incentives have emerged to dynamically match workers with diverse characteristics to tasks to achieve high collective productivity. However, existing approaches are mostly designed based on expert knowledge grounded in well-established theoretical frameworks. They often fail to leverage on user-generated data to capture the complex interaction of crowdsourcing participants’ behaviours. This paper aims to address this challenge. Design/methodology/approach The paper proposes a policy network plus reputation network (PNRN) approach which combines supervised learning and reinforcement learning to imitate human task allocation strategies which beat artificial intelligence strategies in this large-scale empirical study. The proposed approach incorporates a policy network for the selection of task allocation strategies and a reputation network for calculating the trends of worker reputation fluctuations. Then, by iteratively applying the policy network and reputation network, a multi-round allocation strategy is proposed. Findings PNRN has been trained and evaluated using a large-scale real human task allocation strategy data set derived from the Agile Manager game with close to 500,000 decision records from 1,144 players in over 9,000 game sessions. Extensive experiments demonstrate the validity and efficiency of computational complex crowdsourcing task allocation strategy learned from human participants. Originality/value The paper can give a better task allocation strategy in the crowdsourcing systems.
Highlights
Crowdsourcing system is an emerging platform for completing tasks from crowds
We suggest that the workers be selected based on the reputation because the reputation represents the past performance of the worker, and this approach conforms to the selection of human, which can be explained from the real human decision-making in section of Experimental Evaluation
In the process of training, whether the score is obtained or not is based on the reputation and the judgement described in equation (5), which is shown in Algorithm 1
Summary
Crowdsourcing system is an emerging platform for completing tasks from crowds. Existing algorithmic crowdsourcing approaches focus on designing mechanisms to match tasks to suitable workers taking into account factors such as their skill sets, reputation, availability and current workload. Such approaches are either based on recommendations (i.e. telling wor kers what to do) (Ho and Vaughan, 2012; Nath and Narayanaswamy, 2014), (Yu et al, 2013, 2016, 2015a, 2015b, 2015c, 2013) or incentives (i.e. influencing workers’ behaviours through economic means) (Liu et al, 2015; Miao et al, 2016; Yu et al, 2015a, 2015b, 2015c), (Tran-Thanh et al, 2014, 2015). Problem formulation we formulate the crowdsourcing task allocation problem to provide a foundation based on which PNRN can be proposed
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
