A Crowd Wisdom Management Framework for Crowdsourcing Systems

Abstract

A great many online users, with diverse backgrounds, act as powerful resources that mobile social networks (MSNs) can utilize for crowdsourcing. Exploiting these online users as crowd workers is promising yet nontrivial. To efficiently leverage human intelligence or crowd wisdom, we need to address the following issues: 1) how to motivate users to participate and 2) how to discourage malicious behaviors, such as copying answers or making random guesses. Furthermore, as low-quality answers may degrade the accuracy of synthetic results sharply, the last issue is how to weed them out. In this paper, we present MacroWiz, a simple yet effective framework to manage crowd wisdom on MSNs. Given a crowdsourcing task, MacroWiz motivates online users to contribute their knowledge or opinions, and assists the task requester in collecting answers, selecting reliable ones, and making ultimate decisions. The platform consists of two functional units: wisdom collection and answer selection. The former estimates and gathers the minimum number of answers required to satisfy the task requirement conservatively, while the latter analyzes the accuracy, the effectiveness, and the cost of each answer, based on which it selects those with high accuracy and low cost by solving a dual-objective optimization problem. We validate the effectiveness of our platform by using MovieLens data sets, which contain over one million anonymous ratings of movies. The experimental results show that MacroWiz significantly reduces the latency in making decisions and provides high-quality answers with low cost.