Considering human aspects on strategies for designing and managing distributed human computation

Lesandro Ponciano,Francisco Brasileiro,Lívia Sampaio,Nazareno Andrade

doi:10.1186/s13174-014-0010-4

Lesandro Ponciano, Francisco Brasileiro + Show 2 more

Open Access

https://doi.org/10.1186/s13174-014-0010-4

Copy DOI

Abstract

A human computation system can be viewed as a distributed system in which the processors are humans, called workers. Such systems harness the cognitive power of a group of workers connected to the Internet to execute relatively simple tasks, whose solutions, once grouped, solve a problem that systems equipped with only machines could not solve satisfactorily. Examples of such systems are Amazon Mechanical Turk and the Zooniverse platform. A human computation application comprises a group of tasks, each of them can be performed by one worker. Tasks might have dependencies among each other. In this study, we propose a theoretical framework to analyze such type of application from a distributed systems point of view. Our framework is established on three dimensions that represent different perspectives in which human computation applications can be approached: quality-of-service requirements, design and management strategies, and human aspects. By using this framework, we review human computation in the perspective of programmers seeking to improve the design of human computation applications and managers seeking to increase the effectiveness of human computation infrastructures in running such applications. In doing so, besides integrating and organizing what has been done in this direction, we also put into perspective the fact that the human aspects of the workers in such systems introduce new challenges in terms of, for example, task assignment, dependency management, and fault prevention and tolerance. We discuss how they are related to distributed systems and other areas of knowledge.

Highlights

Many studies have focused on increasing the performance of machine-based computational systems over the last decades
We propose a theoretical framework that integrates theories about human aspects, design and management (D&M) strategies, and quality of service (QoS) requirements
4.4 Dependency management We focus on analyzing the dependency management strategies that take into account human aspects, while ensuring temporal, serialization, visibility, and cooperation dependencies

Summary

Introduction

Many studies have focused on increasing the performance of machine-based computational systems over the last decades. Two basic guidelines to highlight in this context are: 1) given that most of platforms are based on job boards and that in this environment the effectiveness of a task relies on its ability to gain attention of suitable workers, requesters must provide task descriptions with information that allows workers to identify if the task match their skills, preferences and interests; 2) requesters must avoid generating too restrictive tasks in order to take advantage of the diversity and larger quantities of workers that job boards and recommendation strategies give access to. Example of human computation applications without loops are those that deal with planning activities [90] Such applications usually include a sequence of steps such as: decomposition of the problem into small tasks, execution of each task and aggregation of these partial task outputs to obtain the final output as a result to the problem. This state of affairs indicates a large amount of research still to be conducted after mapping the impact of human aspects

Motivation

Conclusions