Crowdsourcing in Software Development: Empirical Support for Configuring Contests

Stamatia Bibi,Ioannis Stamelos,Ioannis Zozas,Apostolos Ampatzoglou,George Kalampokis,Panagiotis G Sarigiannidis

doi:10.1109/access.2020.2982619

Abstract

Despite the extensive adoption of crowdsourcing for the timely, cost-effective, and high-quality completion of software development tasks, a large number of crowdsourced challenges are not able to acquire a winning solution, on time, and within the desired cost and quality thresholds. A possible reason for this is that we currently lack a systematic approach that would aid software managers during the process of designing software development tasks that will be crowdsourced. This paper attempts to extend the current knowledge on designing crowdsourced software development tasks, by empirically answering the following management questions: (a) what type of projects should be crowdsourced; (b) why should one crowdsource-in terms of acquired benefits; (c) where should one crowdsource-in terms of application domain; (d) when to crowdsource-referring to the time period of the year; (e) who will win or participate in the contest; and (f) how to crowdsource (define contest duration, prize, type of contest etc.) to acquire the maximum benefits-depending on the goal of crowdsourcing. To answer the aforementioned questions, we have performed a case study on 2,209 software development tasks crowdsourced through TopCoder platform. The results suggest that there are significant differences in the level to which crowdsourcing goals are reached, across different software development activities. Based on this observation we suggest that software managers should prioritize the goals of crowdsourcing, decide carefully upon the activity to be crowdsourced and then define the settings of the task.

Highlights

The term crowdsourcing combines two words: crowd and outsourcing
To reach a conclusion whether the crowdsourced software engineering challenges can be considered successful we followed the steps described in Fig.4: Step 1: To define success in terms of solution acquisition (Solution Acquisitionbinary, V24) we examined the value of the Number of Winning Solutions (V21) variable and compared it to the value of Number of Winners (V12)
We identified trends and common practice when it comes to crowdsourcing different software development activities; we do not claim that these form causal relationships

Summary

Introduction

Howe [12]—who first coined the term in 2006—defined crowdsourcing as ‘‘the act of taking a job traditionally performed by a designated agent and outsourcing it to an undefined, generally large group of people in the form of an open call’’. In this context crowdsourcing is a new business model that enables the co-creation between a ‘‘provider’’ that is the one that sets the details of the problem, the ‘‘supplier’’ / ‘‘crowd’’ that suggests a solution and the ‘‘host’’ who offers the crowdsourcing platform, enabled by Web 2.0 [32], presenting the problem.

Objectives

Methods

Results

Discussion

Conclusion