A field experiment on trialsourcing and the effect of contract types on outsourced software development

Abstract

To ensure the success of software projects, it is essential to select skilled developers and to use suitable work contracts. This study tests two hypotheses: (i) the use of work-sample testing (trialsourcing) improves the selection of skilled software developers; and (ii) the use of contracts based on hourly payment leads to better software project outcomes than fixed-price contracts. Fifty-seven software freelancers with relevant experience and good evaluation scores from previous clients were invited to complete a two-hour long trialsourcing task to qualify for a software development project. Thirty-six developers completed the trialsourcing task with acceptable performance, and, based on a stratified random allocation process, were asked to give a proposal based on an hourly payment or a fixed-price contract. Eight hourly payment-based and eight fixed-priced proposals were accepted. The process and product characteristics of the completion of these 16 projects were collected and analysed. While the use of trialsourcing may have prevented the selection of developers with insufficient skills, the performance on the trialsourcing task of the selected developers did, to a large extent, not predict their performance on the projects. The use of hourly payments seems to have led to lower costs than fixed-price contracts, but not to improved processes or products. We plan to follow up these results with research on how to design more skill-predictive trialsourcing tasks, and when and why different project contexts give different contract consequences.