Data-driven agile software cost estimation models for DHS and DoD

Abstract

Problem:Since the Manifesto for Agile Software Development, the cost analysis community has struggled to find the most appropriate size measures for developing accurate agile software development cost estimates at an early phase to establish baseline budgets and schedules, and for the selection of competitive bidders. At this time, common agile sizing measures such as Story Points are not practical since these are reported months after contract award in the Department of Homeland Security (DHS) and Department of Defense (DoD). The problem is compounded with the lack of data to build estimation models for informed decisions. Aims:The primary objective is to investigate how well two new size measures (Functional Story and Issues) accurately relate to total effort, and how these compare and rank against four popular software size measures (Story, Story Point, Unadjusted Function Points, and Simple Function Points). The second objective is to rank the six sizing measures based on how well each estimate software development effort at an early phase or after contract award. Method:The experimental framework relied on an analysis of variance and goodness-of-fit tests to examine and compare the accuracy of effort estimation models using six competing size measures. The analysis is based on data from 17 agile projects implemented between 2014 to 2021. Results:Our two new size measures (Functional Story, Issues) proved to be good predicters of total software development effort. Functional Story is better at predicting total effort at early phase than Function Points and Story. Functional Story and Function Points are better at predicting total effort after contract award than Story Point and Issues. Conclusion:The DHS and DoD cost community can choose one or more of these estimation models to evaluate agile software development cost proposals or track agile developer’s progress after contract award.