Abstract
BACKGROUND-Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE-This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD-The systematic literature review includes 148 papers published between 1991 and 2008. The selected papers were classified according to SPI initiative, applied evaluation strategies, and measurement perspectives. Potential confounding factors interfering with the evaluation of the improvement effort were assessed. RESULTS-Seven distinct evaluation strategies were identified, wherein the most common one, “Pre-Post Comparison,” was applied in 49 percent of the inspected papers. Quality was the most measured attribute (62 percent), followed by Cost (41 percent), and Schedule (18 percent). Looking at measurement perspectives, “Project” represents the majority with 66 percent. CONCLUSION-The evaluation validity of SPI initiatives is challenged by the scarce consideration of potential confounding factors, particularly given that “Pre-Post Comparison” was identified as the most common evaluation strategy, and the inaccurate descriptions of the evaluation context. Measurements to assess the short and mid-term impact of SPI initiatives prevail, whereas long-term measurements in terms of customer satisfaction and return on investment tend to be less used.
Highlights
W ITH the increasing importance of software products in industry as well as in our every day’s life [62], the process of developing software has gained major attention by software engineering researchers and practitioners in the last three decades [93], [97], [98], [106]
A first increased interest in evaluating Software Process Improvement (SPI) initiatives appears in the period between 1998 and 2000 (35, 24%)
As a substantial part of the publications fall within a period of four years before this review was conducted (2008), it increases the likelihood for the results of the studies being relevant, elevating the potential value obtained in this systematic review
Summary
W ITH the increasing importance of software products in industry as well as in our every day’s life [62], the process of developing software has gained major attention by software engineering researchers and practitioners in the last three decades [93], [97], [98], [106]. Software Process Improvement (SPI) encompasses the assessment and improvement of the processes and practices involved in software development [25]. SPI initiatives are referred to activities aimed at Manuscript received May 11, 2010; revised February 8, 2011; accepted February 15, 2011 improving the software development process (see Section 3.4.3 for a definition of the different types of initiatives). The CMMI reference documentation, both for the staged and the continuous representation [113], [114], provides measurement suggestions for each process area as an informative supplement to the required components of the model. The ISO / IEC 15504 standard documentation [112], on the other hand, prescribes that the process improvement has to be confirmed and defines a process measurement framework. The informative part of the ISO standard provides some rather limited examples of process measures without showing how the measurement framework is applied in practice
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
