Validating the ISO/IEC 15504 measures of software development process capability

Abstract

ISO/IEC 15504 is an emerging international standard on software process assessment. It defines a number of software engineering processes, and a scale for measuring their capability. A basic premise of the measurement scale is that higher process capability is associated with better project performance (i.e., predictive validity). This paper describes an empirical study that evaluates the predictive validity of the capability measures of the ISO/IEC 15504 software development processes (i.e., develop software design, implement software design, and integrate and test). Assessments using ISO/IEC 15504 were conducted on projects world-wide over a period of two years. Performance measures on each project were also collected using questionnaires, such as the ability to meet budget commitments and staff productivity. The results provide evidence of predictive validity for the development process capability measures used in ISO/IEC 15504 for large organizations (defined as having more than 50 IT staff). Furthermore, it was found that the “Develop Software Design” process was associated with most project performance measures. For small organizations evidence of predictive validity was rather weak. This can be interpreted in a number of different ways: that the measures of capability are not suitable for small organizations, or that software development process capability has less effect on project performance for small organizations.