Evaluating the internal consistency of ISO/IEC TR 15504 process capability measures

Abstract

AbstractA software process assessment based on the emerging International Standard ISO/IEC 15504 (Software Process Assessment) can be considered a subjective measurement procedure since assessors assign ratings to measures (i.e. PA: process attributes) to measure the capability of processes. Since measurement always includes some amount of random measurement error, evaluating the reliability of empirical measurement in ISO/IEC 15504–based process assessment is crucial to give confidence to the assessment results. This study estimated the internal consistency (reliability) of process attributes by utilizing Cronbach's alpha. For this purpose, we analyzed 364 process instances from 29 assessments on the basis of ISO/IEC 15504 in Korea during February 1999 and December 2001. Our result shows that Cronbach's alpha has a high value of 0.88 for use in practice. This is the same value as obtained in the data from the Phase 2 SPICE Trials (between September 1996 and June 1998), where SPICE denotes Software Process Improvement and Capability dEtermination. In addition, this study explored whether the change of the current four‐category PA rating scale increases the internal consistency. Our findings indicate that the current four‐category rating scale gives higher internal consistency of process capability measures than a two‐ or a three‐category rating scale. Copyright © 2004 John Wiley & Sons, Ltd.