A quantitative evaluation of the effectiveness of quality assurance as experienced on a large-scale software development effort

Abstract

The purpose of quality assurance on software projects is to achieve high quality products on schedule, within cost, and in compliance with contract requirements. However, historically, the effectiveness of these activities on software projects has not been quantitatively demonstrable because of a lack of data collected on the project combined with a lack of insight into the operational reliability of the system. Quality assurance is a collection of activities on a contractual deliverable whose purpose is to impart a degree of confidence that the deliverable will conform to the customer's concept of what was procured. Under these conditions, quality assurance must be performed with respect to a documented baseline of the concept. This baseline can address the need in the form of requirement statements; the conceptual approach to be followed in the form of a functional specification; or the design to be implemented in the form of a design specification. Further, these baselines are hierarchical in the sense that when quality assurance is applied to a level it is implicitly applied to all lower levels; e.g., if the need is to be satisfied, the conceptual approach must be satisfied. Effective quality assurance programs impart a high degree of confidence to the customer without significant impacts on schedule or cost. Historically, this effectiveness has not been quantitatively demonstrable because of a lack of data collected on the project combined with a lack of insight into the operational reliability of the system.