Software acceptance testing that goes beyond the book

Abstract

The design of software acceptance tests is as important to meeting contract goals as is the design of algorithms. This statement is particularly significant on fixed price contracts with tight schedules. An extreme instance of the demand placed on acceptance testing can be found in software projects wherein the only rigorous testing that required the Computer Program Configuration Item (CPCI) to exercise its repertoire of load and store instructions was the Formal Qualification Test (FQT). This paper is about such a project, the lessons learned from it, and provides an effective test approach for fixed price contracts. A word or two about the project is appropriate to establish the context that underscores the impact of the above assertion. Initially 30K (core words), 16-bit program instructions were to be developed within one year using a Varian 73 computer with 32K words of memory for a Command and Control application under a fixed price contract. A set of a priori conditions existed that tended to convey the impression that the inherent risks of this endeavor were reasonable. They were the “facts” that: Of the 30K (core words) to be written, 30% of this code already existed and would be used. Contractor standards would be allowed for documentation with limited use of Military Specifications No formal Design Reviews or audits would accompany the deliverable CPCI. Existent executive software would suffice. A competent and enthusiastic team was committed to the effort.