Assessment of string tests strategy for an en route air traffic control system

Abstract

The Federal Aviation Administration (FAA) depends upon several highly available large and complex computer systems to support air traffic control operations. One of the FAA's major initiatives, the Ada based En-Route Automation Modernization (ERAM) program leverages and significantly expands upon previous developed systems. To determine the effectiveness of the string test implementation before full system level testing, the FAA performed an audit of the software requirements that were verified by string tests. An industry standard approach was used to determine the number of Shalls to be inspected. Detailed evaluation using multiple software documents and test plans were compared to the string test results provided by the development contractor. This paper presents details of the audit, its results and reconciliation of the data with the development contractor. The approach to have selected software Shalls verified through the string test method versus full system test resulted in a cost savings of 50% of the cost of the selected software Shalls' full system test, by avoiding duplication of testing efforts. The resulting high pass rate of the audit gave confidence in the development contractor's string test results and also demonstrated a high confidence level in the code, thus confirming the high quality of the Ada code effort.