Experimental evidence of sensitivity analysis predicting minimum failure probabilities

Abstract

The authors discuss a theoretical statistical technique complementary to black-box testing, called sensitivity analysis. Black-box testing establishes an upper limit on the likely probability of software failure. Software sensitivity analysis establishes a lower limit on the probability of failures that are likely to occur. Together, these estimates can be used to establish confidence that software does not contain faults. Experimental results show that sensitivity analysis predicts a realistic, lower limit on the probability of failure. This limit is lower than can be generally predicted using testing results only. Sensitivity analysis was applied to three versions of NASA's specification for the sensor management of a redundant strapped down inertial measurement unit (RSDIMU). An RSDIMU is a component of a modern inertial navigation system to provide acceleration data that is integrated to determine velocity and position. The programs used in the experiment were originally produced for use in an N-version system for the RSDIMU. >