Modeling Space Shuttle software failures at varying criticality levels

Abstract

Regression methods are employed in analyzing a Space Shuttle software failure data set, classified into three criticality levels. A family of models based on transforms of cumulative time and cumulative failures is considered for the purposes of generating long-term future predictions of the software failure process for any criticality level. This family includes several established software reliability models including the exponential, logarithmic, and power models. It also includes models based on transforms of the time per failure, the time-varying analogue to the mean time between failures. Models are compared through crossvalidation on the basis of various predictive performance measures related to the established predicted residual sum of squares (PRESS) criterion. Predictions are generated through the observed failure data for the minor and the noncritical major failure processes since ample failures occur at each of these criticality levels. Predictions for the critical failure process on the other hand are generated using observed major failures, both critical and noncritical, since the critical failures by themselves are too few in number to predict that process well. Predictions are generated using models that produce minimal prediction intervals, and the sizes of these prediction intervals are compared to those generated using established software reliability models. Nonparametric prediction intervals are proposed as alternatives to the usual ones based on the assumption of approximate normality since that assumption is likely to not apply in the context of long-term predictions, especially when predicting the critical failure process.