Model Validation, Sensitivity Analysis, and Utilization with Micro SAINT: A Case Study

Abstract

This paper presents the preliminary results of validation, sensitivity analysis, and utilization efforts with Micro SAINT models of the Close-In Weapon System (CIWS) loading operation. In the absence of an external validation sample, an internal validation/verification was conducted to assess how adequately each model represented the data sample from which it was derived. The results of the validation showed that the simulation models were generally well validated by the methods used. Sensitivity analyses were conducted to estimate the effects of inaccurate parameter estimates and to evaluate the effects of shortened or lengthened activity elements on the simulation's output. Results of the sensitivity analyses indicated that, for the CIWS loading operation, changes in task and total completion times were relative to the percentage change in the individual activity element duration and proportional to the duration of the activity element duration. Utilization efforts involved manipulating existing models to assess the impact of crew size and procedural changes on total completion time. CPM and a scheduling heuristic, combined with inspection of the results, were used to develop ‘optimal’ or good feasible manning allocations. Results of this utilization of the Micro SAINT models are presented in the form of completion time statistics.