The Use of Design of Experiments Methodology in a Modeling and Simulation World

Abstract

The Design of Experiments (DOE) methodology is traditionally associated with a testing environment. It is an attempt to address the cost/time issue of test programs and to generate more useful data from a given program. Can this DOE methodology also be used in the modeling and simulation world? This paper examines that question, looking at the specific area of survivability modeling. When dealing with survivability analysis, the types of parameters of interest deal with vulnerability and susceptibility. When using DOE in this manner, each of the vulnerability reduction features (e.g., VUL-1, VUL-2, VUL-3) and each of the susceptibility reduction features (e.g., SUS-1, SUS-2, SUS-3) is considered a factor, or variable. For a given number of factors, an appropriate test matrix is defined. The HIGH and LOW settings required by DOE would be ON for HIGH and OFF for LOW. The test matrix also defines the number of to be conducted. In this case, experiments would be runs of the model or simulation. The test matrix would define the number of computer runs and the level settings for each instance. The DOE methodology would calculate the effect each of these factors had on the response variable and which feature had the greatest impact. In addition, DOE output would provide calculations on the effect of interactions between the various factors on the response variable. These calculations would provide excellent estimators of the synergism between the various features. This type of information could provide marginal cost and marginal benefit estimates of the various combinations of features being considered. In an era when modeling and simulation is being touted as a viable cost-effective supplement to full-scale testing, the use of DOE methodology to evaluate the synergistic effects of vulnerability and susceptibility reduction features is a logical extension of this philosophy. This same process could be used to evaluate the benefits and synergism between various new technologies and provide valuable insights into where scarce resources should be allocated.