Optimizing test confidence based on life cycle cost

Abstract

Current Army acquisition policy states that systems shall demonstrate reliability requirements with high confidence, but the policy fails to state what `high' confidence means. Despite this, for every system there is an optimal confidence level with which to demonstrate the reliability requirement. The optimal confidence level may be driven by many competing factors including cost, schedule, Soldier safety, and mission criticality. Most of these factors are difficult to quantify, and leave the community to make a judgment call on the appropriate confidence level. In the absence of other driving or quantifiable metrics, one way to choose an optimal confidence level is to balance test cost against the risk to life cycle cost. This method deals in a natural metric: money. It is easily understood by the layman, and is straightforward to compare to most other programmatic risks. Conveniently it provides objective support for the common sense solution that inexpensive systems (to buy and test) that will be purchased in large quantities should be tested at higher confidence, while expensive systems (to buy and test) bought in low quantities possibly should be tested at lower confidence. If the other factors are not overwhelming, or do not suggest a particular confidence level, confidence can be optimized by balancing test cost against life cycle cost.