Evaluating Risk in an Abnormal World: How Arbitrary Probability Distributions Affect False Accept and Reject Evaluation

Abstract

Risk mitigation strategies commonly use the test uncertainty ratio (TUR) and end-of-period reliability (EOPR) to ensure a measurement is adequate for making acceptance decisions. Unfortunately, the common guidance of maintaining a TUR of at least 4:1 was developed to simplify the underlying calculus in an era predating modern computing and assumes that the probability distributions describing the product and the measurement uncertainty are unbiased and normally distributed. The Guide to the Expression of Uncertainty in Measurement and its supplements describe several situations where uncertainty in the measurement will not follow a normal distribution. Despite the evidence of non-normal behavior in measurements and products, risk evaluations typically assume normality in both distributions. While evaluating the probability of false accept (PFA) and the probability of false reject (PFR) is more challenging when the probability distributions are non-normal, the calculus is straightforward using either numerical integration or Monte Carlo techniques. This work considers several case studies of risk evaluation, including both global and specific risk, when the product or the test measurement uncertainty do not follow normal distributions. Neglecting non-normal behavior might greatly affect PFA and PFR by either over- or underestimating the probabilities depending on the parameters of the distributions.