The effectiveness of risk management: measuring what didn’t happen

Abstract

Two of the most common reasons for not implementing a risk management program are cost and benefit. This paper focuses on whether the benefits of intervention can be shown to justify the costs. A confounding factor is that the acts of intervention during a risk management program may alter the outcome in ways we cannot separate and therefore cannot cost out. A second confounding factor is response bias – the tendency of individuals consistently to underestimate or overestimate risk, resulting in interventions that may be ineffective or excessively wasteful. The authors demonstrate that signal detection theory (SDT) can be used to analyze data collected during a risk management program to disambiguate the confounding effects of intervention and response bias. SDT can produce an unbiased estimate of percent correct for a risk management program. Furthermore, this unbiased estimator allows comparison of results from one program to another.