Inherent Flaws in Risk Matrices May Preclude Them From Being Best Practices

Abstract

This article, written by Special Publications Editor Adam Wilson, contains highlights of paper SPE 166269, ’The Risk of Using Risk Matrices,’ by Philip Thomas, SPE, and Reidar B. Bratvold, SPE, University of Stavanger, and J. Eric Bickel, SPE, The University of Texas at Austin, prepared for the 2013 SPE Annual Technical Conference and Exhibition, New Orleans, 30 September-2 October. The paper was peer reviewed and published in the February 2014 Oil and Gas Facilities, p. 56. Risk matrices (RMs) are among the more commonly used tools for risk prioritization and management in the oil and gas industry. RMs are recommended by several influential standardization bodies, and a literature search found more than 100 papers that document the application of RMs in a risk-management context. This paper illustrates and discusses inherent flaws in RMs and their potential effect on risk prioritization and mitigation, addressing several previously undocumented RM flaws. Introduction In the oil and gas industry, risk-intensive decisions are made daily. In their attempt to implement a sound and effective risk-management culture, many companies use RMs and specify this in “best practice” documents. Furthermore, RMs are recommended in numerous international and national standards such as those from the International Organization for Standardization (ISO); NORSOK, the Norwegian standards organization; and the American Petroleum Institute (API). The popularity of RMs has been attributed in part to their visual appeal, which is claimed to improve communications. Despite these claimed advantages, the authors were unable to find instances of published scientific studies demonstrating that RMs improve risk-management decisions. However, several studies indicate the opposite—that RMs are conceptually and fundamentally flawed. The complete paper summarizes the known flaws of RMs, identifies several previously undiscussed problems with RMs, and illustrates that these shortcomings can be seen in SPE papers that either demonstrate or recommend the use of RMs. Risk Matrices An RM is a graphical presentation of the likelihood, or probability, of an outcome and the consequence should that outcome occur. Consequences are often defined in monetary terms. RMs, as their name implies, tend to be focused on outcomes that could result in a loss rather than a gain. The purported objective of the RM is to prioritize risks and risk-mitigation actions. Within the context of RMs, “risk” is defined as consequence multiplied by its probability, which yields the expected downside consequence or the expected loss. The consequences and probabilities in an RM are expressed as a range. For example, the first consequence (loss) category might be USD 100,000, the second might be USD 100,000–250,000, and so on. The first probability range might be less than1%, the second might be between 1 and 5%, and so on. A verbal label and a score are also assigned to each range. (Some RMs use these instead of a quantitative range.) For example, probabilities from 10 to 20% might be labeled as “seldom” and assigned a score of 4. Probabilities greater than 40% might be termed “likely” and given a score of 6. Consequences (losses) from USD 5 million to 20 million might be termed “severe” and given a score of 5; losses above USD 20 million might be labeled as “catastrophic” and given a score of 6. Such an RM would treat losses of USD 50 billion (on the scale of BP’s losses stemming from the Macondo blowout) or USD 20 million in the same way, despite the difference of three orders of magnitude. Because there is no scientific method of designing the ranges used in an RM, many practitioners simply use the ranges specified in their company’s best-practice documents. The cells in RMs are generally colored green, yellow, and red. Green means “acceptable.” Yellow stands for “monitor, reduce if possible.” Red is “unacceptable, mitigation required.” Previous work has detailed the way in which the colors must be assigned if one seeks consistency in the ranking of risks. Most of the papers examined failed to assign colors in a logically consistent way. For example, some of the cells designated as red were “less risky” than some of the cells that were designated as yellow. Current Industry Practices In order to use the RM for risk prioritization and communication, several steps must be carried out.