An Alternative Metric for Detecting High-Severity Incident Reports Analyzed Using FMEA

Abstract

Our department has maintained an incident learning system since 2011. As part of our QA program, failure mode, and effects analysis (FMEA), as described in AAPM TG-100, is used to analyze and rank reported incidents. The properties of the three numbers used to determine the risk priority number (RPN) are not suitable for calculating the product of the numbers. We proposed the use of an alternative metric to the RPN. Failure mode and effects analysis applied to the failure modes calculates the RPN based on the severity (S), probability of occurrence (O) and probability the event will go undetected (D). Each parameter was ranked from 1 to 10, and the RPN was their product. These numbers were ordinal, there was an order in the continuum; 1 was less than 2, 2 was less than 3 and so on; however, before operations such as multiplication could be meaningful, the following prerequisites had to be met: 1) the data must be interval scale (distance between consecutive integers is equivalent) and 2) ratio-scale data (an absolute zero must exist). The 3 parameters used to calculate the RPM exhibited neither interval- nor ratio-scale properties. We proposed the use of an alternative metric, a 3-digit composite number (3-tuple) composed of the severity, occurrence and detectability (SOD). In a previous abstract, FMEA was applied to reported incidents and the incidents were ranked by RPN. For this study, 374 incidents were re-ranked according to a new 3-digit composite number (SOD) comprised of the severity, occurrence and detectability. The reports were sorted first by severity, next by occurrence and then by detectability to determine their ranking. The top ten reports according to each metric (RPN10 and SOD10) were ranked using the other system. Using the RPN ranking system, the average ranking of the SOD10 was 38, ranging from 3 to 124. Using the SOD ranking system, the average ranking of the RPN10 was 18 ranging from 3 to 27. Because the properties of the three parameters used to calculate the RPN make them unsuitable for multiplication, an alternative metric should be considered for the ranking of incident reports analyzed using FMEA. Reports with a high-severity but lower occurrence and detectability will be ranked lower using the RPN metric compared with the SOD metric. Using the RPN metric, the SOD10 reports (reports with the highest severity) are only in the top 33% of all incident reports. Using the SOD metric, the RPN10 are in the top 7% of all incident reports. Using the alternative metric, SOD, the ranking of the high RPN reports is not compromised, while at the same time the ranking of the high-severity reports is increased.