A Proposed Figure of Merit for Evaluating the Performance of Radiation Waste Measurement Systems

Abstract

Several different types of detectors, software, and hardware designs are employed in instruments used to monitor radioactive content of freight shipments in order to detect illicit trafficking of nuclear materials. The instruments can be container monitors, hand-held radiation detectors, mobile analysis systems, or fixed radiation portal monitors. However, within the various groupings (e.g., portal monitors), all instruments are expected to solve the same problem, that is, to detect and identify any radioactive material present according to the prescribed investigative procedure or CONOPS. The best way to compare the performance of different instruments is with a numerical score or Figure of Merit (FOM). The FOM must quantify the performance of the instrument with respect to true positives (TP), false positives (FP), and false negatives (FN). The minimization of FN for certain radionuclides (e.g., uranium and plutonium or other Special Nuclear Material (SNM)) is more important than the minimization of FN for non-threat nuclides (e.g., low level NORM). In a similar way, the minimization of FP for SNM is more important than falsely reporting the common NORM nuclides which represent no threat. System performance depends on the measurement detail (e.g., transit speed through a portal, measurement time in general, amount and distribution of NORM), therefore the test conditions should also be included in the statement of the FOM. The FOM is expected to vary significantly with the above measurement details. A FOM has been developed based on the number of true positives (TP), the number of false important positives (FIP), the number of FP, the number of true positives for SNM (TPSNM), and the number of false positives for SNM (FPSNM). This formula rates the overall performance with extra weight given to FP and FN for SNM. Examples will be shown for testing of portal monitors.