Receiver Operating Characteristics Curves In The Ordnance Detection And Discrimination Study At The Former Fort Ord, California

Abstract

A phased geophysical ordnance detection and discrimination study (ODDS) specific to the Fort Ord, California environment has been performed. This study evaluated existing state-of-the-art OE detection/discrimination systems. Fundamental criteria for instrument selection were proven (demonstrated, documented) success at Government OE detection test sites and/or in OE removal actions. Digital data acquisition capability, ease of deployment and cost effectiveness and system sensitivity, accuracy, and resolution were also considered. Tested systems were the Geonics EM61 and EM61-HH, Geometrics G-858, Schonstedts GA-52/Cx, GA-72/Cv, and GA-52/C, Geophex GEM-3, and both BlackHawk and NRL MTADS, vehicular-towed array systems. The analog Schonstedt instruments were included as part of an evaluation of previous work at Fort Ord. Data analysis included developing Receiver Operating Characteristic (ROC) curves for results obtained over seeded grids and representative field sites at Fort Ord. These ROC curves have been constructed using False Alarm Rates (FARs) defined and calculated three ways: 1) per Acre, 2) per Item Found, and 3) as the percent of target picks that are False. The ROC curves have been used to rank the effectiveness of the tools involved in the study as to their effectiveness on sites with varying OE types, terrain, vegetation, and culture. Additionally, ROC curves have been used to study the effectiveness of variable survey traverse separations and, using a ROC curve developed for data acquired in one grid, refine and predict the anomaly selection process in neighboring grids. Finally, a simple way of using the ROC curves to make comparisons between instruments is to calculate an “Efficiency Rating” which is defined as the ratio of the Probability of Detection to the FAR per Item Detected. An instrument with a high rating will produce fewer false alarms per successful excavation than an instrument with a lower rating.