Outcome-Based blind tests for leakage diagnosis in underground watermains by acoustic technologies

Abstract

Underground utility (UU) survey technologies have traditionally been regarded as specialised tests to diagnose the health of UUs. With the advent of new imaging technologies and computing power, UU survey technologies have gradually evolved into a regular form of diagnosis analogous to radiography in medicine. This means that the methods used to assess the performance of these technologies must also evolve from a simple test of “success or failure” to a rigorous examinations of their sensitivity and specificity. This aim can be achieved via carefully designed blind tests in line with the blinding strategies in MacCoun and Perlmutter (2015). Outcome-based blind tests in UU were first introduced for ground-penetrating radar by Lai et al.(2018), which led to the development of the similar but more complex blind tests for acoustic leak diagnosis (ALD) presented in this paper. We introduce how true positive (TP), true negative (TN), false positive (FP), and false negative (FN) results occur in two single-blind ALD tests at Q-Leak at the Water Supplies Department of the HKSAR Government. TP, TN, FP and FN respectively denote that “A leak is found and it is correct (HIT),” “A leak is not found and there is indeed no leak (SAFE),” “A leak is found but there is no leak (false alarm),” and “A leak is not found but a leak exists (MISS).” ALD blind tests are powerful tools for objectively evaluating the capabilities of individuals and teams, equipment, and detection methods within the 4M1E accreditation framework under ISO (2017) and HKAS (2021) supplementary criteria no. 55.