UNDERSTANDING OF ENVIRONMENTAL MONITORING EVIDENCE FOR RISK MANAGEMENT DECISION-MAKING

Abstract

Rationale To explore intuition and comprehension among environmental scientists of the quantitative interpretation of environmental monitoring evidence for guiding environmental risk management decision-making. Methods The well-established quantitative approach for analyzing evidence from medical diagnostic screening in terms of sensitivity, specificity and positive predictive value (PPV) was evaluated for its applicability to the assessment of environmental monitoring evidence. Differences in terminology and meaning were evident in relation to common practice in the environmental sciences. A brief questionnaire survey was conducted among scientists and scientific managers at a water quality laboratory to explore the degree of understanding of the quantitative concepts underlying the analysis of screening evidence. In particular, the appreciation that false positives will predominate when monitoring for infrequent or hazards, despite using methods that appear to be very accurate, was evaluated by presenting respondents with a hypothetical, but realistic scenario. The case was presented without using the terms sensitivity, specificity or PPV to avoid any confusion that might have arisen from the different meanings and uses of terminology in the environmental sciences. Results Our evaluation of applying the quantitative framework that is accepted in medical diagnostic screening to evaluating environmental monitoring evidence revealed no fundamental or theoretical barriers to pursuing this approach. We surveyed 21 water quality personnel, the majority of whom had more than 10 years experience. We found that given a scenario with a PPV of only 2%, 11 respondents expected it to be >95%, 6 expected it to be between 80 and 95%, one expected 5 to 20%, one had no idea and only two responded within the correct range of 0 to 5%. These results are consistent with an earlier pilot trial of the questionnaire among environmental laboratory scientists where we found that 9 out of 10 respondents expected the evidence to be more than 80% likely to be correctly making a detection when the scenario PPV was only 2%. Conclusion There is considerable scope for applying the quantitative analytical framework from medical diagnostic screening to the planning and interpretation of environmental quality monitoring programs. This analysis highlights important limitations and should be pursued because environmental scientists are apparently overconfident about the ability of their laboratory methods to correctly detect an infrequent hazard in environmental samples. The insights that can be gained from pursuing this line of inquiry offer promise for developing more informed environmental risk management decision-making.