Eliciting fuzzy distributions from experts for ranking conceptual risk model components

Abstract

An expert elicitation methodology was developed to integrate scientific knowledge from many studies at different spatial and temporal scales. The methodology utilised a structured one-to-one interview to elicit scale-dependent conceptual models and expert-weightings for conceptual model components. It was designed to inform large scale catchment risk analyses but, equally, could be applied to many other environmental applications where expert opinion is required to fill knowledge-gaps. Both quantitative (fuzzy rankings) and qualitative information was collected. The risk analyses relate to those carried out by the Environment Agency of England and Wales to meet their European Water Framework Directive obligations associated with the protection of surface water ecology. Specifically, the information elicited was required to inform future risk analyses and inform strategies to reduce the associated uncertainties.Development of the methodology focussed on minimising potential biases associated with the information elicited and on the obtaining fuzzy rankings consistent with experts’ reasoning. Minimisation of biases was afforded by making the experts aware of potential biases before the elicitation began; the success of this strategy was however difficult to assess within the scope of the study. However, the one-to-one interview provides enough feedback to give some confidence that this strategy has value. The main limitation of the approach is the time-consuming nature of one-to-one interviews, which may lead to interviewee fatigue.There was generally good agreement between experts on the components chosen to be included in the conceptual models and on the assigned fuzzy rankings: although very broad distributions indicating significant uncertainty was a common response. The principal components chosen were dominated by physical factors that control hydrological pathways and connectivity of the landscape to surface waters. Uncertainties were generally associated with the heterogeneity and variability of unique catchments, which combined with sparse observations, makes it difficult to apply current scientific knowledge. These uncertainties are compounded by the fact that current process understanding is largely informed by small scale experiments, where the rules for upscaling remain under-researched: the experts were required to undertake this upscaling during the elicitation. In the absence of knowledge at the appropriate scales, the scale-dependent information elicited is necessary to utilise many scientific theories and ultimately provides hypotheses to be tested using large scale experimentation.