A web-based tool for eliciting probability distributions from experts

David E Morris,Jeremy E Oakley,John A Crowe

doi:10.1016/j.envsoft.2013.10.010

Abstract

We present a web-based probability distribution elicitation tool: The MATCH Uncertainty Elicitation Tool. The Tool is designed to help elicit probability distributions about uncertain model parameters from experts, in situations where suitable data is either unavailable or sparse. The Tool is free to use, and offers five different techniques for eliciting univariate probability distributions. A key feature of the Tool is that users can log in from different sites and view and interact with the same graphical displays, so that expert elicitation sessions can be conducted remotely (in conjunction with tele- or videoconferencing). This will make probability elicitation easier in situations where it is difficult to interview experts in person. Even when conducting elicitation remotely, interviewers will be able to follow good elicitation practice, advise the experts, and provide instantaneous feedback and assistance.

Highlights

When modelling an environmental system, suitable data are not always available for obtaining robust estimates of all the model parameters
We present a web-based probability distribution elicitation tool: The MATCH Uncertainty Elicitation Tool
We recommend following the guidance set out in the templates provided within the SHELF package

Summary

Introduction

When modelling an environmental system, suitable data are not always available for obtaining robust estimates of all the model parameters. O’Hagan et al (2006) emphasize the important role of a facilitator: an individual with experience in probability elicitation, who is able to provide training and assistance to the expert. They recommend giving feedback to the expert: highlighting any inconsistent judgements and, when fitting probability distributions to the expert’s judgements, showing the fitted distribution to the expert to invite further reflection and check that the fitted distribution is acceptable. In the quartile method, the expert provides a median X0.5, lower quartile X0.25, and upper quartile X0.75 The facilitator can fit a probability distribution to these judgements using a least squares procedure. This requires software, and it is helpful for the facilitator to be able to discuss the fitted distribution with the expert

The MATCH Uncertainty Elicitation Tool

Elicitation methods available in the Tool

Fitting and feedback

The share option

Data flow

Discussion