Optimal elicitation of probabilistic information from experts

Abstract

Abstract It is often desirable to describe information derived from the cumulative experience of human experts in a quantitative and probabilistic form. Pertinent examples include assessing the reliability of alternative models or methods of data analysis, estimating the reliability of data in cases where this cannot be measured, and estimating ranges and probable distributions of rock properties and architectures in complex geological settings. This paper presents a method to design an optimized process of elicitation (interrogation of experts for information) in real time, using all available information elicited previously to help in designing future elicitation trials. The method maximizes expected information during each trial using experimental design theory. We demonstrate this method in a simple experiment in which the conditional probability distribution or relative likelihood of a suite of nine possible 3-D models of fluvial-deltaic geologies was elicited from a geographically remote expert. Although a geological example is used, the method is general and can be applied in any situation in which estimates of expected probabilities of occurrence of a set of discrete models are desired.