How expert are ‘experts’? Comparing expert predictions and empirical data on the use of farmland restoration sites by birds

Abstract

Expert knowledge is widely used to assist decision-making in conservation, typically where information is lacking or involves complex interacting factors. While there is growing attention to effective elicitation of knowledge, empirical testing of the accuracy of predictions by experts is rare. We tested the accuracy of expert predictions of the occurrence of bird species at restored sites in farmland by comparing predictions with empirical data and with random assemblages of species. Generally, there was a positive relationship between the averaged expert prediction of likelihood of occurrence and the observed frequency of occurrence for individual species, though accuracy varied between species. Bias in predictions was related to habitat-use and body size: experts tended to over-estimate the likelihood of common, open-country species occurring at sites and underestimate small, woodland species. For bird assemblages, the collective predictions of experts differed significantly from observed assemblages but performed better than random selection of species. Notably, however, predictions of species occurrence and of assemblages at sites varied markedly between individual experts. Expert knowledge will continue to have a valuable role in decision support, but quantifying the nature of biases in predictions and how they contribute to uncertainty is essential. Limitations in expert prediction can be countered by treating them as a guide rather than source of truth; and combining the judgements from multiple experts will help reduce the variation among individuals. While capitalising on the wealth of knowledge held by experts, conservation science and management must be underpinned by sound empirical evidence.