Approaches to evaluating model quality across different regime types in environmental and public health governance

Abstract

A reliance on mathematical modelling is a defining feature of modern global environmental and public health governance. Initially hailed as the vanguard of a new era of rational policy-making, models are now habitually subject to critical analyses. Their quality, in other words, is routinely queried, yet what exactly is quality in this context? The prevailing paradigm views model quality as a multi-dimensional concept, encompassing technical dimensions (e.g. precision and bias), value judgments, problem-framing, treatment of “deep” uncertainties, and pragmatic features of particular decision contexts. Whilst those technical dimensions are relatively simple to characterise, the broader dimensions of quality are less easily formalised and as a result are difficult to take account of during model construction and evaluation. Here, we present a typology of governance regimes (risk-based, precautionary, adaptive and participatory) that helps make explicit what these broader dimensions of model quality are, and sketches out how the emphasis placed on them differs by regime type. We show that these regime types hold distinct positions on what constitutes sound evidence, on how that evidence should be used in policy-making, and to what social ends. As such, a model may be viewed within one regime as providing legitimate evidence for action, be down-weighted elsewhere for reflecting a flawed problem-framing, and outright rejected in another jurisdiction on the grounds that it does not cohere with the preferred ethical framework for decision-making. We illustrate these dynamics by applying our typology to a range of policy domains, emphasising both the disconnects that can occur, as well as the ways that modellers have adapted their practices to ensure that their evidence is brought to bear on policy problems across diverse regime types.