Improving Transparency of Decision Models Through the Application of Decision Analytic Models with Omitted Objects Displayed (DAMWOOD).

Abstract

The coronavirus disease 2019 (COVID-19) pandemic has increased public awareness of the influence of epidemiological and economic decision models on public policy decisions. Alongside this is an increased scrutiny on the development, analysis, reporting and utilisation of decision models for public policy making. Therefore, it is important that model developers can clearly explain and justify to all stakeholders what is included and excluded from a model developed to support decision-making, to both improve transparency and trust in decision-making. Our aim is to provide tools for improving communication between modellers and decision-makers, leading to improved transparency in decision-making. To do so, we extend the recently described directed acyclic graphs with omitted objects displayed (DAGWOOD) approach from Haber et al. (Ann Epidemiol 68:64-71, 2022) to decision analytic models, giving the decision analytic models with omitted objects displayed (DAMWOOD) approach. DAMWOOD is a framework for the identification of objects omitted from a decision model, as well as for consideration of the effects of omissions on model outcomes. Objects omitted from a decision model are classed as either an exclusion (known and unknown confounders), misdirection (alternative model pathways) or structure (e.g. model type, methods for estimating relationships between objects). DAMWOOD requires model developers to use explicit statements and provide illustration of included and omitted objects, supporting communication with model users and stakeholders, allowing them to provide input and feedback to modellers about which objects to include or omit in a model. In developing DAMWOOD, we considered two challenges we encountered in modelling for pandemic policy response. First, the scope of the decision problem is not always made sufficiently explicit by decision-makers, requiring modellers to intuit which policy options should be considered, and/or which outcomes should be considered in their evaluation. Second, there is rarely sufficient transparency to ensure stakeholders can see what is included in models and why. This limits stakeholders' ability to advocate to decision-makers for the prioritisation of specific outcomes and challenge the model results. To illustrate the application of DAMWOOD, we apply it to a previously published COVID-19 vaccine allocation optimisation model. The DAMWOOD diagrams illustrate the ways in which it is possible to improve the communication of model assumptions. The diagrams make explicit which outcomes are omitted and provide information on the expected impact of the omissions on model results. We discuss the usefulness of DAMWOOD for framing the decision problem, communicating the model structure and results and engaging with those making and affected by the decisions the model is developed to inform.