Marshall and Galea respond to "data theory in epidemiology".

Abstract

Over the past decade, the use of agent-based models (ABMs) and the development of causal inference methods have proceeded rather independently in epidemiology. In our article (1), we aimed to provide an initial foundation to bridge the two. We argued for a more rigorous causal inference framework in epidemiologic applications of ABMs and sought to generate discussion about how best to accomplish this. We therefore wish to thank Diez Roux (2) and Hernan (3) for extending this discussion and note here broad points of unanimity and motivating next steps. Centrally, we support Hernan's call for research and consensus on acceptable methodological standards for agent-based modeling in epidemiology. Our paper was a first shot across the conceptual bow. However, there is much work that remains to be done in identifying specific agent-based modeling “best practices” that are relevant to epidemiology. These include, but are not limited to, methods for ABM construction and calibration (including the incorporation of data from different populations and various sources); model validation procedures (including reproducibility of observed real-world phenomena); and acceptable analytic practices (including sufficient sensitivity analyses to evaluate model robustness). Such standards have been developed in other disciplines (4, 5) but remain in their infancy throughout epidemiology. It is our hope that the debate from which our work was motivated continues to inspire the development and adoption of rigorous standards for agent-based modeling in our field. We also agree with Hernan that agent-based modeling requires a shift away from causal inference pursuits that are exclusively data driven. However, we do not advocate for the application of ABMs that are entirely suppositional and void of empirical foundation. In addition to theory and prior knowledge, the construction and calibration of valid simulation models rest on the availability of relevant and unbiased data. We concur with Diez Roux that systems modeling will not replace experimental or observational research—in fact, “best practice” ABMs are wholly reliant on it. We have learned from our past work (6, 7) that the utility of agent-based modeling is significantly improved when relevant data are readily accessible. Ideally, modeling and primary data collection should explicitly inform each other and proceed in tandem. Successful examples of iterative, multimethod approaches to epidemiologic research are currently limited and should be supported. Although elucidation of the dynamics and evolution of complex health systems are worthy avenues of investigation in their own right, epidemiology has identified, and should continue to identify, tangible disease causes that are both modifiable and amenable to public health intervention. As Hernan and Diez Roux discuss, agent-based modeling allows us to move away from questions of causal isolation with downstream foci to a more systems-based and life course–oriented epidemiology. We add to the challenges noted by each author that there is a clear risk of building ABMs so complex that meaningful causal inference (and subsequent public health action) is obfuscated. ABMs informed by all relevant theory and all available data tend to be cumbersome and of limited scientific utility (8). The process of simplification—noted by Diez Roux to be a prerequisite for empirical science—is therefore equally essential in complex systems science. In observational studies, we select confounders based on prior knowledge (informed by data) and conceptual models (e.g., directed acyclic graphs). So too should the construction, calibration, and analysis of ABMs be informed by prior knowledge, valid data, and grounded theory. The charge for epidemiologists, then, is to identify public health questions that are sufficiently complex to warrant a systems science approach and about which relevant data and adequate understanding of the system's subcomponents exist to construct and calibrate valid models.