Evidence synthesis of Observational Studies in Environmental Health: Lessons Learned from a Systematic Review on Traffic-Related Air Pollution

Abstract

Background: There is a long tradition in environmental health of using frameworks for evidence synthesis. Recently, frameworks were developed for evidence synthesis in clinical medicine, including Grading of Recommendations Assessment, Development and Evaluation (GRADE). The US Office of Health Assessment and Translation (OHAT) elaborated a GRADE type protocol for evidence synthesis in environmental health to assess the "quality in a body of evidence”. Methods: We describe lessons learned in synthesizing evidence from observational studies of environmental health effects, drawing on our experience of applying the OHAT approach in a large systematic review on traffic-related air pollution and health outcomes. Results: We outlined key elements of and suggested improvements to the GRADE/OHAT approach. To maximize what can be learned from observational studies in environmental health, a "narrative” approach is needed to complement the mechanistic up-and-downgrading of certain factors in the evaluation of the quality of a body of evidence. This "narrative” approach links more directly to current and widely accepted evidence synthesis frameworks, such as from the US EPA and IARC. We argue that observational studies can offer high confidence evidence in environmental health, where randomized controlled trials are generally not feasible. All relevant studies should be included in evidence synthesis, beyond the subset of the available evidence often included in meta-analyses. Sources of heterogeneity can both strengthen and weaken the confidence in the evidence. Consistency of associations across study designs, populations, and exposure assessment methods provides additional confidence in the results. Publication bias should be explored beyond statistical methods and is not necessarily expected in large and collaborative studies. We call for identifying and quantifying possible key biases, their most likely direction, and their potential impacts on the results. Conclusions: GRADE-type frameworks require substantial modification to align better with features of environmental health questions and the studies that address them.