Comparison of Integrated Risk Information System and Navigation Guide approaches for study evaluation in systematic reviews of environmental epidemiology studies

Abstract

BACKGROUND AND AIM: There are several approaches available for evaluating risk of bias in environmental epidemiology studies, but few case studies comparing them have been done. METHODS: The Integrated Risk Information System (IRIS) approach is a domain-based framework where each study is evaluated for risk of bias and study sensitivity. Eick et al. (2020) applied multiple approaches, including IRIS, to studies from a Navigation Guide (NG) systematic review of polybrominated diphenyl ethers (PBDEs) and neurodevelopment (Lam et al., 2017). In the current analysis, trained IRIS staff applied the IRIS approach, including PBDE exposure and cognitive testing-specific considerations, to the 10 studies of IQ and compared the results to those using the NG. This analysis was restricted to comparable domains: Exposure and Outcome Assessment, Confounding, Population/Selection, and Selective Reporting (and NG domains considered within the broader IRIS Exposure and Outcome domains). The IRIS and NG approaches have four rating levels that may not be strictly analogous but were interpreted as similar for identifying study deficiencies. RESULTS:We identified three studies with multiple “deficient” domain ratings (low confidence overall) using the IRIS approach. These studies also had multiple domains with “probably high risk of bias” in the NG systematic review. Among studies rated as high or medium confidence overall using the IRIS approach, there were differences in ratings between the approaches, but generally within the top two rating levels (IRIS good/adequate; NG low/probably low). CONCLUSIONS:The approaches identified the same studies as concerning for risk of bias. Among higher quality studies, some rating differences between tools may be attributed to different expertise/interpretations of evaluators. Additional comparisons would be elucidating, ideally with collaboration to ensure application of the approaches consistent with their intended use. Disclaimer: The views expressed in this abstract are those of the author and do not necessarily reflect the views or policies of the U.S. EPA. KEYWORDS: risk of bias, systematic review, environmental exposures, risk assessment