Implementing Systematic Review at the National Toxicology Program: Status and Next Steps

Abstract

Vol. 121, No. 4 CorrespondenceOpen AccessImplementing Systematic Review at the National Toxicology Program: Status and Next Steps Linda S. Birnbaum, Kristina A. Thayer, John R. Bucher, and Mary S. Wolfe Linda S. Birnbaum Search for more papers by this author , Kristina A. Thayer Search for more papers by this author , John R. Bucher Search for more papers by this author , and Mary S. Wolfe Search for more papers by this author Published:1 April 2013https://doi.org/10.1289/ehp.1306711Cited by:18AboutSectionsPDF ToolsDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InReddit The National Toxicology Program (NTP), an interagency program headquartered at the National Institute of Environmental Health Sciences (NIEHS), carries out a broad range of toxicology research and testing and serves as a resource for identification of substances in our environment that are hazards for human health. One of the ways that the NTP identifies hazards is through carrying out literature-based health assessments. Approximately 2 years ago we began exploring systematic-review methodology as a means to enhance transparency and increase efficiency in summarizing and synthesizing findings from studies in our literature-based health assessments. A systematic review uses an explicit, prespecified approach to identify, select, assess, and appraise the data from studies that focus on addressing a specific scientific question (Institute of Medicine 2011). Although traditionally used to grade the quality of evidence and strength of scientific support for recommendations for clinical practice guidelines and healthcare interventions [Agency for Healthcare Research and Quality (AHRQ) 2012; Guyatt et al. 2011; Higgins and Green 2011], we—and others—were interested in how systematic review methodology might be applied to environmental health questions (Agency for Toxic Substances and Disease Registry 2012; National Research Council 2011; Silbergeld and Scherer 2013; U.S. Environmental Protection Agency 2013; Woodruff and Sutton 2011).With the establishment of the Office of Health Assessment and Translation (OHAT) in 2011, the NIEHS launched a new problem-solving resource for the NTP, particularly with respect to identification of noncancer hazards in our environment (Bucher et al. 2011). OHAT took the lead in investigating how systematic review methodology might be used by the NTP. We embraced systematic review methodology as a useful approach for providing thorough documentation of the steps, inputs, and decisions in a literature-based evaluation. However, we also recognized the necessity to extend existing systematic review methods to accommodate our need in environmental health to integrate data from multiple evidence streams (human, animal, in vitro) and focus on observational human studies rather than on the randomized clinical trials more commonly encountered in the field of health-care intervention (NTP 2012a, 2012b).In late February 2013, the NTP released the Draft OHAT Approach for Systematic Review and Evidence Integration for Literature-based Health Assessments – February 2013 [Draft OHAT Approach; Department of Health and Human Services (DHHS) 2013] for public comment; the deadline for receipt of comments is 11 June 2013. The Draft OHAT Approach adopts or adapts guidance from authoritative systematic review groups (AHRQ 2012; Guyatt et al. 2011; Higgins and Green 2011) to handle the breadth of data from human, animal, in vitro, and mechanistic studies relevant for addressing environmental health questions. In developing a draft approach, OHAT sought advice on systematic review through educational webinars and consultation with technical experts, the NTP Executive Committee, a working group of the NTP Board of Scientific Counselors, the NTP Board of Scientific Counselors, and the public. The draft approach involves a seven-step framework for incorporating systematic review methodology into OHAT literature-based health assessments. In early April of 2013, OHAT will release protocols for two case studies to illustrate application of this framework in specific evaluations. We will test our approach in these case studies to help determine whether additional refinement or revision to the Draft OHAT Approach might be needed. To help the public understand the draft approach and protocols, the NTP will hold a web-based informational meeting on 23 April 2013 to provide an overview of the framework, describe the contents of the case-study protocols, and respond to questions (DHHS 2013). Our intent is to carefully consider all public comments received on the draft approach and to present the Draft OHAT Approach to the NTP Board of Scientific Counselors at its meeting on 25–26 June 2013, with discussion by the NTP of any plans to update the document on the basis of the public’s input. Moving forward, our goal is to increase efficiency and provide greater transparency to the rigorous and objective approach that has been the hallmark of OHAT literature-based health assessments.The authors declare they have no actual or potential competing financial interests.ReferencesAgency for Toxic Substances and Disease Registry. 2012. Update on the “Future of Science at ATSDR Symposium.” In: Board of Scientific Counselors Meeting, May 17–18, 2012, Atlanta, Georgia. Atlanta, GA:Agency for Toxic Substances and Disease Registry, 14–17.Available: http://www.atsdr.cdc.gov/science/docs/BSC%20MINUTES%20MAY%202012.pdf [accessed 14 March 2013]. Google ScholarAHRQ (Agency for Healthcare Research and Quality). 2012. Grading the Strength of a Body of Evidence When Assessing Health Care Interventions: An Update (Draft Report).Available: http://effectivehealthcare.ahrq.gov/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=1163 [accessed 30 July 2012]. Google ScholarBucher JR, Thayer K, Birnbaum LS. 2011. The Office of Health Assessment and Translation: a problem-solving resource for the National Toxicology Program. [Editorial]Environ Health Perspect 119:A196-A19721531652. Link, Google ScholarDHHS (Department of Health and Human Services). 2013. Draft Office of Health Assessment and Translation approach for systematic review and evidence integration for literature-based health assessments—February 2013; request for comments; notice of a meeting. Fed Reg 78(37):12764–12765.Available: http://www.gpo.gov/fdsys/pkg/FR-2013-02-25/pdf/2013-04254.pdf [accessed 25 February 2013]. Google ScholarGuyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek Jet al.. 2011. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables.J Clin Epidemiol 64(4):383-39421195583. Crossref, Medline, Google ScholarHiggins JPT, Green S, eds. 2011. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 (updated March 2011).Available: http://handbook.cochrane.org/ [accessed 3 February 2013]. Google ScholarInstitute of Medicine. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC:National Academies Press.Available: http://www.iom.edu/reports/2011/finding-what-works-in-health-care-standards-for-systematic-reviews.aspx [accessed 14 March 2013]. Google ScholarNational Research Council. 2011. Committee to Review of the Environmental Protection Agency’s Draft IRIS Assessment of Formaldehyde. Washington, DC:National Academies Press.Available: http://www.nap.edu/openbook.php?record_id=13142 [accessed 30 July 2012]. Google ScholarNTP (National Toxicology Program). 2012a. National Toxicology Program Board of Scientific Counselors: Summary Minutes, December 11, 2012.Available: http://ntp.niehs.nih.gov/NTP/About_NTP/BSC/2012/December/BSCMinutes20121211_508.pdf [accessed 11 March 2013]. Google ScholarNTP (National Toxicology Program). 2012b. National Toxicology Program Board of Scientific Counselors: Summary Minutes, June 21–22, 2012.Available: http://ntp.niehs.nih.gov/NTP/About_NTP/BSC/2012/June/BSCMinutes20120622_508.pdf [accessed 11 March 2013]. Google ScholarSilbergeld E, Scherer RW. 2013. Evidence-based toxicology: strait is the gate, but the road is worth taking.ALTEX 30(1):67-7323338807. Crossref, Medline, Google ScholarU.S. Environmental Protection Agency. 2013. Materials Submitted to the National Research Council. Part I: Status of Implementation of Recommendations.Available: http://www.epa.gov/iris/pdfs/IRIS%20Program%20Materials%20to%20NRC_Part%201.pdf [accessed 22 February 2013]. . Google ScholarWoodruff TJ, Sutton PThe Navigation Guide Work Group. 2011. An evidence-based medicine methodology to bridge the gap between clinical and environmental health sciences.Health Aff 30(5):931-9. Crossref, Google ScholarFiguresReferencesRelatedDetailsCited by Tweedale A (2016) The inadequacies of pre-market chemical risk assessment's toxicity studies-the implications, Journal of Applied Toxicology, 10.1002/jat.3396, 37:1, (92-104), Online publication date: 1-Jan-2017. Howard J, Piacentino J, MacMahon K and Schulte P (2017) Using systematic review in occupational safety and health, American Journal of Industrial Medicine, 10.1002/ajim.22771, 60:11, (921-929), Online publication date: 1-Nov-2017. Hoffmann S, de Vries R, Stephens M, Beck N, Dirven H, Fowle J, Goodman J, Hartung T, Kimber I, Lalu M, Thayer K, Whaley P, Wikoff D and Tsaioun K (2017) A primer on systematic reviews in toxicology, Archives of Toxicology, 10.1007/s00204-017-1980-3, 91:7, (2551-2575), Online publication date: 1-Jul-2017. Manservisi F, Marquillas C, Buscaroli A, Huff J, Lauriola M, Mandrioli D, Manservigi M, Panzacchi S, Silbergeld E and Belpoggi F (2016) An Integrated Experimental Design for the Assessment of Multiple Toxicological End Points in Rat Bioassays, Environmental Health Perspectives, 125:3, (289-295), Online publication date: 1-Mar-2017. Whaley P, Halsall C, Ågerstrand M, Aiassa E, Benford D, Bilotta G, Coggon D, Collins C, Dempsey C, Duarte-Davidson R, FitzGerald R, Galay-Burgos M, Gee D, Hoffmann S, Lam J, Lasserson T, Levy L, Lipworth S, Ross S, Martin O, Meads C, Meyer-Baron M, Miller J, Pease C, Rooney A, Sapiets A, Stewart G and Taylor D (2016) Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations, Environment International, 10.1016/j.envint.2015.11.002, 92-93, (556-564), Online publication date: 1-Jul-2016. Williams-Nguyen J, Bueno I, Sargeant J, Nault A and Singer R (2016) What is the evidence that point sources of anthropogenic effluent increase antibiotic resistance in the environment? Protocol for a systematic review, Animal Health Research Reviews, 10.1017/S1466252316000037, 17:01, (9-15), Online publication date: 1-Jun-2016. Samuel G, Hoffmann S, Wright R, Lalu M, Patlewicz G, Becker R, DeGeorge G, Fergusson D, Hartung T, Lewis R and Stephens M (2016) Guidance on assessing the methodological and reporting quality of toxicologically relevant studies: A scoping review, Environment International, 10.1016/j.envint.2016.03.010, 92-93, (630-646), Online publication date: 1-Jul-2016. Rooney A, Cooper G, Jahnke G, Lam J, Morgan R, Boyles A, Ratcliffe J, Kraft A, Schünemann H, Schwingl P, Walker T, Thayer K and Lunn R (2016) How credible are the study results? Evaluating and applying internal validity tools to literature-based assessments of environmental health hazards, Environment International, 10.1016/j.envint.2016.01.005, 92-93, (617-629), Online publication date: 1-Jul-2016. Bucher J and Birnbaum L (2016) Commemorating Toxicology at the National Institute of Environmental Health Sciences on the Occasion of Its 50th Anniversary, Environmental Health Perspectives, 124:11, (A192-A195), Online publication date: 1-Nov-2016. Beronius A and Vandenberg L (2016) Using systematic reviews for hazard and risk assessment of endocrine disrupting chemicals, Reviews in Endocrine and Metabolic Disorders, 10.1007/s11154-016-9334-7, 16:4, (273-287), Online publication date: 1-Dec-2015. Meyer-Baron M, Schäper M and van Thriel C (2014) Toward better research practice—Shortcomings decreasing the significance of epidemiological studies in the toxicological field, NeuroToxicology, 10.1016/j.neuro.2014.03.002, 45, (238-246), Online publication date: 1-Dec-2014. Beronius A, Hanberg A, Zilliacus J and Rudén C (2014) Bridging the gap between academic research and regulatory health risk assessment of Endocrine Disrupting Chemicals, Current Opinion in Pharmacology, 10.1016/j.coph.2014.08.005, 19, (99-104), Online publication date: 1-Dec-2014. Buonsante V, Muilerman H, Santos T, Robinson C and Tweedale A (2014) Risk assessment׳s insensitive toxicity testing may cause it to fail, Environmental Research, 10.1016/j.envres.2014.07.016, 135, (139-147), Online publication date: 1-Nov-2014. Woodruff T and Sutton P (2014) The Navigation Guide Systematic Review Methodology: A Rigorous and Transparent Method for Translating Environmental Health Science into Better Health Outcomes, Environmental Health Perspectives, 122:10, (1007-1014), Online publication date: 1-Oct-2014.Rooney A, Boyles A, Wolfe M, Bucher J and Thayer K (2014) Systematic Review and Evidence Integration for Literature-Based Environmental Health Science Assessments, Environmental Health Perspectives, 122:7, (711-718), Online publication date: 1-Jul-2014. Sutton P and Woodruff T (2013) Risk communication and decision tools for children's health protection, Birth Defects Research Part C: Embryo Today: Reviews, 10.1002/bdrc.21029, 99:1, (45-49), Online publication date: 1-Jan-2013. Weed D (2018) The Need for Systematic Reviews in Oncology, JNCI: Journal of the National Cancer Institute, 10.1093/jnci/djy050 Hooijmans C, de Vries R, Ritskes-Hoitinga M, Rovers M, Leeflang M, IntHout J, Wever K, Hooft L, de Beer H, Kuijpers T, Macleod M, Sena E, ter Riet G, Morgan R, Thayer K, Rooney A, Guyatt G, Schünemann H, Langendam M and Minnerup J (2018) Facilitating healthcare decisions by assessing the certainty in the evidence from preclinical animal studies, PLOS ONE, 10.1371/journal.pone.0187271, 13:1, (e0187271) Vol. 121, No. 4 April 2013Metrics About Article Metrics Publication History Originally published1 April 2013Published in print1 April 2013 Financial disclosuresPDF download License information EHP is an open-access journal published with support from the National Institute of Environmental Health Sciences, National Institutes of Health. All content is public domain unless otherwise noted. Note to readers with disabilities EHP strives to ensure that all journal content is accessible to all readers. However, some figures and Supplemental Material published in EHP articles may not conform to 508 standards due to the complexity of the information being presented. If you need assistance accessing journal content, please contact [email protected]. Our staff will work with you to assess and meet your accessibility needs within 3 working days.