I-REFF diagrams: enhancing transparency in systematic review through interactive reference flow diagrams

Abstract

Systematic review methods are recognized for their rigor and transparency and are widely adapted to frameworks that cover review types such as systematic reviews, scoping reviews, and systematic evidence maps. Reporting guidelines help promote better systematic review practices and detailed documentation of the review process for different types of health research (e.g., PRISMA—Preferred Reporting Items for Systematic Reviews and Meta-Analyses; CONSORT—Consolidated Standards of Reporting Trials; and STROBE—Strengthening the Reporting of Observational Studies in Epidemiology). Transparency in the systematic review process and reporting of results is one of the key advantages of the methods and particularly important for hazard and risk assessments due to the high level of scrutiny these reviews face from scientific, political, and public communities. Data visualizations are important to clearly convey information from a review by helping readers perceive, understand, and assess the displayed information easily and quickly. The study flow diagram is a required element of a systematic review and maps out the number of included and excluded records identified, and the reasons for exclusion. Static literature flow diagrams help viewers readily understand the general review methodology and summarize the number of records included or excluded at each stage of the review. However, such diagrams can be time-consuming to develop and maintain during a systematic review or scoping review, and they provide limited summary-level information. We explored how the use of online systematic review tools such as DistillerSR coupled with visualization software such as Tableau can efficiently generate an Interactive REFerence Flow (I-REFF) diagram that is linked to the literature screening data, thus requiring minimal preparation, and resulting in a simplified process for updating the diagram. Furthermore, I-REFF diagrams enhance transparency and traceability by not only summarizing the records in the review but also allowing viewers to follow specific records throughout the review process. We present an example I-REFF diagram and discuss recommendations for key interactive elements to include in these diagrams and how this workflow can improve efficiency and result in an accessible and transparent interactive literature flow diagram without advanced programming.