Abstract
Effect direction (evidence to indicate improvement, deterioration, or no change in an outcome) can be used as a standardized metric which enables the synthesis of diverse effect measures in systematic reviews. The effect direction (ED) plot was developed to support the synthesis and visualization of effect direction data. Methods for the ED plot require updating in light of new Cochrane guidance on alternative synthesis methods. To update the ED plot, statistical significance was removed from the algorithm for within‐study synthesis and use of a sign test was considered to examine whether patterns of ED across studies could be due to chance alone. The revised methods were applied to an existing Cochrane review of the health impacts of housing improvements. The revised ED plot provides a method of data visualization in synthesis without meta‐analysis that incorporates information about study characteristics and study quality, using ED as a common metric, without relying on statistical significance to combine outcomes of single studies. The results of sign tests, when appropriate, suggest caution in over‐interpreting apparent patterns in effect direction, especially when the number of included studies is small. The revised ED plot meets the need for alternative methods of synthesis and data visualization when meta‐analysis is not possible, enabling a transparent link between the data and conclusions of a systematic review. ED plots may be particularly useful in reviews that incorporate nonrandomized studies, complex systems approaches, and diverse sources of evidence, due to the variety of study designs and outcomes in such reviews.
Highlights
IntroductionBOON AND THOMSON cannot be obtained for all included studies
This article describes a revised method for the effect direction plot[1] in light of recent Cochrane guidance.Effect direction can be used as a standardized metric to encompass a wide variety of data in systematic reviews in which standardized effect sizesRes Syn Meth. 2021;12:29–33.wileyonlinelibrary.com/journal/jrsmBOON AND THOMSON cannot be obtained for all included studies
The updated effect direction plot provides a method of synthesis and visualization of effect direction that removes the reliance on statistical significance to combine multiple similar outcomes from single studies, in line with updated methodological guidance from Cochrane
Summary
BOON AND THOMSON cannot be obtained for all included studies Such situations, in which some form of “narrative synthesis” is conducted rather than meta-analysis, are common[2,3]; approximately half of Cochrane reviews include a narrative approach to synthesis for some or all outcomes.[4]. The effect direction plot was devised as an approach to support synthesis where effect direction is used as the common metric, principally by providing a way of visualizing the data and promoting transparent links between the data and the narrative.[1] In the 2013 Cochrane review of housing improvements for health, for example, the effect direction plot was implemented to combine and present findings on four key health outcome domains for outcomes that were conceptually similar but measured in disparate ways.[5] In each individual study, similar outcomes (eg, cough frequency, cough at night, wheeze, lower respiratory symptoms) are combined into a single outcome domain (in this example, “respiratory health”). In the housing improvement review, use of the effect direction plot enabled a 10 000 word review of complex interventions to be summarized and visually displayed in a graphic tabulation on a single page.[1]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
