The communication of systematic review research findings: a randomised experiment of whether readers can understand the key messages from summaries of Cochrane reviews without reading the full review

Abstract

BackgroundCochrane reviews provide an important source of evidence. Many readers will only access the various summaries available for Cochrane reviews rather than the full reviews, so it is essential that the summaries are clear and understandable. We explored whether readers could understand key messages without having to read the full review, and whether there were differences in understanding between various types of summary. MethodsIn August, 2013, we conducted a randomised prospective experiment to determine readers' effectiveness of retrieving information from summaries of systematic reviews provided by the Cochrane Library. Participants were a convenience sample of 36 members of university staff. We focused on one review, which assessed the health effects of the use of electric fans during heat waves. We selected this review because of its topical nature, in that we were experiencing hot weather in August. Participants were asked about their expectation of the effect on mortality of using electric fans during heat waves. They were then randomly assigned to reading one of four summaries of the review (ie, abstract, plain language, podcast, or podcast transcript) and asked to spend no more than 15 min reading or listening to the summary. Afterwards they gave their opinion on the effect of electric fans on mortality and whether they would want to read the full review. The main outcome measure was the correct identification of a key review outcome, which was that the research evidence is mixed. FindingsOf the 36 participants, nine received the abstract, eight the plain language summary, eight the transcript, and 11 the podcast. After considering their summary, 20 participants (53%) identified what the authors of the review regarded as its key message on mortality (three participants [33%] in the abstract group, four [50%] in each of the plain language and transcript groups, and nine [78%] in the podcast group). The differences between these groups were not statistically significant (p=0·31, Fisher's exact test). Only 13 (37%) of the respondents said that they would want to read the full review after having considered their summary. InterpretationThe differences between the groups were not statistically significant but suggest that the audio summary might improve knowledge transfer compared with written summaries. Further research should incorporate a larger sample size, and use more than one review alongside qualitative research to understand how people engage with summaries. FundingNone.