Methods reporting in human laboratory studies

Abstract

There is growing concern in the behavioural and biomedical sciences regarding the replicability of much published research 1, 2. In response to this, initiatives such as the Reproducibility Project are emerging. This project is attempting the replication of findings reported in three major psychology journals in 2008, with the hope of improving understanding of the rate and predictors of replication 3. However, for replication to be possible, accurate and complete reporting of scientific methods is essential. To ensure this, a number of journals now require authors to use checklists prior to manuscript submission. The Consolidated Standards of Reporting Trials (CONSORT) checklist 4 was created in the early 1990s through a realization that the reporting of randomized controlled trials was suboptimal. Similar checklists are now commonplace for reporting meta-analyses and systematic reviews [Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)] 5, observational studies in epidemiology [STrengthening the Reporting of OBservational studies in Epidemiology (STROBE)] 6, evaluations with non-randomized designs [Transparent Reporting of Evaluations with Nonrandomized Designs (TREND)] 7 and in-vivo animal studies [Animal Research: Reporting In Vivo Experiments (ARRIVE)] 8. These checklists assist in both the reporting of the work and in replicating it, by ensuring that all the necessary information is available. A recent paper, published in Addiction 9, drew our attention to the need for a similar checklist when reporting human laboratory studies. In the paper, a visual probe task was used to investigate visual attention to cigarette package health warnings, with and without text-captions. Health warnings and neutral stimuli were presented simultaneously on either side of a screen, followed by a probe replacing the position occupied previously by one of the stimuli. Participants pressed one of two buttons as quickly as possible in response to the probe location. Faster reaction times to respond to probes replacing health warnings with text-captions among smokers were taken to represent an attentional bias to these stimuli. However, this conclusion can be made only when presentation times of visual stimuli are short, allowing time for a single orientating eye movement. Longer presentation times make results difficult to interpret as multiple eye movements are possible, allowing the disengagement of attention as well as refixations before probe onset. Knowing the presentation time of stimuli in this task is therefore vital. However, this detail was unreported in this paper, rendering the results impossible to interpret for the reader and the study impossible to replicate without contacting the authors for further information. Oversights and omissions such as these are not uncommon in scientific reporting, and noticing them should not be the sole responsibility of authors, but of reviewers and journal editors as well. While we recognize that human laboratory studies are extremely diverse, and creating a one-size-fits-all checklist might be complex, this is something that should be considered seriously. Nature Neuroscience has recently announced the introduction of its own methods reporting checklist 10, to be completed by authors prior to submission, describing the study design, procedure, data analysis and results in detail. Preliminary data indicate that reviewers using the checklist described their methods and statistics more thoroughly than reviewers not using the checklist. We feel that a similar model at Addiction would pave the way for more transparent, accurate and easily replicable studies. None.