Protocols for laboratory research

Abstract

When we established this journal, one of our ambitions was to provide a forum for the publication of protocols for systematic reviews of animal data. We have also published the SYRCLE suggested format for such protocols, and SYRCLE and CAMARADES have been working with PROSPERO (an online registry of protocols for systematic reviews of data from human studies) to extend their coverage also to animal studies. Why this interest in protocols? Simply, it allows scientists to provide, ex ante, a description of their research plans. This is important because, particularly for observational research, it allows the reader to establish whether the analyses presented were of interest before data collection began, or whether they have been performed in the light of the data that were collected. One might imagine, for instance, a meta-analysis that explored the impact of 20 potential risks of bias, and reported findings only for the one that happened to reach statistical significance. Given the rapid increase in the number of systematic reviews and meta-analyses of sometimes dubious quality,2 the increased value which is attained through prior registration of a systematic review protocol is an important defence; and we believe it is no longer tenable to publish systematic reviews where a protocol for that review is not available, in the public domain, and dated prior to data analysis. Of course, the systematic review as conducted may differ from the systematic review as planned, but where changes have occurred we expect authors to be clear about when the change occurred, and why. A further criticism of systematic reviews is that they can only be as valid as are the contributing data, and as well as the risks of bias attendant in individual studies3 there are potential problems with selective outcome reporting bias,4 p-hacking (where there is flexibility in the choice of statistical analyses) and “hypothesising after results are known” (HARKing), where instead of the hypothesis being tested by the collection of experimental data, the data are collected and then the hypothesis is derived. Clinical trials take years and systematic reviews take months, and in both of these research domains there is adequate opportunity to assert a reasonably detailed protocol before data collection is complete. However, for other research domains, including in vivo research, the timescale for the “premise—hypothesis—test” research cycle may be very short, such that there is not always time to elaborate a detailed ex ante protocol. Further, the highly competitive nature of laboratory research means that some investigators may be reluctant to share their ideas and research plans with their competitors while that work is incomplete. Registration of preclinical studies has been proposed previously.5 Here we propose the adoption of a simple six component “PHISPS” protocol which would address many of these issues and which would not be unreasonably burdensome. Such protocols could be stored online in one of the repositories available (such as the Open Science Framework, https://osf.io), where it is possible to make the protocol available publically, or only to those with whom the url is shared. For instance, the PHISPS protocol may be kept private while the research is in progress, shared with peer reviewers at the time of manuscript submission, and made publically available when the work is published. Reducing an in vivo study to six sentences necessarily means that much useful information will not be available, but the purpose here is to provide evidence that the experiments were conducted and analysed as planned, not to provide a “recipe” for the research to allow others to attempt replication. Most progress usually occurs in little steps, by evolution not revolution. As laboratory scientists come under increasing pressure to be productive (grants, papers) and to comply with institutional, funder and national regulatory mandates it is important that efforts to increase the value of in vivo research create as little additional burden as possible. We believe that PHISPS protocols strike a reasonable balance between burden and benefit.