Misleading Results: Don't Blame the Mice

Abstract

In the news focus story “when mice mislead” (22 November 2013, p. [922][1]), J. Couzin-Frankel raises concerns about practices that give rise to misleading scientific findings. The case studies cited demonstrate procedural errors that can occur in the design or execution of experiments. These unfortunate mistakes can happen in any area of research and are not the exclusive domain of mice or animal models; clinical research is certainly not immune to error. Even well-executed studies can give rise to misleading results for reasons that are insidious and hard to control ([ 1 ][2]). With regard to mice, the first “risk of bias question” that a reviewer should ask is, “Was the result replicated in more than one genetic background?” Idiosyncrasies of particular mouse strains can be misleading ([ 2 ][3]). Ignoring the effect of genetic background is the most pervasive source of bias in animal studies. To reduce the problem of false or irreproducible scientific findings, we need to address two root causes. Science today is driven by an incentive system that often rewards precedence and impact over quality of the work. Statistical training of scientists often emphasizes analytical techniques over experimental design and quantitative reasoning. These are systemic problems that will not change without substantial effort. However, the message in this News Focus story does suggest some simple advice that can redress many of the pitfalls of experimental studies: Be wise, randomize. 1. [↵][4]1. J. P. A. Ioannidis , PLOS Med. 2, e124 (2005). [OpenUrl][5][CrossRef][6][PubMed][7] 2. [↵][8]1. V. Kumar 2. et al ., Science 342, 1508 (2013). [OpenUrl][9][Abstract/FREE Full Text][10] [1]: /lookup/doi/10.1126/science.342.6161.922 [2]: #ref-1 [3]: #ref-2 [4]: #xref-ref-1-1 View reference 1 in text [5]: {openurl}?query=rft.stitle%253DPLoS%2BMed%26rft.aulast%253DIoannidis%26rft.auinit1%253DJ.%2BP.%26rft.volume%253D2%26rft.issue%253D8%26rft.spage%253De124%26rft.epage%253De124%26rft.atitle%253DWhy%2Bmost%2Bpublished%2Bresearch%2Bfindings%2Bare%2Bfalse.%26rft_id%253Dinfo%253Adoi%252F10.1371%252Fjournal.pmed.0020124%26rft_id%253Dinfo%253Apmid%252F16060722%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [6]: /lookup/external-ref?access_num=10.1371/journal.pmed.0020124&link_type=DOI [7]: /lookup/external-ref?access_num=16060722&link_type=MED&atom=%2Fsci%2F343%2F6169%2F370.1.atom [8]: #xref-ref-2-1 View reference 2 in text [9]: {openurl}?query=rft.jtitle%253DScience%26rft.stitle%253DScience%26rft.aulast%253DKumar%26rft.auinit1%253DV.%26rft.volume%253D342%26rft.issue%253D6165%26rft.spage%253D1508%26rft.epage%253D1512%26rft.atitle%253DC57BL%252F6N%2BMutation%2Bin%2BCytoplasmic%2BFMRP%2Binteracting%2Bprotein%2B2%2BRegulates%2BCocaine%2BResponse%26rft_id%253Dinfo%253Adoi%252F10.1126%252Fscience.1245503%26rft_id%253Dinfo%253Apmid%252F24357318%26rft.genre%253Darticle%26rft_val_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Ajournal%26ctx_ver%253DZ39.88-2004%26url_ver%253DZ39.88-2004%26url_ctx_fmt%253Dinfo%253Aofi%252Ffmt%253Akev%253Amtx%253Actx [10]: /lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEzOiIzNDIvNjE2NS8xNTA4IjtzOjQ6ImF0b20iO3M6MjQ6Ii9zY2kvMzQzLzYxNjkvMzcwLjEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9