Publication bias in medical informatics.

Abstract

This issue of JAMIA includes two articles submitted in response to a special call, issued on February 29, 2000, for papers reporting “null, negative, or disappointing results.” The motivation for this call was a working hypothesis that medical informatics, like other fields, is afflicted with the academic malady known as publication bias. Publication bias exists when well-executed studies with null results (no significant differences between groups), negative results (favoring the control or placebo arm of the study), or disappointing results (effects that may be positive but of little practical significance) do not find their way into the archival literature. Publication bias therefore skews the archival literature toward work with positive findings. Negative studies have been shown to be 2.6 times less likely than positive studies to reach publication, which creates the potential to distort conclusions drawn from systematic reviews and meta-analyses.1 This phenomenon may have multiple causes—manuscripts may not be recommended for publication by reviewers and editors who see negative or null results as scientifically unimportant, and investigators may not write up such results because they are seen as unlikely to be published—or perhaps embarrassing if they are. Potential embarrassment may be a potent deterrent to informatics researchers, because many investigators also hold administrative positions that might be compromised if these individuals' own studies revealed negative or ambiguous effects of expensive technology that they themselves built or purchased. In response to the call for manuscripts, we received six letters of intent and four completed manuscripts. After review and revision, two manuscripts addressing complementary topics were accepted and appear in this issue. The first, by Patterson and Harasym,2 examines the impact of an educational program for medical students; the second, by Rocha and colleagues,3 explores the impact of a clinical decision support system on clinicians. As the process …