Comparator bias: why comparisons must address genuine uncertainties.

Abstract

Researchers may believe – and patients and physicians may hope – that a particular treatment (perhaps because it is new) is better than other available treatments; but it may often turn out to be worse. When the British Medical Research Council’s controlled trial of streptomycin for pulmonary tuberculosis was conceived in 1946, none of the therapies used to treat the disease had been shown in controlled clinical trials to be useful; indeed, one controlled trial had shown gold salt therapy to do more harm than good. Although streptomycin was known to be useful in forms of tuberculosis which had previously always been fatal, there was uncertainty about how useful the new drug would be in pulmonary tuberculosis, from which patients often recovered after treatment with bed rest alone. Patients in the MRC trial were accordingly randomized either to bed rest alone, or to bed rest and streptomycin. The same reasoning is applicable when controlled trials are designed today. After considering systematic reviews of the relevant existing evidence, patients and their doctors must be substantially uncertain about which among the treatment options – including no active treatment – is preferable. This implies ensuring that no patient who agrees to participate in the trial will knowingly be disadvantaged, whichever one of the comparison treatments the patient is assigned to receive. Clinical trials are done to reduce uncertainties, and they should only be done if clinicians and their patients are uncertain which of the existing alternatives is preferable. This requirement is sometimes referred to as ‘the uncertainty principle’ or ‘equipoise’. If one or more of the treatments selected for the comparison in a trial is known to be worse than others, not only will some participants in the trial be denied effective treatment, but this ‘comparator bias’ will result in unfair tests of treatments. Even if other sources of bias have been well controlled in such studies, their results will mislead patients and their doctors. Unfortunately, comparator bias is sometimes deliberately introduced for just this purpose, usually with a view to showing that new treatments are preferable to existing alternatives.