The calibration of treatment effects from clinical trials to target populations

Abstract

Standard practice teaches that randomized clinical trials should be preferred to observational studies if the aim is to obtain unbiased estimates of efficacy. It is often acknowledged that treatment efficacy might differ in the general population than in the clinical trial, but the source of those differences is often poorly understood or not regarded as important. Substantive differences between RCT and observational study results are typically ascribed to confounding of the latter. Recent literature [1,2] suggests that differences between the results seen in an RCT and in an observational study can be larger than expected not because of confounding, but because of differences between their respective patient populations. These efficacy estimates can differ enough to change treatment recommendations. The paper by Weisberg et al. in this issue [3] is a very clear and innovative example of such emerging literature. But there is another important message that one should derive from this work. That is, RCT results are useful only if we can calibrate their results to predict treatment efficacy in the target population of interest. This editorial makes the case for why calibration requires both clinical knowledge from observational studies, and new statistical insights.