Azathioprine and Ulcerative Colitis: A “Second-Look” Meta-Analysis

Abstract

Dear Editor, We read with interest the article by Leung et al., a systematic review and meta-analysis of the efficacy of azathioprine in maintaining clinical remission in ulcerative colitis [1]. Since its publication, this article has been cited at local and national educational conferences as a synthesized estimate of the efficacy of azathioprine in maintenance of remission in ulcerative colitis. We embarked on replicated statistical analysis, analogous to a ‘‘secondlook’’ endoscopy, in order to provide clinicians and investigators with another perspective on employing the statistical tool of meta-analysis in the study of comparative effectiveness. Our conclusion is that there is not sufficient homogeneity to warrant a conclusion in either direction. We advocate for employing meta-analysis here as a closer look at the differences in this small number of individual studies rather than as an attempt to estimate a pooled effect of azathioprine. The choice of metrics, heterogeneity of controls, small sample size, and pitfalls of quality score reporting are important ingredients in this analysis. The choice of metric in a meta-analysis is important and should address the clinical question. In this meta-analysis, the relative risk metric showed a lack of statistical significance of a difference between the effect of azathioprine and controls at the greater than 95% confidence level in the random effects model, yet not in the fixed effects model. On our reexamination of the data, the log risk ratio difference, risk difference, and arc sine difference metrics all showed a small significant advantage of azathioprine over controls in fixed effects models. The risk difference metric showed a significant advantage of azathioprine over controls in a random effects model; however, the other metrics did not (Fig. 1). Overall, the authors’ conclusion of a trend toward efficacy based on limited data is supported by these analyses. However, the significance of this trend depends on which metric and model are utilized. There was significant heterogeneity in the control groups and response rates. Whereas random effects models may be able to account for unmeasured sources of heterogeneity among studies, the comparison here of a treatment to either placebo [2–4] or an active control [5, 6] is analogous to comparing major league baseball batters to a group of minor and little league batters. The efficacy of these controls ranged from 19 to 64%, i.e. two studies ranged from 19 to 23%, one study 44%, and two studies 62–64%. Compounding these differences in control rates, the studies compare therapies in patients with different levels of severity and length of disease (steroid-dependent [5], mild-to-severe [2], severe [3, 6], and newly diagnosed/ severe [4]). An example of how these factors confound the analysis is the inclusion of the study by Sood et al. in 2003 [6] in an estimate of average effect of azathioprine where the control rate of response was 62%. This study compares azathioprine and sulfasalazine in patients with newly diagnosed ulcerative colitis who often respond to first-line high-dose 5-aminosalicylic acid therapy alone. Removing this study from the meta-analysis, the relative risk random effects models now shows a significant advantage of azathioprine over controls (Fig. 2). B. G. Levesque (&) Center for Health Policy/Primary Care and Outcomes Research and Division of Gastroenterology, Stanford University School of Medicine, 117 Encina Commons, Stanford, CA 94305, USA e-mail: bglevesq@stanford.edu