Beware of Over-Interpreting Negative Trials

Abstract

The report by Tyrer et al. (2008) raises questions about the fit between study aims and study design. The study randomly assigned adults with intellectual disability to either risperidone (N = 29), haloperidol (N = 28) or placebo (N = 29) under double blind conditions. The primary target of treatment was aggressive behavior as measured by the Modified Overt Aggression Scale (MOAS) (Ratey and Gutheil 1991). The investigators note that aggressive behavior is a frequent problem in this population, that antipsychotic medications are commonly used in clinical practice and that the evidence supporting their use to treat aggression in this population is meager. The investigators conducted a pragmatic trial that included a relatively broad range of subjects across a range of settings. We agree with current practice guidelines that antipsychotic medications should not be administered without careful consideration of alternative treatments and documented evidence of recurring episodes of aggression (Rush and Frances 2000). It is not clear that the subjects in this study satisfied these clinical criteria. Therefore, the results of this study are difficult to interpret and fail to provide much direction for clinicians and families faced with trying to manage aggressive behavior in developmentally disabled adults. Eligible subjects were evaluated at Baseline, week 4, week 12 and week 26. The primary analysis was conducted on the change in the MOAS from baseline to week 4. Several secondary outcomes were also evaluated. At week 4 and all subsequent time points, there were no significant differences between groups on any outcome measures. Indeed, on some measures at week 4, subjects on placebo did slightly better than those on active medication. Upon initial review, these results appear to be straightforward and easy to interpret. However, designing a pragmatic trial and then suggesting that the results bear on the question of efficacy undermines interpretation. The goal of a pragmatic (or practical) trial is to address a problem of clinical decision-making (Tunis et al. 2003). In its pure form, a pragmatic trial would compare two evidenced based treatments to assist clinicians in selecting which one should be selected first (based on cost, convenience of use, lower adverse effect burden, etc). By contrast, an efficacy trial is designed to determine whether a treatment is effective. The decision to conduct a pragmatic trial or an efficacy trial has a major impact on study design. First, efficacy trials are more restrictive in sample selection. For example, it is common to set a reasonably high threshold on the primary outcome measure to ensure that there is room for improvement. By contrast, pragmatic trials tend to be more permissive in subject selection. In the Tyrer et al. trial, subjects with two aggressive episodes in the past week were eligible to enroll, suggesting that enrollment was based on recent incidents rather than persistent aggression. Similarly, the range at baseline on the MOAS (8–30) suggests a wide range of severity. If L. Scahill (&) Child Study Center, Yale University, 230 S. Frontage Rd, P.O. Box 207900, New Haven, CT 06520, USA e-mail: lawrence.scahill@yale.edu