Effect size matters: a key neglected indicator of comparative trial quality

Abstract

The non-inferiority trial by Nidich and colleagues1Nidich S Mills PJ Rainforth M et al.Non-trauma-focused meditation versus exposure therapy in veterans with post-traumatic stress disorder: a randomised controlled trial.Lancet Psychiatry. 2018; 5: 975-986Summary Full Text Full Text PDF PubMed Scopus (49) Google Scholar reported that transcendental meditation was non-inferior to prolonged exposure therapy for the treatment of post-traumatic stress disorder. They used random assignment, blinded assessment, a reasonable sample size, and other hallmarks of an adequate randomised controlled trial. However, the within-group effect for the prolonged exposure therapy had a Cohen's d effect size for change from baseline of 0·66. This effect is far below that which has previously been reported for prolonged exposure. Although it can be difficult to define a normative result for treatments, particularly experimental ones, in the case of prolonged exposure the threshold is informed by over 30 years of evidence. A meta-analysis of randomised controlled trials in which psychotherapy was given for post-traumatic stress disorder identified 13 trials that investigated exposure therapies and nine trials that investigated exposure therapies plus some additional cognitive behavioural therapy components.2Bradley R Greene J Russ E Dutra L Westen D A multidimensional meta-analysis of psychotherapy for PTSD.Am J Psychiatry. 2005; 162: 214-227Crossref PubMed Scopus (1333) Google Scholar The authors reported a mean within-group intention-to-treat effect size for exposure of 1·57 (95% CI 1·11–2·04) and a mean effect size for exposure plus cognitive behavioural therapy components of 1·66 (1·18–2·14). Another review focusing only on treatment of military-related post-traumatic stress disorder identified eight open trials of prolonged exposure reporting intention-to-treat outcomes,3Steenkamp MM Litz BT Psychotherapy for military-related posttraumatic stress disorder: review of the evidence.Clin Psychol Rev. 2013; 33: 45-53Crossref PubMed Scopus (104) Google Scholar which showed a weighted mean within-group effect size of 2·06. Large prolonged exposure effect sizes have also been reported in trials done by proponents of non-exposure oriented evidence-based therapies4Resick PA Nishith P Weaver TL Astin MC Feuer CA A comparison of cognitive-processing therapy with prolonged exposure and a waiting condition for the treatment of chronic posttraumatic stress disorder in female rape victims.J Consult Clin Psychol. 2002; 70: 867Crossref PubMed Scopus (948) Google Scholar and for outcomes related to therapists giving prolonged exposure for the first time in their careers.5Eftekhari A Ruzek JI Crowley JJ Rosen CS Greenbaum MA Karlin BE Effectiveness of national implementation of prolonged exposure therapy in Veterans Affairs care.JAMA Psychiatry. 2013; 70: 949-955Crossref PubMed Scopus (257) Google Scholar We are not primarily concerned with highlighting potential aspects of Nidich and colleagues' study that could account for the attenuated clinical outcomes. Instead, we note that the prolonged exposure effect size is well below expectations, and this in itself is an indicator of trial quality. Nidich and colleagues' Article represents a worrying trend in comparative randomised controlled trials on post-traumatic stress disorder, namely the ostensible validation of alternative therapies in comparison with gold-standard treatments that did not actually achieve the gold standard expected effect size, thus raising questions over claims of non-inferiority. Accordingly, we believe that consideration of threshold effect sizes should be included in the method of rigorous comparison trials to support trial quality and generalisability. We do not wish to detract from the relative general benefits of transcendental meditation or discourage scientists from doing valid examinations of alternative or adjunctive therapies for post-traumatic stress disorder, as these are clearly needed. However, investigators researching head-to-head comparisons with established evidence-based therapies should have the burden of proof that the evidence-based therapy, as delivered, is adequately effective before comparing outcomes to the experimental therapy. Non-trauma-focused meditation versus exposure therapy in veterans with post-traumatic stress disorder: a randomised controlled trialA non-trauma-focused-therapy, TM, might be a viable option for decreasing the severity of PTSD symptoms in veterans and represents an efficacious alternative for veterans who prefer not to receive or who do not respond to traditional exposure-based treatments of PTSD. Full-Text PDF