Beyond Jacobson and Truax: Estimation of Clinical Significance Trajectories in the Coping Power Intervention Using Measurement Error-Corrected Multilevel Modeling

Abstract

Coping Power (CP) is a preventive intervention that focuses on reducing child externalizing problems. Although it is typically delivered in a group format (GCP), individually delivered CP (ICP) has produced greater mean reductions in externalizing problems. However, standard analysis of randomized trials loses individual-level information regarding which youth improve, fail to improve, or get worse, whereas clinically significant change (CSC) metrics capture information on individual change. The present study is a secondary analysis of an ICP/GCP trial (N = 360) that examines differences in CSC-based individual-level inferences on externalizing. A novel method for assessing CSC under measurement error-corrected multilevel modeling was used, overcoming three limitations of traditional CSC methods: (a) restriction to two time points, (b) use of total scores, and (c) assumption of constant reliability across time and participants. Because of concerns about Type II errors with all CSC methods, an individual-level effect size metric for CSC was also developed. Based on individualized Cohen’s d estimates, individual-level improvements in externalizing from 4th through 11th grades of d ≥ 0.5 were significantly greater in ICP (73%) versus GCP (45%). Further, GCP saw significantly higher percentages of youth with worsening of externalizing, underscoring concerns about diminished effects for GCP. Half of the sample had improvement that was not statistically significant but exceeded d ≥ 0.5, highlighting the susceptibility to Type II errors of CSC’s results based on statistical significance. An examination of ICP/GCP differences under advanced CSC analysis gives more nuanced information than conventional randomized controlled trial analysis and greater precision in estimating individual-level outcomes than standard CSC methodologies.