Assessing Change with Preventive Interventions: The Reliable Change Index

Abstract

Determining what constitute meaningful outcomes when evaluating the effectiveness of marital and family interventions is a complex task. Traditionally, the efficacy of a treatment mode has been evaluated by statistical analyses involving comparisons of group means. However, sole reliance on this avenue of analysis has come under criticism as being clinically insufficient. Group means represent averages that say little about the variability of individual outcomes within a sample (Hollon & Flick, 1988; Jacobson & Truax, 1991). Furthermore, if sample sizes are large enough, significant differences may be found in cases where the actual change is relatively small Piercy & Sprenkle, 1990), producing results that are statistically significant but not clinically meaningful. Consequently, a need exists to implement other strategies of analysis that are more useful to clinicians than are traditional comparisons of group means. A number of alternatives have been suggested as supplemental strategies for evaluating treatment outcomes, including frequency distributions (Gurman and Kniskern, 1981), effect sizes (Smith, Glass, & Miller, 1980), and individualized assessments such as goal attainment scaling (Fleuridas, Rosenthal, Leigh, & Leigh, 1990; Kiresuk & Sherman, 1968). Each of these approaches has limitations, however. Frequency distributions provide an overview regarding the proportion of cases exhibiting change but offer no criteria for interpreting what amount of change is meaningful. Effect sizes standardize change in terms of standard deviation units but are not easily translatable for clinical purposes (Saunders, Howard, & Newman, 1988) and are subject to showing large effect sizes with small changes if variability is low (Jacobson & Truax, 1991). Individualized approaches are useful for measuring change on a case-by-case basis, but the fact that assessments are tied to individual cases makes cross comparisons with other participants and studies problematic (Saunders et al., 1988). Jacobson and colleagues (Jacobson, Follette, & Revenstorf, 1984; Jacobson & Revenstorf, 1988; Jacobson & Truax, 1991) have suggested a method for assessing clinically significant change that can be used as a supplement to traditional comparisons of group means. Briefly, they recommend examining data according to two criteria: (a) an established cutoff point for clinically significant change and (b) an index measuring the reliability of that change. Cases that move past the clinical cutoff point with changes large enough so as not to be attributable to error are deemed clinically significant. This approach allows clinicians to have the overview provided by a frequency distribution along with criteria for interpreting the meaningfulness of the data. Although this method has generated discussion in the clinical psychology literature (Hollon & Flick, 1988; Jacobson & Truax, 1991; Saunders et al., 1988), it appears to be largely unnoticed in the marital and family literature, despite the fact that much of Jacobson's work has focused on marital interactions. Further, this approach has been used in assessing change among clinical populations (Davis, Olmstead, & Rockert, 1990; Jacobson, Follette, Revenstorf, Baucom, et al., 1984) but not among normal populations who participate in primary or secondary prevention. The need for outcome research evaluating treatment efficacy is as important for preventive interventions as it is for those of a clinical nature. The purpose of this article, therefore, is threefold: (a) to provide an overview of Jacobson's method, (b) to present data using this method from a normal sample participating in marriage enrichment, and (c) to discuss implications of using this approach with a non-clinical sample. DETERMINING RELIABLE CHANGE Jacobson's method is broken into two phases. The first involves calculating a cutoff point that is clinically significant. …