Evaluating single-case research data: A comparison of seven statistical methods

Abstract

This study examined and compared the performances of seven popular or promising techniques for analyzing between-phase differences in single-case research designs. The techniques are: (a) Owen White's binomial test on extended Phase A baseline (White & Haring, 1980), (b) D. M. White, Rusch, Kazdin, and Hartmann's Last Treatment Day technique (1989), (c) Gorsuch's trend analysis effect size (Faith, Allison, & Gorman, 1996; Gorsuch, 1983), (d) Center's mean-only and mean-plustrend models (Center, Skiba, & Casey, 1985-1986), and (e) Allison's mean-only and mean-plus-trend models (Allison & Gorman, 1993; Faith et al., 1996). The techniques were assayed by applying them to a set of 50 single-case AB design (baseline and intervention) data sets, constructed to represent a range of type and degree of intervention effects. From analysis of these 50 data sets, four questions were answered about the analytic techniques: (a) How much statistical power is possessed by the more promising techniques? (b) What typical R 2 effect sizes are evidenced for graphed data sets which, according to visual analysis, show a positive intervention effect? (c) How do the five analytic techniques covary with one another? and (d) To what extent does each technique tend to produce autocorrelated residuals? The debate over whether visual analysis is sufficient for single-case research data has reached stasis, if it has not been resolved. The advantages of visual analysis (Baer, 1977; Michael, 1974; Parsonson & Baer, 1978, 1986) are widely acknowledged, but statistical analysis is now recommended to supplement visual analysis in most cases. Kazdin (1982) acknowledged that statistical procedures may be of value when (a) there is no stable baseline, (b) expected treatment effects cannot be well predicted, as with a new treatment, or (c) statistical control is needed for extraneous factors in naturalistic environments. Huitema (1986), another strong proponent of visual analysis, recommends adding statistical analyses when unambiguous results must be shared with other professionals (p. 228). Franklin, Gorman, Beasley, and Allison (1996) conclude a recent book chapter on the subject by emphasizing