Reliability and accuracy of visually analyzing graphed data from single-subject designs.

Abstract

As the number of single-subject research reports in the occupational therapy literature increases, the reliability of clinical decisions made from single-subject data must be determined. This study examined the ability of occupational therapists to reliably and accurately analyze data from single-subject designs using visual inspection of graphed data. Forty-six therapists provided a rating on whether or not a clinically significant change in performance had occurred across the baseline and treatment phases for five graphs of hypothetical data. The results revealed that considerable disagreement existed in making visual judgments based on the five graphs. The data sets were also analyzed using a recently developed statistical procedure for use with single-subject data. The level of agreement between the quantitative analysis and the visual analysis was also poor, suggesting inconsistency in the ability to make accurate visual judgments of single-subject data. The advantages and limitations of graphic presentation and visual inspection of single-subject data are discussed and the argument is made that some form of quantitative analysis should generally be used in conjunction with the visual inspection of graphed data.