Misjudgment of interrupted time-series graphs due to serial dependence: Replication of

Abstract

AbstractInterrupted time-series graphs are often judged by eye. Such a graph might show, for example, patient symptom severity (y) on each of several days (x) before and after a treatment was implemented (interruption). Such graphs might be prone to systematic misjudgment because of serial dependence, where random error at each timepoint persists into later timepoints. An earlier study (Matyas & Greenwood, 1990) showed evidence of systematic misjudgment, but that study has often been discounted due to methodological concerns. We address these concerns and others in two experiments. In both experiments, serial dependence increased mistaken judgments that the interrupting event led to a change in the outcome, though the pattern of results was less extreme than in previous work. Receiver operating characteristics suggested that serial dependence both decreased discriminability and increased the bias to decide that the interrupting event led to a change. This serial dependence effect appeared despite financial incentives for accuracy, despite feedback training, and even in participants who had graduate training relevant to the task. Serial dependence could cause random error to be misattributed to real change, thereby leading to judgments that interventions are effective even when they are not.