Using Concurrent Verbal Reports to Examine Data Analyst Verbal Behavior

Abstract

Evidence concerning effects of graphed data on the verbal behavior of data analysts was reviewed and revealed the fact that no verbal protocol analyses of this behavior appears in the OBM literature. A task analysis of data analysts' verbal behavior was conducted, verbal responses of two expert and two novice data analysts were collected as they viewed time-series data one data point at a time from three data sets, and these data were subjected to protocol analyses. Results were the following: (1) experienced data analysts made considerably more frequent references to data points in addition to the most recently presented data point for all three data sets than did novices, (2) experienced data analysts took more time to complete the task than did novice analysts, and (3) experienced data analysts were somewhat more accurate at identifying onset (location in time) of interventions. These results were discussed in terms of their implications for future research concerning analysis and development of visual inspection skills among data analysts. It was concluded that there exists a need to develop data sets or tasks that more reliably discriminate among expert and novice performances if verbal protocol methodology is to be fully exploited by OBM researchers.