Does Supplementing IS Analysts’ User Observations With Hands-on Training Help Them Better Understand Users’ Work?

Abstract

IS analysts need to acquire knowledge about users’ work processes to design high-quality systems. While researchers have proposed hands-on activities in cognitive learning theories to improve knowledge acquisition, current approaches rely on analysts verbally communicating with users or observing them perform their tasks in order to learn these work processes. We draw on social cognitive theory (SCT) to hypothesize and examine how effectively two learning approaches (an observation-only approach and an observation plus hands-on approach) help analysts better understand users’ computer-mediated work processes. Accordingly, we conducted an experimental study to compare these two learning approaches. We found that, while participants who had low prior domain knowledge about users’ work processes ended up understanding them better in the observation plus hands-on treatment than in the observation- only treatment, the difference between the two approaches was not significant for participants who had high prior domain knowledge.