The Note-Taker’s Perspective During Usability Testing: A Hands-On Approach to Recognizing What’s Important, What Isn’t

Abstract

It is a well-known psychological phenomenon that one’s perceptions are influenced by one’s expectations and biases. This is manifest in behavioral situations ranging from detecting signals in noise to more complex cognitive situations like recounting eyewitness accounts of observed events. Such dynamics also affect our thought processes as we observe usability test participants. Yet the approach to observing user behavior, determining what to look for and what to consider important, seems highly individualized and not often explicated. Even how-to texts on usability testing give relatively little guidance on the process of observing test participants and taking notes on their performance and comments. Consequently, one suspects that some of the variability in usability test results that has plagued the field may be due to a lack of consensus about how to observe user behavior or how to prepare for observation. This alternative format presentation will use collective viewing of video clips from sample usability tests as a stimulant for audience discussion about these issues. We see this as a consensus building exercise, exploring the extent to which the audience can, as a group, agree on what is noteworthy in the clips we present. In doing so we hope to offer some tips and lessons learned that may point towards defining best practices for observing user behavior and comments during usability testing.