Thinking-aloud about web navigation

Abstract

The concurrent think-aloud method is a highly valued technique that is used to understand peoples’ taskbased cognitive processes, within both usability testing and the broader study of human-computer interaction. We investigated whether an explicit explanation-based think-aloud instruction leads to differences in navigation performance over the classic think-aloud method. The results showed an interaction between think-aloud instructions and task difficulty. For the low difficulty tasks, there was no difference in task success or in the number of link traversals between the two think-aloud conditions. However, for the more difficult tasks there were differences: participants in the classic condition completed fewer tasks successfully and engaged in more link traversals than participants in the explicit think-aloud condition. A qualitative analysis of participants’ verbal data showed that both think-aloud conditions were dominated by procedural descriptions. The explicit think-aloud did yield proportionally more explanatory utterances, however the performance differences in respect of task success and link traversal suggest that an explicit think-aloud may threaten test validity and therefore evaluators must exercise caution in using this technique.