A field study of the relationship and communication between Chinese evaluators and users in thinking aloud usability tests

Abstract

Thinking aloud is the most widely applied usability evaluation method. In order to get reliable usability problems, it is necessary for the evaluators to establish a supportive relationship and communicate effectively with the users. This study investigated the relation and communication between the evaluators and test users in Chinese usability testing sessions. Field observations and interviews were conducted in five companies in Beijing. This research was based mainly on Nisbett's cultural theory and Boren and Ramey's thinking aloud model. The results of the study showed that Chinese users focused mainly on tasks, while evaluators focused on both users and tasks. Further, Chinese users did not think aloud actively; thus, in order to encourage users to speak out, effective communication skills were required for Chinese evaluators. Retrospective thinking aloud and explanation were also used in the tests. Finally, it discussed that communication was appropriate for the formative evaluations, but not for the summative evaluations.