A Comparison of Synchronous and Asynchronous Remote Usability Testing Methods

Abstract

ABSTRACT This paper presents the results of a study that compared two types of remote usability testing methods: the synchronous and asynchronous methods. The two methods were compared through an evaluation of hospital websites and involved three points of comparison: number and nature of usability problems discovered, overall task performance, and participants’ testing experiences. The results show significant differences between the two methods. The asynchronous method led to the detection of a greater number of usability problems relating to content and navigation. The participants also found the asynchronous method to be more comfortable and convenient for them to use. They also felt it was easier to concentrate on tasks. However, no differences were found between the methods in terms of tasks performance metrics. The study concludes that the asynchronous method seems to outperform the synchronous one for testing hospital websites, as it is equally useful in revealing high-severity problems, and more profitable for detecting considerable number of usability issues, as well as ensuring that the usability testing experience is as natural as possible for participants. Moreover, it can be conducted with a larger sample size and without the need for the presence of any evaluators during the testing sessions, thus reducing financial and temporal costs.