Statistical Verification of Remote Usability Testing Method

Abstract

In this paper we present our approach to statistical verification of the proposed remote usability testing method based on data gathered from participants of the test. Remote usability testing is becoming more and more popular, because it allows to perform usability tests faster and cheaper than with traditional laboratory testing, while providing similar results.We have constructed a new, hybrid, usability testing method that allows to perform remote unmoderated usability evaluation of web-based systems. This method works as an application that is injected into the website web pages, and gathers data from its users while they are performing given tasks in the evaluated website. To verify the effectiveness of this method we have created a benchmark website with implemented specified number of different usability issues. After that we have performed an usability evaluation of that website using the hybrid method. We wanted to find out what percentage of the implemented usability issues will be found with this method with the specified number of participants. We assumed that the satisfactory percentage of found usability issues is 85% and we distinguished the number of participants of the remote usability test to obtain such threshold. The next step was to verify the statistical significance of obtained results.This paper presents an overview of the created method, the description of the performed experiment and statistical verification that was performed for the obtained data.