The Relationship Between System Effectiveness and Subjective Usability Scores Using the System Usability Scale

Abstract

This article examines the relationship between users’ subjective usability assessments, as measured using the System Usability Scale (SUS), and the ISO metric of effectiveness, using task success as the measure. The article reports the results of two studies designed to explore the relationship between SUS scores and user success rates for a variety of interfaces. The first study was a field study, where stereotypical usability assessments on a variety of products and services were performed. The second study was a well-controlled laboratory study where the level of success that users were able to achieve was controlled. For both studies, the relationship between SUS scores and their attendant performance were examined at both the individual level and the average system level. Although the correlations are far from perfect, there are reliable and reasonably strong positive correlations between subjective usability measures and task success rates, for both the laboratory and field studies at both the individual and system level.