How People Judge the Usability of a Desktop Graphic User Interface at Different Time Points: Is there Evidence for Memory Decay, Recall Bias or Temporal Bias?

Abstract

Abstract The System Usability Scale (SuS) survey is a widely respected tool for measuring usability. Generally, a SuS score is administered directly after a usability test to assess the usability and user experience of digital products. However, some researchers have used SuS as a survey as part of longitudinal ‘in the wild’ trials where SuS is often completed some period after the trial. The aim of this research was to determine if a participant’s memory of a product’s usability would change if a SuS survey was administered at different times after a test. Hence, we sought to understand if recalling the usability of a digital technology was affected by temporal bias or memory decay. This paper includes results and findings from two studies, study 1 involved evaluating a web application and study 2 involved evaluating a virtual learning environment. Collectively the two studies had 212 participants (n = 212). The findings conclude that there is no significant change of the user’s recollection of the usability of digital product as evidenced by an analysis of users who completed multiple SuS surveys over a short term period of 3 weeks or over an extended period of time of 6 months. RESEARCH HIGHLIGHTS The system usability scale is used to test for memory decay and temporal bias in judging the user experience of technologies at different time points. 212 participants took part in two studies ranging from 3 weeks to 6 months. There is no evidence that there is a temporal bias or memory decay when users complete a SuS survey at the two different time points of 3 weeks and 6 months.