To Animate or Not to Animate Usability Scales: The Effect of Animation on Questionnaire Experience and Psychometric Properties

Abstract

The Hybrid Usability Inventory (HUI) is a usability questionnaire that uses a combination of pictorial and verbal information to express the meaning of its items. This study aimed to extend the static pictorial representation by using animations. Previous research has not yet addressed the positive or negative outcomes of using animations in pictorial questionnaires. We hypothesized that an animated questionnaire would have an additional positive effect on respondents’ questionnaire experience, motivation, and preferences without impinging psychometric properties. The goal of the present study was to compare the static HUI with an animated version (AniHUI) in an online test setting. Respondent-centered aspects (questionnaire experience) and psychometric properties (sensitivity, validity, reliability) were assessed. Participants (N = 192) interacted with a website prototype (either high or low usability) and subsequently assessed the website’s usability either with HUI or AniHUI, the System Usability Scale (SUS), and further measures of interest. Results suggest that AniHUI did not differ substantially from HUI. However, both the static and animated scales were superior to the SUS regarding respondent-centred measures. Findings suggest that the HUI and the AniHUI are engaging and reliable scales that can be used in research and practice.