Initial validation of the GUESS-18 for usability in virtual reality gaming environments: a pilot study

Abstract

Virtual reality (VR) technology has developed rapidly in recent years with head-mounted displays (HMDs), which provide immersive VR experiences by inserting users into an artificially constructed digital world. However, there is currently no standardized tool for measuring usability in VR environments. The purpose of this study was to provide initial validation of the GUESS-18, a usability questionnaire for video games, for measuring usability of VR in a gaming environment. Participants played a racing game with an HMD in VR and a traditional monitor and completed the GUESS-18 and the System Usability Scale (SUS). Confirmatory factor analysis was performed to determine GUESS-18 validity. The Comparative Fit Index (CFI) and Root Mean Square Error of Approximation (RMSEA) values indicate appropriate model fit, with values of 0.967 and 0.056, respectively. Internal consistency (Cronbach’s alpha) was satisfactory for all factors except for social connectivity. Scores from the GUESS-18 were significantly higher with VR by about 6.4%, but SUS scores were equivalent between display types. Playing the game in VR resulted in greater immersion and personal gratification than playing with the monitor. Overall, the GUESS-18 can measure usability of VR games as it distinguishes VR from traditional monitor environments and shows satisfactory validity and consistency.