There is Always a Way: Organizing VR User Tests with Remote and Hybrid Setups during a Pandemic—Learnings from Five Case Studies

Sanni Siltanen,Paulina Becerril Palma,Viveka Opas,Alisa Burova,Markku Turunen,Phong Truong,Hanna Heinonen

doi:10.3390/mti5100062

Abstract

(1) COVID-19 pandemic restrictions caused a dramatic shift in research activities, forcing the adoption of remote practices and methods. Despite the known benefits of remote testing, there is limited knowledge on how to prepare and conduct such studies in the industrial context where the target users are experts and company employees. (2) In this article, we detail how we organized VR user tests in five industrial cases during the pandemic, focusing on practicalities and procedures. We cover both on-site testing, including disinfecting and other safety protocols, as well as remote and hybrid setups where both remote and on-site participants were involved. Subject matter experts from eight countries were involved in a total of 22 tests. (3) We share insights for VR user test arrangements relevant to the pandemic, remote and hybrid setups, and an industrial context, among others. (4) Our work confirms that with careful planning it is possible to organize user tests remotely. There are also some limitations in remote user testing, such as reduced visibility and interaction with participants. Most importantly, we list practical recommendations for organizing hybrid user tests with safety and disinfecting procedures for on-site VR use.

Highlights

Published: 11 October 2021The COVID-19 pandemic has caused critical economic and social disruption all over the world [1,2], leading to a significant transformation of work and research-related activities [3,4]
With the spread of COVID-19 related restrictions, many authors proposed methods to maintain remote XR research [44,45,46,47] and demonstrated the methods and practicalities of remote user studies, which we present in this sub-section
The combined results of the five user tests can be categorized into the following groups: general user test arrangements, insights that are important because of new and fascinating technologies, insights that are important due to a pandemic, insights that are important because of the remote setup, insights that are important because of the industrial context, and insights regarding remote interviews

Summary

Introduction

The COVID-19 pandemic has caused critical economic and social disruption all over the world [1,2], leading to a significant transformation of work and research-related activities [3,4]. While the regulations to follow “work from home” and, “research from home” [5] come from the governmental level, academia and industry research organizations have faced several challenges in managing their traditional workflows. Restrictions on physical contact and social isolation have massively affected the research projects related to the use of Extended Reality (XR) and immersive environments, where in-lab experiments with the presence of a moderator and a participant in one space accounted for more than. XR technologies, which include augmented, mixed, and virtual realities (AR, MR, and VR), is a hot topic of HCI (human-computer interaction) research among academic and industrial practitioners. XR technologies offer unlimited benefits in the industrial context via blurring the boundaries between what is real and what is virtual, extending the possibilities of traditional technology appliances and supporting industrial

Objectives

Methods

Results

Discussion

Conclusion