Abstract
<p class="first" id="d167451e74">Synaesthesia is a phenomenon where senses naturally combine resulting in, for example, ‘seeing’ music or ‘hearing’ colours. It is of interest in the field of Human-Computer Interaction as a way of creating new or enhanced experiences and interactions with Mixed Reality technologies. In Virtual Reality, research has mainly focused on evaluating advanced graphics and capturing immersion levels and User Experience within ‘typical’ and ‘expected’ interactions. This paper investigates how multimodal design characteristics can lay the foundations to a more ‘synaesthetic’ design approach in Mixed Reality to identify how ‘atypical’ interactions can also affect User Experience. 20 participants completed a maze activity, emotion and immersion surveys and interviews. Results suggest a significant increase in surprise, pride and inspiration and a decrease in interest and enthusiasm. The visual and audio aspects were well received by participants and the sensory elements had a positive effect on User Experience. Time perception was measured and 90 per cent of participants’ time estimations were longer than the actual time. Change blindness was investigated with most participants not noticing the visual or audio changes. Finally, we discuss how this study can inform future projects which aim to implement a synaesthetic-oriented and multimodal approach in Mixed Reality design.
Highlights
In recent years, Mixed Reality (MR) technologies have become more advanced and more prominent in fields of healthcare (McLay et al, 2014; StriemAmit, Guendelman & Amedi, 2012), commerce (Van Kerrebroeck, Brengman & Willems, 2017), as well as leisure, with Virtual Reality (VR) headsets such as the Oculus Rift (Oculus, 2019) and Augmented Reality (AR) games such as Pokémon Go (Niantic, 2016)
In VR, research has mainly focused on evaluating advanced graphics and capturing immersion levels and User Experience (UX) of interactions that stem out of the ‘typical’ and ‘expected’
Our study suggests that visual and audio aspects had a positive effect on UX both statistically and within interviews
Summary
In recent years, Mixed Reality (MR) technologies have become more advanced and more prominent in fields of healthcare (McLay et al, 2014; StriemAmit, Guendelman & Amedi, 2012), commerce (Van Kerrebroeck, Brengman & Willems, 2017), as well as leisure, with Virtual Reality (VR) headsets such as the Oculus Rift (Oculus, 2019) and Augmented Reality (AR) games such as Pokémon Go (Niantic, 2016). In VR, research has mainly focused on evaluating advanced graphics and capturing immersion levels and User Experience (UX) of interactions that stem out of the ‘typical’ and ‘expected’. Immersion and presence were measured by surveys taken before and after the study. This was followed by a semi-structured interview conducted over videoconferencing about their experiences
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have