Multimodality for Passive Experience: Effects of Visual, Auditory, Vibration and Draught Stimuli on Sense of Presence

Abstract

Adequate use of multimodal stimuli plays a crucial role in help forming the sense of presence within a virtual environment. While most of the presence research attempts to engage more sensory modalities to induce a higher sense of presence, this paper investigates the relevance of each sensory modality and different combinations on the subjective sense of presence using a specifically designed scenario of a passive experience. We chose a neutral test scenario of “waiting at a train station while a train is passing by” to avoid the potential influence of story narrative on mental presence and replicated realistic multimodal stimuli that are highly relevant to our test setting. All four stimuli - visual, auditory, vibration, and draught - with 16 possibilities of combinations were systematically evaluated with 24 participants. The evaluation was performed on one crucial aspect of presence – “realness” to reflect user presence in general. The perceived realism value was assessed using a scalometer. The findings of main effects indicate that the auditory stimuli had the most significant contribution in creating the sense of presence. The results of interaction effects suggest the impact of draught stimuli is significant in relation to other stimuli - visual and auditory. Also, the gender effects revealed that the sense of presence reported by female participants is influenced by more factors than merely adding more sensory modalities.