Bringing reality to virtual reality: the role of environmental richness in improving construction practitioners’ VR experience

The use of virtual reality (VR) technology to induce emotional responses has recently become more common in psychological studies. The majority of these studies have been restricted to seated VR experiences where the participant remains in a sedentary position. The purpose of the current thesis is to utilise room-scale VR to increase presence, agency and potency of virtual environments (VE) designed to induce embodied emotional responses. The Evaluative Space Model (ESM) [Cacciopio et. al 2012] was used as the theoretical basis for this programme of research, which was particularly concerned with avoidance responses to negative stimuli, perception of threat and negativity bias. A number of unique VEs were created using Unreal Engine 4 designed to create an illusion of height and the potential for a virtual fall as a source of threat. These VEs were supplemented by additional tracking sensors and an integrated approach to data collection wherein behavioural interactions and movements within the VE were synchronised with ambulatory methods from psychophysiology, e.g. facial electromyography (fEMG), skin conductance level (SCL). The first study (N=20) utilised a VE that requires participants to walk on a wooden plank between the rooftops of two buildings, two versions of the VE were created: sedentary version operated via gamepad controller and a room scale version with natural sensorimotor mappings. The study revealed greater psychophysiological reactivity for the room-scale version of the environment. The second study (N=34) introduced an elaborated room-scale VE where participants must traverse a grid of translucent ice blocks suspended at height in order to reach an end-goal within a physical space of 9m2. This grid contained three types of ice block: solid (low-threat), crack (mid-threat) or fall (high threat). The number of crack and fall blocks were increased over three levels of the VE in order to manipulate threat. The foot movements of participants were tracked as the primary mode of interaction with the VE. The study revealed: (i) higher incidence of risk-averse behaviours as threat increased, (ii) this pattern of behaviour was enhanced for participants with higher levels of trait neuroticism, and (iii) greater reactivity from the corrugator muscle in the period following a two-feet movement. The third study (N=20) represented an extension of study two where a significantly larger version of the ice block VE was created in a physical space of 27m2. In this experiment, the level of threat (i.e. number of crack and fall blocks) was increased, sustained and decreased in order to study adaptation to reduced threat level. In addition, a ‘ground level’ version of the VE was utilised as a control to study the effect of virtual height in isolation. The results of this study revealed: (i) participants adjusted behaviour to increased threat and decreased threat, but only in the presence of virtual height, and (ii) increased activation of zygomaticus during interactions with crack blocks, which suggests this muscle may be associated with a ‘grimace’ response in this context. The final experimental chapter represents a re-analyses of the data from studies 2 and 3 designed to explore individual differences as predictors of risk averse behaviour in response to the threat. These analyses identified trait neuroticism and age as traits that significantly influenced the magnitude of the negativity gradient in response to threat. The implications of the research for studying emotional experiences in VR are discussed.

User Experience and Engagement in the Reality–Virtuality Continuum: A Special Issue Guest Editorial

Gradual transitions and their effects on presence and distance estimation

Virtual reality stroke rehabilitation - hype or hope?

The effectiveness of Virtual Environments (VEs) may be defined in terms of different aspects. A common one is the degree to which the VE creates in the user the subjective illusion of presence, the sense of being in the VE. Since presence is a subjective condition, it has most commonly been measured by subjective methods, especially by post-questionnaires. However, objective measures have been proposed, mainly based on Electrodermal Activity (EDA), Heart Rate (HR) and Behavior Observation (BO). In this work, it was investigated the possibility of dynamically modulating the sense of presence in a Virtual Reality (VR) experience, based on physiological measures on the fly. Although these measures have already been applied for assessment in VR, whether they can be used to dynamically modulate the presence in VR experiences remain an open question. Subjective, physiological and behavioral reactions of 27 subjects during a fully immersive experience under three different conditions (NS: None Stimulus, PS: Pre-set Time Stimulus and DS: Dynamic Stimulus) were observed, varying the stimulus based on HR and EDA variation. Although it was not found a difference for the SUS presence questionnaire, it was found a significant difference for the physiological measures between the NS and DS condition, showing that it was possible to dynamically modulate the VR experience using real-time HR and EDA to increase the effectiveness of VR experiences.

Table for Two is a short parallel interactive narrative in cinematic virtual reality. This virtual reality experience is a creative artefact that has adopted screen production enquiry as a practice-based research. This research aims to explore alternative narrative possibilities in virtual reality. Virtual reality is one of the latest disruptions in cinema that has challenged filmmakers with new methods for 360-degree storytelling. This challenge has led to several studies recommending grammar to guide the users’ gaze toward intended areas in the virtual environment. However, it is argued that this aspect limits the full potential of virtual reality. This study on parallel interactive narratives in virtual reality addresses this limitation. Here, the user is addressed as a navigator of the experience. Table for Two offers navigation freedom to look anywhere they desire in the virtual reality environment. A number of studies on virtual reality narratives suggests that theatre, cinema, amusement park and video game design techniques are relevant for virtual reality productions. This research describes how the fusion of these research fields and art forms contributed to new methods for exploring cinematic virtual reality and interactive 360-degree video narratives. Several studies suggest that virtual reality experiences have similarities with theatre, and therefore knowledge relating to theatre experiences can benefit virtual reality production. For this reason, the casting talent of Table for Two was selected from the local theatre group. It was observed that their experience with improvisations and performance for long takes significantly contributed to the virtual reality production. This paper refers to Linda Aronson’s description of parallel narratives in the context of cinema. Parallel narratives have been present in limited quantity through artistic experiments in early cinema. However, the rapid growth of interactivity in narratives has increased in parallel narrative forms in mainstream cinema. It is observed that parallel narratives are gradually emerging in virtual reality experiences. However, there is a lack of knowledge of the design and development processes. While there is a difference between cinematic and VR narrative, Aronson’s parallel narrative forms in cinema, such as tandem, multiple protagonist, double journey, split screen and consecutive narratives, can be applied in cinematic virtual reality. Additionally, space design in amusement parks reveals the significance of infusing the narrative into the set. One key feature called embedded narratives has been incorporated into the set design of Table for Two. Furthermore, interactive narratives for video game design played an essential role in shaping the navigation experience in Table for Two. Along with the influence of these mediums, Table for Two encountered unique challenges with the performance from multiple 360-degree cameras, customized script writing methods, integrating all filming material into the set design, and navigation interactivity, which does not alter the outcome of the story world. This paper discusses the challenges and considerations for creating a parallel interactive narrative in cinematic virtual reality. The theoretical discourse presented in this paper suggests how the convergence of various mediums contributes to new narrative possibilities in virtual reality.

Psychophysiology of rhythmic stimuli and time experience in virtual reality

This research article introduces a virtual reality (VR) experience designed as a “memoryscape” to immerse users in the life and work of Nikos Skalkottas, a significant yet relatively unknown Greek composer of the 20th century. The VR experience integrates archival material, such as Nikos Skalkottas’s compositions, personal letters, and photographs, along with fictional snippets of the composer’s thoughts and memories, to engage users in an interactive journey through his reconstructed memories of 1930’s Berlin. The VR experience has been the focus of a user study with 37 participants, aiming to assess the effectiveness of the experience to promote a deeper connection with the archival material and the composer’s life and work. The study employed a mixed-method approach, with pre- and post-experience questionnaires and interviews. The outcomes revealed a positive attitude towards the VR experience, with participants expressing enjoyment, curiosity, and a sense of immersion. Users particularly enjoyed interacting with the virtual environment and appreciated its aesthetic quality. While the experience evoked historical empathy, helping users learn new aspects of the composer’s life and work, it had a limited impact on their perception of his music, especially for those who had no prior familiarity. Overall, the VR experience was effective in introducing users to Nikos Skalkottas and his world. Its immersive nature sparked curiosity and interest among the participants, providing a unique opportunity for users to engage with classical music history in an innovative and engaging way.

Being there again – Presence in real and virtual environments and its relation to usability and user experience using a mobile navigation task

The modern eating environment has been implicated as a driving force of the obesity epidemic. Mixed reality applications may improve traditional methodological assessments of eating behavior by improving the ecological validity of the laboratory setting. Research experts evaluated the utility and ecological validity of a mixed reality application that allowed immersion within virtual environments through utilizing the passthrough cameras of the head mounted display to view and interact with real foods. An initial evaluation was conducted that involved three virtual environments: a traditional laboratory booth, a non-textured restaurant, and a full-textured restaurant. The feedback from the initial evaluation was used to create a new virtual restaurant environment and a subsequent evaluation was conducted. Nearly all research experts suggested adding social cues such as people and background noise to create a more authentic and ecologically valid experience. The experts scored the new virtual restaurant environment to be more acceptable than eating or conducting research in a sensory booth but scored lower when compared to conducting research in a real-world restaurant setting. The results of this evaluation suggest that mixed reality applications may be a new methodology to assess environmental influences of eating behavior and may be a promising direction for eating behavior and sensory science research.

We describe a multimodal dataset of paired head and eye movements acquired in controlled virtual reality environments. Our dataset includes head and eye movement for n = 25 participants who interacted with four different virtual reality environments that required coordinated head and eye behaviors. Our data collection involved two visual tracking tasks and two visual searching tasks. Each participant performed each task three times, resulting in approximately 1080 seconds of paired head and eye movement and 129,611 data samples of paired head and eye rotations per participant. This dataset enables research into predictive models of intended head movement conditioned on gaze for augmented and virtual reality experiences, as well as assistive devices like powered exoskeletons for individuals with head-neck mobility limitations. This dataset also allows biobehavioral and mechanism studies of the variability in head and eye movement across different participants and tasks. The virtual environment developed for this data collection is open sourced and thus available for others to perform their own data collection and modify the environment.

Although the interaction technology for virtual reality (VR) systems has evolved significantly over the past years, the text input efficiency in the virtual environment is still an ongoing problem. We deployed a word-gesture text entry technology based on gesture recognition in the virtual environment. This study aimed to investigate the performance of the word-gesture text entry technology with different input postures and VR experiences in the virtual environment. The study revealed that the VR experience (how long or how often using VR) had little effect on input performance. The hand-up posture has a better input performance when using word-gesture text entry technology in a virtual environment. In addition, the study found that the perceived exertion to complete the text input with word-gesture text entry technology was relatively high. Furthermore, the typing accuracy and perceived usability for using the hand-up posture were obviously higher than that for the hand-down posture. The hand-up posture also had less task workload than the hand-down posture. This paper supports that the word-gesture text entry technology with hand-up posture has greater application potential than hand-down posture.

Cybersickness, or feelings of nausea, discomfort or unease, are common in virtual reality experiences with head-mounted displays. With the widespread availability of virtual reality headsets across a wide domain of uses including industry, defence, education and the commercial market, is it critical that virtual environments are developed that minimise cybersickness. Unfortunately, determining whether a virtual reality experience will induce cybersickness is difficult. Typically this requires user studies with a completed, or almost completed, virtual environment. This is time consuming and expensive, both to run participant-based user studies and for any rework to the virtual environment needed due to identified issues. As part of modern iterative development processes it would be useful to pre-evaluate virtual environments for cybersickness before engaging user studies. This paper presents a new approach and metric to compare virtual environments’ susceptibility to induce cybersickness. The approach combines visual optical flow, an entropy metric of complexity and a cumulative time-series measure. Virtual environments with known cybersickness attributes are used to demonstrate the approach. Results indicate that the approach can successfully differentiate between known levels of cybersickness and attributes contributing to cybersickness, such as motion direction and field of view.

Enhancing VR experience with RBF interpolation based dynamic tuning of stereoscopic rendering

An integrated emotional and physiological assessment for VR-based active shooter incident experiments