Virtual Indoor Environment Research Articles

The effect of distance on audiovisual temporal integration in an outdoor virtual environment

In this study, we explore the influence of stimulus distance on human tolerance for (physical) asynchronies in virtual reality (VR). A repeated audiovisual (AV) stimulus with sound and light bursts was presented to the participants in an outdoor virtual environment (VE) using a head-mounted display (HMD). The investigation focused on quantifying the point of subjective simultaneity (PSS) for both visual and auditory stimuli. A synchrony judgment method (SJ-3) was used for 11 stimulus onset asynchronies (SOA) and five egocentric distances from 10 m up to 50 m with 10 m increments. The data analysis showed negative PSS values that decreased with distance, resulting in a negative slope (-3 ms/m) of the regression line between PSS values and simulated distances. In contrast to the recent study conducted in the indoor VE, we conclude that the presented study in the outdoor VE does not incorporate a distance compensation mechanism and refutes the hypothesis of an ‘implicit estimation’ of sound-arrival time. The reasons behind the observed contrast are discussed in this paper. Moreover, the negative slope of the regression line (−3 ms/m) is similar to previous research, concluding that there is a temporal integration of auditory-visual information within human neural processes without distance compensation.

The influence of spatial size of indoor VR environments on preference evaluation

ABSTRACT Visual perception is vital for people’s spatial experience. Many factors correlate with visual perception including spatial size, material, color, and geometry. However, studies systematically exploring spatial size are sparse, especially with regard to the interactions between the factors of spatial size. In this study, 15 indoor virtual reality environments with five widths and three heights were used to examine the influence of spatial size on visual perception. The study adopted a preference survey as the primary data-gathering tool assessed using a factorial analysis of variance. The results revealed that the influence of width and height on peoples’ preference levels for the indoor virtual reality environments was significant. However, the preference level did not always improve with an increase in width and/or height. A certain spatial size may enhance or contradict visual perception, but changing the width and height does not always exert a significantly different effect on the preference evaluation for the virtual reality environment. This study proposes a new idea that an indoor virtual environment that can adjust its spatial size can be used as a supplement to improve the quality of a real environment.

Veterinary students’ proximity to and interpretation of a simulated “aggressive” dog before and after training

Dog “aggression” in the veterinary practice is commonplace. Therefore, student knowledge and education about dog behaviour and the ability to interpret “aggressive” behaviour is important from a human injury prevention and dog welfare perspective. The study aimed to compare first-year veterinary students’ perceived safest proximity to both an “aggressive” and non-reactive simulated dog, both before and after a teaching intervention about canine behaviour and a handling practical. It also examined student confidence and their ability to identify “aggressive” behaviours. Forty first year veterinary students took part in two surveys. Each survey included two videos: one of a simulated dog displaying “aggressive” behaviour, based on the ‘Canine Ladder of Aggression’; and another displaying non-reactive (passive behaviours without reaction to the participants) behaviours. Each video depicted the slow and consistent approach towards the virtual dog within a virtual indoor environment, and participants were asked to press stop if or when they would stop approaching the dog. In the “aggressive” scenario, there was a reduction in the approach-stop time from survey 1 (median = 17.8 s) to survey 2 (median = 15.2 s) in the intervention group (p = 0.018) but not in the control group (p = 0.147). Regarding confidence, there was a significant increase in the self-reported confidence rating relating to a participant’s ability to interpret canine behaviour in both the control (p = 0.011) and intervention (p = 0.003). In conclusion, these results indicate that students using approach-stop videos stayed further away from an “aggressive” virtual dog model if they had undertaken a canine behaviour educational intervention. This novel approach has the potential for further use in teaching and assessment of student knowledge and behaviour which may otherwise be difficult to demonstrate.

Context-dependent memory recall in HMD-based immersive virtual environments.

The article introduces an original VR-based experiment which explores context-dependent memory recall in humans. It specifically examines the recall of correct and falsely induced semantic memories. With the aid of VR head-mounted displays, 92 students of psychology were placed in a computer-generated indoor virtual environment and asked to memorize the presented lists of words. Afterwards, the participants were placed in the same indoor virtual environment or an alternative outdoor virtual environment and asked to recall the words. The number of correct and falsely induced words was then measured. On average, women recalled significantly more correct words from the list than men, regardless of the environmental context. Despite the assumptions, we did not observe a separate effect of exposure to different environments during learning and recall of material on memory performance. Likewise, we did not detect any effects of the learning context or biological sex in the case of the production of false memories. These results provide a novel insight into previous knowledge regarding the memory processes that occur in virtual environments. Although we failed to confirm the role of context in recalling learned material in general, we found a hint that this context might interact with specific memory processes of biological sexes. However, the design of this study only captured the effect of changing the environment during memory recall and did not address the role of specific context in remembering learning material. Further research is therefore needed to better investigate these phenomena and examine the role of biological sex in context-dependent memory processes.

Quantifying Egocentric Distance Perception in Virtual Reality Environment

In virtual reality (VR) studies, where object distance plays a role of an independent variable, unknown egocentric distance perception values can affect the interpretation of the collected data. It is known that the perceived egocentric distance in VR is often underestimated, which may affect other judgments that implicitly depend on it. In order to prepare later experiments on the effect of distance on audiovisual (a)synchrony perception, this study quantifies the egocentric distance perception in a virtual indoor environment using two methods: verbal judgment (VJ) and position adjustment (PA). For the VJ method, participants verbally estimated the distance between their own position and a cardboard box position at a distance between a nominal 5 m and 30 m, with increments of 5 m. For the PA method, participants were asked to position a cardboard box to an instructed distance of a nominal 5 m to 13 m, with increments of 1 m. Both methods (VJ and PA) showed significant and substantial levels of underestimation, where simulated distance was underestimated on average by 38.5%. Our study suggests taking these findings into account when treating distance as an independent parameter in experiments conducted in purely simulated virtual spaces which do not exist physically in the real world.

EFFECTS OF IMMERSIVE VIRTUAL REALITY IN ENHANCING CREATIVITY

AbstractIn recent years, the use of virtual reality (VR) has been a promising opportunity to improve the immersive experience in virtual environments. In this study, we explore the effectiveness of immersive VR experiences on an individual's creativity. To do this, we first identified the characteristics of VR that are closely related to creative performance. Considering these factors, we designed and implemented the interaction method and the three outdoor and indoor virtual environments (Wilderness, City Park, and Office). We evaluated the effectiveness of the virtual environments with 12 participants. The user study results show that outdoor and indoor immersive virtual experiences improved their creativity skills. We concluded by analysing how factors such as simulator sickness and perceived workload influence creativity levels. Moreover, our study showed the potentiality of using immersive virtual environments to enhance individuals’ creative performance.

Acquisition and retention of spatial knowledge through virtual reality experiences: Effects of VR setup and locomotion technique

The fidelity level of virtual reality (VR) setups can affect different aspects of virtual experiences, but its effects on knowledge acquisition and retention need clarification. The two studies in this paper focus on spatial knowledge. The first study compared three VR setups, one using a VR headset and two using a tablet, differing in display and interaction fidelity. Since the type of virtual environment (VE), and the locomotion technique employed to explore it, might affect spatial knowledge acquisition, we studied each setup in an indoor and an outdoor VE with two widely used locomotion techniques (teleport and steering). Results showed that setups offering higher display and interaction fidelity can improve acquisition of spatial knowledge in terms of distance estimations (with teleport in the indoor VE, and with steering in the outdoor VE) and object-to-object spatial relations (with steering in the outdoor VE), but not in terms of object locations. This can provide guidance about how to choose the appropriate combination of setup and locomotion technique based on the type of VE and the types of spatial knowledge to acquire. Moreover, sickness and usability results showed that VR headset was more usable than tablet, and suggest using teleport on VR headset and steering on tablet. Therefore, the second study focused on VR headset with teleport and tablet with steering, and extended the assessment of their effects to spatial knowledge retention after two weeks. Results showed that spatial knowledge decreased in both conditions, but the VR headset with teleport led to better acquisition and 2-weeks retention of distance estimations.

Moving indoors: a systematic literature review of locomotion in virtual indoor environments

ABSTRACT Virtual indoor environments provide a unique combination of indoor cartography, virtual reality, and virtual locomotion. So far however, the indoor aspect of such an environment seems to be neglected even though it may affect how the locomotion of users is conducted as well as how this affects their spatial behaviour. To get an idea about the state of the research, systematic review of peer-reviewed articles mainly from the Scopus database was conducted in this paper. It aims to get a broad idea about the interest in research of virtual locomotion and to figure out what type of research in the indoor environment was conducted. Special interest was put on the types of locomotion techniques, their interaction aspects and research topics. These characteristics were analysed and compared to earlier systematic reviews in this topic. To accommodate the indoor-related themes, environmental aspects (size) and the motivation for using an indoor environment were also obtained from the relevant articles analysed. This way, the paper tries to provide a basis for further research in using indoor environments in virtual reality.

목표 기반 시각적 이동 작업을 위한 시각적 맥락정보 임베딩과 교착상태 처리

In this paper, we propose a novel deep neural network-based agent model, (VCENet), for performing target-driven visual navigation tasks. Most recent agent models for visual navigation try to recognize the real-time task context by using only objects and their relationships detected from RGB input images. However, such object-oriented visual context does not contain detailed information about the background scene, which is important for ground navigation. Moreover, it may result in a misrecognized context due to because of errors in of the object and relation detectors. To overcome these problems, the proposed VCENet model represents the real-time task context by using as the appearance and geometric features of the background scene extracted from RGB-D input images well as the object relation features derived through from graph embedding. Many existing models, which learn an action policiesy based upon on reinforcement, learning, do not provide any reward functions to avoid deadlocks ahead during of navigation or any effective deadlock recovery policyy learning mechanisms to escape the deadlocks. To address these problems, the proposed VCENet model provides not only reward functions for deadlock avoidance and recovery, but also a deadlock recovery policy based on imitation learning. Conducting a variety of various experiments in photo-realistic virtual indoor environments provided by the 3D simulator AI2THOR, we made sure of demonstrated the superiority of the proposed VCENet model.

Ramp-shaped neural tuning supports graded population-level representation of the object-to-scene continuum

We can easily perceive the spatial scale depicted in a picture, regardless of whether it is a small space (e.g., a close-up view of a chair) or a much larger space (e.g., an entire class room). How does the human visual system encode this continuous dimension? Here, we investigated the underlying neural coding of depicted spatial scale, by examining the voxel tuning and topographic organization of brain responses. We created naturalistic yet carefully-controlled stimuli by constructing virtual indoor environments, and rendered a series of snapshots to smoothly sample between a close-up view of the central object and far-scale view of the full environment (object-to-scene continuum). Human brain responses were measured to each position using functional magnetic resonance imaging. We did not find evidence for a smooth topographic mapping for the object-to-scene continuum on the cortex. Instead, we observed large swaths of cortex with opposing ramp-shaped profiles, with highest responses to one end of the object-to-scene continuum or the other, and a small region showing a weak tuning to intermediate scale views. However, when we considered the population code of the entire ventral occipito-temporal cortex, we found smooth and linear representation of the object-to-scene continuum. Our results together suggest that depicted spatial scale information is encoded parametrically in large-scale population codes across the entire ventral occipito-temporal cortex.

An Exploratory Study on the Effect of Virtual Environments on Cognitive Performances and Psychophysiological Responses.

Research shows that reduced exposure to natural contexts is associated with an increase in psychophysical disorders. Recent evidence suggests that even a brief experience in natural scenarios can positively affect people's health and well-being. However, natural contexts are not always easily accessible. This study investigates the effects of natural and indoor virtual environments (VREs) on psychophysiological and cognitive responses. Following a within-subject design, 34 healthy participants were exposed to two VREs (i.e., a forest and a living room) in a counterbalanced order through a head-mounted display (Oculus Rift). Participants were asked to explore the scenarios and execute a modified version of the Paced Auditory Serial Addition Test. Physiological parameters (heart rate, skin conductance level [SCL], and respiration rate) were recorded during the whole session. After the exposure to VREs, participants filled a set of visual analog scales to rate their subjective experience of presence, relaxation, and stress. Participants reported a higher perceived sense of relaxation in the virtual forest. Moreover, their SCLs were significantly higher in this environment, showing that the forest elicited higher physiological arousal than the living room. Furthermore, their SCLs were significantly higher during the attentional task in the virtual living room. The results suggest that a natural virtual environment can make people feel more relaxed and physiologically engaged than an indoor scenario. The latter instead can be linked to a performing venue, as reported for real contexts. However, these changes were not related to modulations of attentional performance.

Synthesizing scene-aware virtual reality teleport graphs

We present a novel approach for synthesizing scene-aware virtual reality teleport graphs, which facilitate navigation in indoor virtual environments by suggesting desirable teleport positions. Our approach analyzes panoramic views at candidate teleport positions by extracting scene perception graphs, which encode scene perception relationships between the observer and the surrounding objects, and predict how desirable the views at these positions are. We train a graph convolutional model to predict the scene perception scores of different teleport positions. Based on such predictions, we apply an optimization approach to sample a set of desirable teleport positions while considering other navigation properties such as coverage and connectivity to synthesize a teleport graph. Using teleport graphs, users can navigate virtual environments efficaciously. We demonstrate our approach for synthesizing teleport graphs for common indoor scenes. By conducting a user study, we validate the efficacy and desirability of navigating virtual environments via the synthesized teleport graphs. We also extend our approach to cope with different constraints, user preferences, and practical scenarios.

3D Path prediction of moving objects in a video-augmented indoor virtual environment

ABSTRACT Augmented virtual environments (AVE) combine real-time videos with 3D scenes in a Digital Earth System or 3D GIS to present dynamic information and a virtual scene simultaneously. AVE can provide solutions for continuous tracking of moving objects, camera scheduling, and path planning in the real world. This paper proposes a novel approach for 3D path prediction of moving objects in a video-augmented indoor virtual environment. The study includes 3D motion analysis of moving objects, multi-path prediction, hierarchical visualization, and path-based multi-camera scheduling. The results show that these methods can give a closed-loop process of 3D path prediction and continuous tracking of moving objects in an AVE. The path analysis algorithms proved accurate and time-efficient, costing less than 1.3 ms to get the optimal path. The experiment ran a 3D scene containing 295,000 triangles at around 35 frames per second on a laptop with 1 GB of graphics card memory, which means the performance of the proposed methods is good enough to maintain high rendering efficiency for a video-augmented indoor virtual scene.

Impact Sound Generation for Audiovisual Interaction with Real-World Movable Objects in Building-Scale Virtual Reality

In building-scale VR, where the entire interior of a large-scale building is a virtual space that users can walk around in, it is very important to handle movable objects that actually exist in the real world and not in the virtual space. We propose a mechanism to dynamically detect such objects (that are not embedded in the virtual space) in advance, and then generate a sound when one is hit with a virtual stick. Moreover, in a large indoor virtual environment, there may be multiple users at the same time, and their presence may be perceived by hearing, as well as by sight, e.g., by hearing sounds such as footsteps. We, therefore, use a GAN deep learning generation system to generate the impact sound from any object. First, in order to visually display a real-world object in virtual space, its 3D data is generated using an RGB-D camera and saved, along with its position information. At the same time, we take the image of the object and break it down into parts, estimate its material, generate the sound, and associate the sound with that part. When a VR user hits the object virtually (e.g., hits it with a virtual stick), a sound is generated. We demonstrate that users can judge the material from the sound, thus confirming the effectiveness of the proposed method.

Using virtual reality to assess dynamic self-motion and landmark cues for spatial updating in children and adults.

The relative contribution of different sources of information for spatial updating - keeping track of one's position in an environment - has been highly debated. Further, children and adults may differ in their reliance on visual versus body-based information for spatial updating. In two experiments, we tested children (age 10-12 years) and young adult participants on a virtual point-to-origin task that varied the types of self-motion information available for translation: full-dynamic (walking), visual-dynamic (controller induced), and no-dynamic (teleporting). In Experiment 1, participants completed the three conditions in an indoor virtual environment with visual landmark cues. Adults were more accurate in the full- and visual-dynamic conditions (which did not differ from each other) compared to the no-dynamic condition. In contrast, children were most accurate in the visual-dynamic condition and also least accurate in the no-dynamic condition. Adults outperformed children in all conditions. In Experiment 2, we removed the potential for relying on visual landmarks by running the same paradigm in an outdoor virtual environment with no geometrical room cues. As expected, adults' errors increased in all conditions, but performance was still relatively worse in teleporting. Surprisingly, children showed overall similar accuracy and patterns across locomotion conditions to adults. Together, the results support the importance of dynamic translation information (either visual or body-based) for spatial updating across both age groups, but suggest children may be more reliant on visual information than adults.

O&M Indoor Virtual Environments for People Who Are Blind

BACKGROUND: Knowing their current position in the surroundings constitutes one of the biggest challenges faced by people with visual disabilities when they move around. For them, it is difficult to be aware of the direction in which they are going, and the location of nearby objects and obstacles. In this context, obtaining relevant spatial information is always very significant to these individuals. Hence, the research in the development of assistive technologies for needs and perspectives of people who are blind has been a promising area in terms of the orientation and mobility (O&M) challenges. OBJECTIVE: The purpose of this study is to systematically examine the literature on &M virtual environments designed to support indoor navigation to identify techniques for both developing and evaluating the usability and cognitive impact of these applications. METHODS: A systematic literature review (SLR) was performed, considering population, intervention, outcomes, and study design as eligibility criteria. After a filtering process from 987 works retrieved from six databases, we extracted data from 51 papers, which meet the study selection criteria. RESULTS: The analysis of the 51 papers describing 31 O&M indoor virtual environments, indicated that O&M virtual environments to support indoor navigation are usually designed for desktop, adopt spatial audio as way to support orientation, and use joystick as primary interaction device. Regarding evaluation techniques, questionnaires, interviews, user observation, and performance logs are commonly used to evaluate usability in this context. In tests involving users, the participants are usually adults aged 21–59 years, who individually spend about 90 minutes split in usually two evaluation sessions. Most papers do not report any strategies to evaluate the cognitive impact of O&M virtual environments on users’ navigational and wayfinding skills. Thirteen papers (25.49%) reported the conduction of experiments or quasi-experiments and demonstrated pieces of evidence associated with a positive cognitive impact resultant from O&M indoor virtual environments usage. Finally, only four papers (7.84%) reported the development of indoor maps editors for O&M virtual environments. CONCLUSION: Our SLR summarizes the characteristics of 32 O&M virtual environments. It compiles state-of-the-art for indoor simulations in this domain and highlights their challenges and impacts in O&M training. Also, the absence of clear guidelines to design and evaluate O&M virtual environments and the few available computer editors of indoor maps appear as research opportunities.

Mental fatigue level detection based on event related and visual evoked potentials features fusion in virtual indoor environment.

The purpose of this work is to set up a model that can estimate the mental fatigue of users based on the fusion of relevant features extracted from Positive 300 (P300) and steady state visual evoked potentials (SSVEP) measured by electroencephalogram. To this end, an experimental protocol describes the induction of P300, SSVEP and mental workload (which leads to mental fatigue by varying time-on-task) in different scenarios where environmental artifacts are controlled (obstacles number, obstacles velocities, ambient luminosity). Ten subjects took part in the experiment (with two suffering from cerebral palsy). Their mission is to navigate along a corridor from a starting point A to a goal point B where specific flickering stimuli are introduced to perform the P300 task. On the other hand, SSVEP task is elicited thanks to 10 Hz flickering lights. Correlated features are considered as inputs to fusion block which estimates mental workload. In order to deal with uncertainties and heterogeneity of P300 and SSVEP features, Dempster-Shafer (D-S) evidential reasoning is introduced. As the goal is to assess the reliability for the estimation of mental fatigue levels, D-S is compared to multi layer perception and linear discriminant analysis. The results show that D-S globally outperforms the other classifiers (although its performance significantly decreases between healthy and palsied groups). Finally we discuss the feasibility of such a fusion proposal in real life situation.

Visual Odometry Approaches to Autonomous Navigation for Multicopter Model in Virtual Indoor Environment

Using simulators and virtual environments can simplify the development and testing of various control, behavior and navigation scenarios significantly. In this article we describe our experience using the ROS environment and the Gazebo simulator to develop an autonomous multicopter UAV that can navigate an unknown building and locate an exit using computer vision techniques.

The effect of time pressure on stress levels during virtual wheelchair navigation.

In this work stress is assessed based on EEG features. We focus our efforts on the correlation between subjective ratings and cerebral indices during virtual navigation scenarios. Ten male paraplegic subjects took part in the experiment and navigated in a virtual indoor environment. They had to fulfill two missions where time pressure parameter is introduced. Subjects were equipped with Electroencephalography sensors. Temporal and frequency data were filtered and extracted. A correlation study was conducted based on principal component analysis, Fisher's tests and ANOVA. The results showed that the introduction of time pressure has a significant impact on stress levels.

Formal language for the Virtual Geographic Environment

As a new-generation tool for geographic cognition and analysis, the Virtual Geographic Environment (VGE) effectively decreases the difficulty of geographic environment cognition. However, some deeper problems remain in the integration of geographic information and human cognition. This article reviews the relationship between language and cognition, and the formal language for the VGE (VFL) is proposed to solve the problems mentioned above. First, the definition of VFL is given and its characteristics are explained. Second, the construction process of VGE based on VFL is shown. Third, the semantics of VFL are discussed, especially with respect to spatio-temporal relation predications and their functions. VFL’s expression elements are illustrated with a particular focus on classification and representation layers. VFL’s syntax concentrates on the formal model, the rewriting rules and the semantics mapping rules. To narrate VFL’s function and usage in the VGE, two use-cases are identified at different syntax levels: (1) the generation of a thematic point symbol based on the dual semantic structures; and (2) the generation of a virtual indoor environment. Finally, a short summary of the shortcomings of the current VFL is given, and future directions are proposed to facilitate ongoing progress toward forming a comprehensive version of the VGE.