Virtual Reality Scene Research Articles

The presence of an avatar can reduce cybersickness in Virtual Reality

Virtual Reality (VR) applications are increasingly being utilized for research, healthcare, and education. Despite their benefits, many VR users report motion sickness-like sensations (cybersickness), such as headache, disorientation, or nausea. Previous studies suggest that the sense of presence (“being there”) in the virtual world may contribute to the severity of cybersickness; however, results have been contradictory, with some studies reporting a negative and some reporting a positive relationship between the two. The goal of the current study was to further investigate how presence and cybersickness are related. Participants (N = 54) were exposed to a VR scene presented on a head-mounted display showing a 15-minute-long passive movement through space. The level of presence was manipulated by including an avatar (astronaut suit with hand-tracking) or no avatar in the virtual environment. Results showed that the avatar group reported significantly less severe cybersickness compared to the no-avatar group. We also found significant, negative correlations between some of the presence metrics (immersion, sensory fidelity) and cybersickness, indicating that cybersickness severity decreased as the level of presence increased. These findings suggest that more immersive VR experiences using an avatar may potentially reduce the risk of experiencing cybersickness.

Emotion recognition based on a limited number of multimodal physiological signals channels

Emotion constitutes a higher-order cognitive process, moreover, physiological signals can offer a more objective and realistically reflection of human emotional state. Electroencephalograph (EEG) signals, controlled by the central nervous system, are highly sensitive to emotional state fluctuations. Current research primarily focuses on recognizing emotions through feature extraction from multi-channel EEG signals. However, the acquisition of multi-channel EEG signals is hindered by difficulties acquiring from hair occlusion. The objective of this study is to construct a real-time physiological-emotion recognition model based on 2-channel EEG signals from the forehead without hair occlusion combined with 3-channel peripheral physiological (PPS) signals, employing signal preprocessing, feature extraction, feature normalization, and the ensemble learning algorithm Light Gradient Boosting Machine (LightGBM). In this paper, thirty participants sequentially viewed fourteen virtual reality (VR) scenes, rated for valence and arousal using the Self-Assessment Manikin (SAM) scale after each scene. The 2-channel EEG signals and 3-channel PPS signals were recorded synchronously from forehead by VR Head-Mounted Displays (HMD) Bio Pad developed in our research center. Our proposed model achieved median accuracy of 89.68% for valence, 90.11% for arousal in binary classification, and 84.93% for four classification of emotion recognition based on the collected samples. Furthermore, verification result of the proposed method on a public database (DEAP) achieved accuracy of 84.03%, 84.37%, and 72.23% on valence, arousal, and four classification, respectively. The proposed physiological-emotion recognition model, utilizing a limited number of multimodal physiological signals channels from wearable devices, demonstrates higher accuracy compared with the results in some literature, suggesting its potential for real-time implementation due to limited signal channels and low runtime.

Deep learning-driven optimization of real-time ray tracing for enhanced immersive virtual reality

Photorealistic rendering is essential for immersive virtual reality and real-time ray tracing is required for this, yet the necessary computational load has so far hindered its use in scenarios where a fast response and high frame rates are paramount. To overcome this challenge, we present a deep learning-based model that optimises the computational load of ray tracing through utilising convolutional neural networks (CNNs) to predict lightscene interactions. We achieve this by training CNNs from datasets with offline ray-traced images. Using this learning process, we approximate the contents of light transport simulations from scene point sampling. The use of this optimised algorithm in virtual reality scenes, thus, enables renderings of more complex scenes, with dynamic lighting, complex geometry and interactive elements under high frame rates, keeping users immersed and responsive to events. Our results show that this method achieves visual similarities to traditional ray tracing, which is photorealistic rendering, but can be performed in real-time in virtual reality. We anticipate this work leads to many potential applications of ray tracing in many scenarios in virtual reality, such as gaming, architectural rendering and training.

Alone Together, Together Alone: The Effects of Social Context on Nonverbal Behavior in Virtual Reality

Abstract Social virtual reality (VR), by definition, focuses on people using networked VR systems to bring avatars together. Previous studies have examined how different factors affect social interaction, in small groups such as dyads or triads. However, in a typical social VR scene there tends to be dozens of avatars, even those not directly interacting with a given user. Furthermore, beyond the virtual environment, VR users are also situated in various immediate physical social contexts. In two field experiments, we investigate how the presence of virtual and physical people contextualize and influence nonverbal behaviors. Study 1 examines virtual context and asks how interacting with others in a private or public virtual environment influences nonverbal outcomes during interactions in a social VR platform. Across two sessions, participants (n = 104) met either in a private virtual environment with their group members alone or in a public environment surrounded by four other groups. Results showed that participants moved their avatars slower and stood closer to group members in public versus private environments. Study 2 examines physical context and asks how interacting with virtual others while physically together or alone influences nonverbal behaviors. Participants (n = 61) met in virtual environments while they were in either a shared physical environment or separated physical environments. Results showed that, compared to remote participants, participants who were physically together moved their bodies more slowly, but their avatars faster. Moreover, there was more mutual gaze among remote participants. We discuss implications to theories of social influence in VR.

ColocZStats: a z-stack signal colocalization extension tool for 3D slicer.

Confocal microscopy has evolved to be a widely adopted imaging technique in molecular biology and is frequently utilized to achieve accurate subcellular localization of proteins. Applying colocalization analysis on image z-stacks obtained from confocal fluorescence microscopes is a dependable method of revealing the relationship between different molecules. In addition, despite the established advantages and growing adoption of 3D visualization software in various microscopy research domains, there have been few systems that can support colocalization analysis within a user-specified region of interest (ROI). In this context, several broadly employed biological image visualization platforms are meticulously explored in this study to understand the current landscape. It has been observed that while these applications can generate three-dimensional (3D) reconstructions for z-stacks, and in some cases transfer them into an immersive virtual reality (VR) scene, there is still little support for performing quantitative colocalization analysis on such images based on a user-defined ROI and thresholding levels. To address these issues, an extension called ColocZStats (pronounced Coloc-Zee-Stats) has been developed for 3D Slicer, a widely used free and open-source software package for image analysis and scientific visualization. With a custom-designed user-friendly interface, ColocZStats allows investigators to conduct intensity thresholding and ROI selection on imported 3D image stacks. It can deliver several essential colocalization metrics for structures of interest and produce reports in the form of diagrams and spreadsheets.

Optimization design of low-carbon building thermal energy based on optical sensing and virtual reality image scene reconstruction

With the intensification of global climate change and energy crisis, the design of low-carbon buildings has gradually become a research hotspot in the field of architecture. Effective thermal optimization design not only reduces energy consumption, but also improves the comfort and sustainability of the building. This study aims to explore the optimization design of low-carbon building heat energy based on light sensing technology, combined with virtual reality image scene reconstruction, to provide an innovative design method to achieve intelligent management and optimization of building heat energy. Through the introduction of light sensing technology, the light intensity and heat distribution inside and outside the building are monitored, and the combination of data analysis and model simulation is used to evaluate the heat use of the building. On this basis, virtual reality scenes are constructed to reconstruct the heat flow and distribution of buildings, providing designers with intuitive visualization tools. The research shows that the design method based on light sensing can effectively identify the areas of heat waste in the building, and reduce the energy consumption of the building by optimizing the window layout, adding greening and improving the thermal management system. Virtual reality image scene reconstruction makes the design process more intuitive, helps the design team to quickly adjust and optimize, and improves the efficiency and accuracy of the design.

FBSA-Net: A novel model based on attention mechanisms for emotion recognition in VR and 2D scenes

Recent studies have found that electroencephalographic (EEG) features from different frequency bands and different brain regions contribute differently to emotion recognition (ER) in virtual reality (VR) and two-dimensional (2D) scenes. Despite achievements in the task of ER in both VR and 2D environments, there has not been much prior effort to develop a unified ER model that can be applied to both VR and 2D environments. We propose a novel Frequency-Band-Spatial-based Attention Network, named FBSA-Net, for ER in VR and 2D scenes. Specifically, FBSA-Net adaptively captures the frequency-band-spatial relationship of EEG signals through cascade fusion of frequency-band attention (FBA) module and spatial attention (SA) module. The FBA module can automatically assign FBA weights to each band feature. The SA module then applies the ProbeParse attention mechanism to adaptively explore channel relationships both within and across brain regions, thus facilitating a comprehensive understanding of the global features of the EEG signals. The model's efficacy was validated through the VRSDEED dataset, created from emotion induction experiments using VR and 2D induction programs. Multiple comparative experiments have shown that FBSA-Net achieved state-of-the-art recognition results on VRSDEED datasets, achieving average recognition accuracies of 96.63% and 96.49% in VR and 2D scenes, respectively. The research results indicate that the significant contribution of FBSA-Net is not only reflected in the ability to provide ER using the same model in VR and 2D environments, but also in its excellent ability to distinguish between the emotional states in these two scenarios, providing a valuable reference for using the same model for ER.

The comparison between intratympanic gentamicin prehabilitation and postoperative virtual reality exposure to standard vestibular training in patients with vestibular schwannoma.

Resection of the vestibular schwannoma causes acute peripheral vestibular loss. The process of central compensation starts immediately afterward. The rehabilitation goal is to support this process and restore the quality of life. In this prospective single-center study, 67 consecutive patients underwent vestibular schwannoma resection (40 females, mean age 52 ± 12 years). The patients were divided into three groups: the prehabilitation with intratympanic gentamicin group, the virtual reality group (optokinetic stimulation via virtual reality goggles in the first ten days after the surgery), and the control group. All patients were examined with objective methods and completed questionnaires before the prehabilitation, before the surgery, at the hospital discharge, and after three months. Intratympanic gentamicin prehabilitation leads ipsilaterally to a significant aVOR reduction in all semicircular canals (p < 0.050), the increase of the unilateral weakness in air calorics (p = 0.026), and loss of cVEMPs responses (p = 0.017). Prehabilitation and postoperative exposure to virtual reality scenes improved the patient's perception of vertigo problems according to Dizziness Handicap Inventory (p = 0.039 and p = 0.076, respectively). These findings conform with the optokinetic testing results, which showed higher slow phase velocities at higher speeds (40deg/s) in both targeted groups compared to the control group. Preoperative intratympanic gentamicin positively affects peripheral vestibular function, influencing balance perception after VS resection. In long-term follow-up, prehabilitation and postoperative exposure to virtual reality improve patients' quality of life in the field of vertigo problems.

A Collaborative Neural Network-based Automatic Object Recognition Method For Environmental Space Under Virtual Reality Scenes

A Collaborative Neural Network-based Automatic Object Recognition Method For Environmental Space Under Virtual Reality Scenes

Exploring geospatial digital twins: a novel panorama-based method with enhanced representation of virtual geographic scenes in Virtual Reality (VR)

An important step in implementing geospatial digital twins is to enhance the expressiveness of virtual geographical scenes for the physical world. However, the existing virtual geographical scenes cannot quickly express the dynamically changing geographic environment for remote users due to the inefficient handling of modeling processes, user perception, and remote sharing. The research analysed the concept and characteristics of geospatial digital twins, and constructed the virtual geographical scene ontology, based on which we developed geographical spatiotemporal semantic rules and designed a dynamic annotation algorithm to enhance the representation of virtual geographical scenes. Finally, we investigated a real-time transmission method of panoramic video based on 5 G and used immersive virtual reality (IVR) to realize the user experience of remote immersion in geographical scenes. We selected a specific geographic environment containing multiple typical geographic entities to develop three prototype systems for experimental analyses. The results showed that the proposed method enabled users to view the virtual geographical scene on a VR device. The average latency for this process was 14.72 seconds. Compared with the virtual geographical scenes constructed by traditional methods, the experiments showed the proposed method advantageous in comprehensiveness, timeliness, and photorealism and abilities to enhance the user’s geographical scene perception.

Assessment and Application of Digital Museum Visitors' Emotional Experience Based on Virtual Reality Technology and Emotion Recognition Algorithm

Abstract Each person will have a different emotional experience for the same scene or exhibit, which reduces the accuracy of emotional recognition and leads to the complexity of the evaluation of visitors' emotional experience in digital museums. In order to improve the user experience of digital museums, the evaluation and optimization methods of visitors' emotional experience of digital museums based on virtual reality (VR) technology and emotion recognition algorithm are studied. A spectrogram is generated according to the voice sent by tourists when they visit the digital museum, which is based on the CSWNet_CRNN emotion recognition depth learning model input, which evaluates tourists' emotional experience results and draws their emotional responses to digital museums. The visual and auditory features of the digital museum scene with positive emotional experience are extracted, respectively. Using VR technology, the extracted features are applied to each link of the digital museum scene content design, optimizing the digital museum VR scene and improving the digital museum experience. The experiment shows that the tourist emotion recognition accuracy of this method is high, and the emotion recognition accuracy of 300 random tourists can reach 100%. In terms of generating new scenes, the feature extraction results of this scene are consistent with the feature estimation of positive emotions by ordinary people. The use of extracted features to optimize the digital museum scene has better realism and detail accuracy, which can be favored by most people and promotes the sustainable development of digital museums.

Where was this thing again? Evaluating methods to indicate remembered object positions in virtual reality.

A current focus in sensorimotor research is the study of human perception and action in increasingly naturalistic tasks and visual environments. This is further enabled by the recent commercial success of virtual reality (VR) technology, which allows for highly realistic but well-controlled three-dimensional (3D) scenes. VR enables a multitude of different ways to interact with virtual objects, but only rarely are such interaction techniques evaluated and compared before being selected for a sensorimotor experiment. Here, we compare different response techniques for a memory-guided action task, in which participants indicated the position of a previously seen 3D object in a VR scene: pointing, using a virtual laser pointer of short or unlimited length, and placing, either the target object itself or a generic reference cube. Response techniques differed in availability of 3D object cues and requirement to physically move to the remembered object position by walking. Object placement was the most accurate but slowest due to repeated repositioning. When placing objects, participants tended to match the original object's orientation. In contrast, the laser pointer was fastest but least accurate, with the short pointer showing a good speed-accuracy compromise. Our findings can help researchers in selecting appropriate methods when studying naturalistic visuomotor behavior in virtual environments.

Attentional Bias, Pupillometry, and Spontaneous Blink Rate: Eye Characteristic Assessment Within a Translatable Nicotine Cue Virtual Reality Paradigm.

Incentive salience processes are important for the development and maintenance of addiction. Eye characteristics such as gaze fixation time, pupil diameter, and spontaneous eyeblink rate (EBR) are theorized to reflect incentive salience and may serve as useful biomarkers. However, conventional cue exposure paradigms have limitations that may impede accurate assessment of these markers. This study sought to evaluate the validity of these eye-tracking metrics as indicators of incentive salience within a virtual reality (VR) environment replicating real-world situations of nicotine and tobacco product (NTP) use. NTP users from the community were recruited and grouped by NTP use patterns: nondaily (n=33) and daily (n=75) use. Participants underwent the NTP cue VR paradigm and completed measures of nicotine craving, NTP use history, and VR-related assessments. Eye-gaze fixation time (attentional bias) and pupillometry in response to NTP versus control cues and EBR during the active and neutral VR scenes were recorded and analyzed using ANOVA and analysis of covariance models. Greater subjective craving, as measured by the Tobacco Craving Questionnaire-Short Form, following active versus neutral scenes was observed (F1,106=47.95; P<.001). Greater mean eye-gaze fixation time (F1,106=48.34; P<.001) and pupil diameter (F1,102=5.99; P=.02) in response to NTP versus control cues were also detected. Evidence of NTP use group effects was observed in fixation time and pupillometry analyses, as well as correlations between these metrics, NTP use history, and nicotine craving. No significant associations were observed with EBR. This study provides additional evidence for attentional bias, as measured via eye-gaze fixation time, and pupillometry as useful biomarkers of incentive salience, and partially supports theories suggesting that incentive salience diminishes as nicotine dependence severity increases.

Review of joint attention deficit intervention based on virtual reality for children with autism

Joint attention deficit is one of the core disorders in children with autism, which seriously affects the development of multiple basic skills such as language and communication. Virtual reality scene intervention has great potential in improving joint attention skills in children with autism due to its good interactivity and immersion. This article reviewed the application of virtual reality based social and nonsocial scenarios in training joint attention skills for children with autism in recent years, summarized the problems and challenges of this intervention method, and proposed a new joint paradigm for social scenario assessment and nonsocial scenario training. Finally, it looked forward to the future development and application prospects of virtual reality technology in joint attention skill training for children with autism.

Application of Graph Neural Network in Matching Intangible Cultural Heritage Models with Virtual Reality Scenes

Application of Graph Neural Network in Matching Intangible Cultural Heritage Models with Virtual Reality Scenes

Evaluation of Rural Healing Landscape DESIGN Based on Virtual Reality and Electroencephalography

From the user’s perspective, emotional elements are increasingly being used in design. Researchers have indicated that healing landscapes in rural areas play a positive role in soothing human emotions. In this study, a landscape with healing functions was designed, and 32 subjects experienced emotions in a virtual reality (VR) scene while their 32-channel electroencephalography (EEG) signals were collected. This study compared the brain responses with and without the presence of healing landscape elements and conducted correlation coefficient analysis using eight different regression prediction models to examine the relationship between security, comfort, positivity, and corresponding healing landscape elements. The results show significant improvements in emotions of security, comfort, and positivity post-exposure to the landscape design, especially with certain elements, such as seating, shrubs, and tree pools. EEG data indicate enhanced emotional and cognitive states, particularly relaxation, with increased activity in specific brain regions. The decision tree regression model is the most suitable for our data. It reveals strong correlations between specific healing landscape elements and emotional responses. In the comfort category, “shrubs” show the highest correlation (R² = 0.82), while in the security category, “trees” have the highest correlation (R² = 0.77). Similarly, in the positivity category, “trees” again exhibit the highest correlation (R² = 0.71) with EEG data, indicating their significant impact on these emotional dimensions. This study demonstrates the importance of using scientific methods, such as EEG technology, to validate the principles of emotional design and also underscores the role of green environments in enhancing psychological health and emotional comfort.

The impact of virtual reality scenes on stress response characteristics of individuals with different personality traits

Virtual reality based physical stress (VRPS) paradigms could eliminate the influence of social factors on participants, and it may be a desirable tool to explore the impact of personality traits on stress levels. In this study, we attempt to explore the effects of VRPS on stress response among individuals with different personality traits. Forty male participants with an average age of 22.79 ± 0.41 years were divided into two groups based on Harm Avoidance (HA) scores of Tridimensional Personality Questionnaire (TPQ), referred to as the Low-HA group and the High-HA group. The stress levels of the participants were assessed using salivary α-amylase (sAA) activity and heart rate variability (HRV) indices pre- and post-stress. The influence of personality traits on stress response among different groups was analyzed. VRPS significantly affected the sAA activity and HRV indicators of both groups. During and after stress, there were significant differences in sAA activity and HRV indicators between the two groups. The sAA levels and HRV indices of the Low-HA group were lower than those of the High-HA group. Furthermore, sAA levels and HRV indices were correlated with the scores of TPQ. VRPS scenarios elicit different stress responses on individuals with different harm avoidance personality traits. Stress evaluation based on VR scenarios presents potential in personality trait assessments, particularly for distinguishing between individuals with low and high HA tendencies.

Spatial Contraction Based on Velocity Variation for Natural Walking in Virtual Reality.

Virtual Reality (VR) offers an immersive 3D digital environment, but enabling natural walking sensations without the constraints of physical space remains a technological challenge. Previous VR locomotion methods, including game controller, teleportation, treadmills, walking-in-place, and redirected walking (RDW), have made strides towards overcoming this challenge. However, these methods also face limitations such as possible unnaturalness, additional hardware requirements, or motion sickness risks. This paper introduces "Spatial Contraction (SC)", an innovative VR locomotion method inspired by the phenomenon of Lorentz contraction in Special Relativity. Similar to the Lorentz contraction, our SC contracts the virtual space along the user's velocity direction in response to velocity variation. The virtual space contracts more when the user's speed is high, whereas minimal or no contraction happens at low speeds. We provide a virtual space transformation method for spatial contraction and optimize the user experience in smoothness and stability. Through SC, VR users can effectively traverse a longer virtual distance with a shorter physical walking. Different from locomotion gains, the spatial contraction effect is observable by the user and aligns with their intentions, so there is no inconsistency between the user's proprioception and visual perception. SC is a general locomotion method that has no special requirements for VR scenes. The experimental results of our live user studies in various virtual scenarios demonstrate that SC has a significant effect in reducing both the number of resets and the physical walking distance users need to cover. Furthermore, experiments have also demonstrated that SC has the potential for integration with existing locomotion techniques such as RDW.

That was a foul! How viewing angles, viewing distances, and visualization methods influence football referees’ decision-making

A football referee must choose the right position on the field in order to perceive a foul accurately. In previous research, decision-making for foul calls was compared across different situations within a match or even across different matches, which questions the comparability of decisions that often depend on the situational context. In this study, we experimentally controlled this situational context to investigate the effect of viewing distance and viewing angle on referees’ decision-making. We recreated real-life decision-making situations using markerless tracking devices and virtual reality (VR) animation processes to render first-person situations. We compared the decision-making accuracy of ten experienced football referees when judging situations from three different viewing angles and distances. Furthermore, we compared their decision-making accuracy in VR scenes with accuracy when viewing the same scenes from a bird’s eye and slow-motion perspective, i.e., with the perspectives used in official referee training. We found that decision accuracy is not affected by viewing angles and distances, but that the referee’s decision certainty is lower when viewing the foul situation from a greater distance. The visualization of the situations had an impact on the referee’s decision-making but only for detecting a simulation (i.e., a player acts like they have been fouled). Slow-motion scenes led to higher decision-making accuracy than VR scenes. Our study suggests a way to recreate VR situations for experimental studies, allowing researchers to control the context of each situation. Our findings help to disentangle the mixed findings of previous studies on distance and viewing angle effects on decision accuracies. Finally, we propose potential avenues to improve VR animation processes in future studies.

Exploring enigmas: Information seeking after exposure to virtual reality awe elicitors

AbstractBecause of awe's properties as a knowledge emotion, awe elicitors can increase awareness of knowledge gaps, boost scientific interest, and promote inquiry. However, the relationship between awe and exploratory behavior, such as information seeking, remains unclear. Using a multi‐method approach, this study asked how and to what extent awe fosters information seeking. This question was examined through a two‐pronged approach. First, in a laboratory setting, participants (n = 32) were exposed to a variety of awe elicitors through a virtual reality (VR) head‐mounted display. Participants' quantitative and qualitative responses were gathered immediately after exposure in the laboratory as well as 24 h later through questionnaires. Second, after establishing a stratified sample of participants who voluntarily conducted information seeking (n = 8), the study shifted to phenomenologically‐informed interviews. This study found that exposure to specific VR scenes piqued participants' curiosity, especially toward representations of phenomena with unknown or unexplained origins. However, self‐motivated exploration only occurred in limited circumstances, particularly toward awe elicitors tinged with supernatural causality. In sum, this study introduces a new research direction within information science, illustrates how understudied awe elicitors pique curiosity, and provides a nuanced, qualitative report on the phenomenon of technology‐induced awe.