Virtual Reality Scene Research Articles

HandBrush for Efficient Object Grouping in Virtual Environment with Bare-Hand

Object grouping task is an important research direction for fast manipulation of a large number of objects. It can help users to improve the efficiency of multi-object manipulation. However, the current research on this aspect is still immature. For this task, in this paper, based on the brush metaphor, we propose a method for grouping objects based on bare hands in virtual reality scenes. We design a number of interactions to facilitate the user's grouping of objects in the three-dimensional virtual space. Object grouping in virtual reality could encompass two subtasks: group generation and group modification. The emphasis of these tasks varies, with the former focusing on creating groups from ungrouped objects and the latter focusing on modifying group members once they are generated. The results of the empirical study show that our method has better performance in accomplishing both sub-tasks compared to the Ray method, Screen method and Cone method.

Industrial Metaverse-Based Intelligent PID Optimal Tuning System for Complex Industrial Processes.

In this article, the method of dynamic performance monitoring and adaptive self-tuning of parameters for actual PID control systems of industrial processes in virtual reality scenes is proposed. This method combines the digital twin model of the PID control process based on system identification and adaptive deep learning and the PID tuning intelligent algorithm based on reinforcement learning with virtual reality and immersive interaction of industrial metaverse. An industrial metaverse-based intelligent PID tuning system is proposed by combining the above method with the end-edge-cloud collaboration technology of Industrial Internet. The challenging problem that the actual operating PID control system in complex industrial processes cannot be optimized online is solved. Using the energy-intensive equipment, the fused magnesium furnace, as an industrial object, we conducted comparative simulation experiments between the proposed control method and several advanced control methods, as well as industrial experiments for the proposed intelligent system. Simulation experiments demonstrate the effectiveness of the proposed control method. The industrial experimental results indicate that the performance monitoring and adaptive self-tuning of parameters for actual PID control systems of industrial processes in virtual reality scenes can be realized, which achieves excellent control effects.

The presence of an avatar can reduce cybersickness in Virtual Reality

Virtual Reality (VR) applications are increasingly being utilized for research, healthcare, and education. Despite their benefits, many VR users report motion sickness-like sensations (cybersickness), such as headache, disorientation, or nausea. Previous studies suggest that the sense of presence (“being there”) in the virtual world may contribute to the severity of cybersickness; however, results have been contradictory, with some studies reporting a negative and some reporting a positive relationship between the two. The goal of the current study was to further investigate how presence and cybersickness are related. Participants (N = 54) were exposed to a VR scene presented on a head-mounted display showing a 15-minute-long passive movement through space. The level of presence was manipulated by including an avatar (astronaut suit with hand-tracking) or no avatar in the virtual environment. Results showed that the avatar group reported significantly less severe cybersickness compared to the no-avatar group. We also found significant, negative correlations between some of the presence metrics (immersion, sensory fidelity) and cybersickness, indicating that cybersickness severity decreased as the level of presence increased. These findings suggest that more immersive VR experiences using an avatar may potentially reduce the risk of experiencing cybersickness.

Emotion recognition based on a limited number of multimodal physiological signals channels

Emotion constitutes a higher-order cognitive process, moreover, physiological signals can offer a more objective and realistically reflection of human emotional state. Electroencephalograph (EEG) signals, controlled by the central nervous system, are highly sensitive to emotional state fluctuations. Current research primarily focuses on recognizing emotions through feature extraction from multi-channel EEG signals. However, the acquisition of multi-channel EEG signals is hindered by difficulties acquiring from hair occlusion. The objective of this study is to construct a real-time physiological-emotion recognition model based on 2-channel EEG signals from the forehead without hair occlusion combined with 3-channel peripheral physiological (PPS) signals, employing signal preprocessing, feature extraction, feature normalization, and the ensemble learning algorithm Light Gradient Boosting Machine (LightGBM). In this paper, thirty participants sequentially viewed fourteen virtual reality (VR) scenes, rated for valence and arousal using the Self-Assessment Manikin (SAM) scale after each scene. The 2-channel EEG signals and 3-channel PPS signals were recorded synchronously from forehead by VR Head-Mounted Displays (HMD) Bio Pad developed in our research center. Our proposed model achieved median accuracy of 89.68% for valence, 90.11% for arousal in binary classification, and 84.93% for four classification of emotion recognition based on the collected samples. Furthermore, verification result of the proposed method on a public database (DEAP) achieved accuracy of 84.03%, 84.37%, and 72.23% on valence, arousal, and four classification, respectively. The proposed physiological-emotion recognition model, utilizing a limited number of multimodal physiological signals channels from wearable devices, demonstrates higher accuracy compared with the results in some literature, suggesting its potential for real-time implementation due to limited signal channels and low runtime.

Deep learning-driven optimization of real-time ray tracing for enhanced immersive virtual reality

Photorealistic rendering is essential for immersive virtual reality and real-time ray tracing is required for this, yet the necessary computational load has so far hindered its use in scenarios where a fast response and high frame rates are paramount. To overcome this challenge, we present a deep learning-based model that optimises the computational load of ray tracing through utilising convolutional neural networks (CNNs) to predict lightscene interactions. We achieve this by training CNNs from datasets with offline ray-traced images. Using this learning process, we approximate the contents of light transport simulations from scene point sampling. The use of this optimised algorithm in virtual reality scenes, thus, enables renderings of more complex scenes, with dynamic lighting, complex geometry and interactive elements under high frame rates, keeping users immersed and responsive to events. Our results show that this method achieves visual similarities to traditional ray tracing, which is photorealistic rendering, but can be performed in real-time in virtual reality. We anticipate this work leads to many potential applications of ray tracing in many scenarios in virtual reality, such as gaming, architectural rendering and training.

Alone Together, Together Alone: The Effects of Social Context on Nonverbal Behavior in Virtual Reality

Abstract Social virtual reality (VR), by definition, focuses on people using networked VR systems to bring avatars together. Previous studies have examined how different factors affect social interaction, in small groups such as dyads or triads. However, in a typical social VR scene there tends to be dozens of avatars, even those not directly interacting with a given user. Furthermore, beyond the virtual environment, VR users are also situated in various immediate physical social contexts. In two field experiments, we investigate how the presence of virtual and physical people contextualize and influence nonverbal behaviors. Study 1 examines virtual context and asks how interacting with others in a private or public virtual environment influences nonverbal outcomes during interactions in a social VR platform. Across two sessions, participants (n = 104) met either in a private virtual environment with their group members alone or in a public environment surrounded by four other groups. Results showed that participants moved their avatars slower and stood closer to group members in public versus private environments. Study 2 examines physical context and asks how interacting with virtual others while physically together or alone influences nonverbal behaviors. Participants (n = 61) met in virtual environments while they were in either a shared physical environment or separated physical environments. Results showed that, compared to remote participants, participants who were physically together moved their bodies more slowly, but their avatars faster. Moreover, there was more mutual gaze among remote participants. We discuss implications to theories of social influence in VR.

ColocZStats: a z-stack signal colocalization extension tool for 3D slicer.

Confocal microscopy has evolved to be a widely adopted imaging technique in molecular biology and is frequently utilized to achieve accurate subcellular localization of proteins. Applying colocalization analysis on image z-stacks obtained from confocal fluorescence microscopes is a dependable method of revealing the relationship between different molecules. In addition, despite the established advantages and growing adoption of 3D visualization software in various microscopy research domains, there have been few systems that can support colocalization analysis within a user-specified region of interest (ROI). In this context, several broadly employed biological image visualization platforms are meticulously explored in this study to understand the current landscape. It has been observed that while these applications can generate three-dimensional (3D) reconstructions for z-stacks, and in some cases transfer them into an immersive virtual reality (VR) scene, there is still little support for performing quantitative colocalization analysis on such images based on a user-defined ROI and thresholding levels. To address these issues, an extension called ColocZStats (pronounced Coloc-Zee-Stats) has been developed for 3D Slicer, a widely used free and open-source software package for image analysis and scientific visualization. With a custom-designed user-friendly interface, ColocZStats allows investigators to conduct intensity thresholding and ROI selection on imported 3D image stacks. It can deliver several essential colocalization metrics for structures of interest and produce reports in the form of diagrams and spreadsheets.

Optimization design of low-carbon building thermal energy based on optical sensing and virtual reality image scene reconstruction

With the intensification of global climate change and energy crisis, the design of low-carbon buildings has gradually become a research hotspot in the field of architecture. Effective thermal optimization design not only reduces energy consumption, but also improves the comfort and sustainability of the building. This study aims to explore the optimization design of low-carbon building heat energy based on light sensing technology, combined with virtual reality image scene reconstruction, to provide an innovative design method to achieve intelligent management and optimization of building heat energy. Through the introduction of light sensing technology, the light intensity and heat distribution inside and outside the building are monitored, and the combination of data analysis and model simulation is used to evaluate the heat use of the building. On this basis, virtual reality scenes are constructed to reconstruct the heat flow and distribution of buildings, providing designers with intuitive visualization tools. The research shows that the design method based on light sensing can effectively identify the areas of heat waste in the building, and reduce the energy consumption of the building by optimizing the window layout, adding greening and improving the thermal management system. Virtual reality image scene reconstruction makes the design process more intuitive, helps the design team to quickly adjust and optimize, and improves the efficiency and accuracy of the design.

FBSA-Net: A novel model based on attention mechanisms for emotion recognition in VR and 2D scenes

Recent studies have found that electroencephalographic (EEG) features from different frequency bands and different brain regions contribute differently to emotion recognition (ER) in virtual reality (VR) and two-dimensional (2D) scenes. Despite achievements in the task of ER in both VR and 2D environments, there has not been much prior effort to develop a unified ER model that can be applied to both VR and 2D environments. We propose a novel Frequency-Band-Spatial-based Attention Network, named FBSA-Net, for ER in VR and 2D scenes. Specifically, FBSA-Net adaptively captures the frequency-band-spatial relationship of EEG signals through cascade fusion of frequency-band attention (FBA) module and spatial attention (SA) module. The FBA module can automatically assign FBA weights to each band feature. The SA module then applies the ProbeParse attention mechanism to adaptively explore channel relationships both within and across brain regions, thus facilitating a comprehensive understanding of the global features of the EEG signals. The model's efficacy was validated through the VRSDEED dataset, created from emotion induction experiments using VR and 2D induction programs. Multiple comparative experiments have shown that FBSA-Net achieved state-of-the-art recognition results on VRSDEED datasets, achieving average recognition accuracies of 96.63% and 96.49% in VR and 2D scenes, respectively. The research results indicate that the significant contribution of FBSA-Net is not only reflected in the ability to provide ER using the same model in VR and 2D environments, but also in its excellent ability to distinguish between the emotional states in these two scenarios, providing a valuable reference for using the same model for ER.

The comparison between intratympanic gentamicin prehabilitation and postoperative virtual reality exposure to standard vestibular training in patients with vestibular schwannoma.

Resection of the vestibular schwannoma causes acute peripheral vestibular loss. The process of central compensation starts immediately afterward. The rehabilitation goal is to support this process and restore the quality of life. In this prospective single-center study, 67 consecutive patients underwent vestibular schwannoma resection (40 females, mean age 52 ± 12 years). The patients were divided into three groups: the prehabilitation with intratympanic gentamicin group, the virtual reality group (optokinetic stimulation via virtual reality goggles in the first ten days after the surgery), and the control group. All patients were examined with objective methods and completed questionnaires before the prehabilitation, before the surgery, at the hospital discharge, and after three months. Intratympanic gentamicin prehabilitation leads ipsilaterally to a significant aVOR reduction in all semicircular canals (p < 0.050), the increase of the unilateral weakness in air calorics (p = 0.026), and loss of cVEMPs responses (p = 0.017). Prehabilitation and postoperative exposure to virtual reality scenes improved the patient's perception of vertigo problems according to Dizziness Handicap Inventory (p = 0.039 and p = 0.076, respectively). These findings conform with the optokinetic testing results, which showed higher slow phase velocities at higher speeds (40deg/s) in both targeted groups compared to the control group. Preoperative intratympanic gentamicin positively affects peripheral vestibular function, influencing balance perception after VS resection. In long-term follow-up, prehabilitation and postoperative exposure to virtual reality improve patients' quality of life in the field of vertigo problems.

A Collaborative Neural Network-based Automatic Object Recognition Method For Environmental Space Under Virtual Reality Scenes

A Collaborative Neural Network-based Automatic Object Recognition Method For Environmental Space Under Virtual Reality Scenes

Exploring geospatial digital twins: a novel panorama-based method with enhanced representation of virtual geographic scenes in Virtual Reality (VR)

An important step in implementing geospatial digital twins is to enhance the expressiveness of virtual geographical scenes for the physical world. However, the existing virtual geographical scenes cannot quickly express the dynamically changing geographic environment for remote users due to the inefficient handling of modeling processes, user perception, and remote sharing. The research analysed the concept and characteristics of geospatial digital twins, and constructed the virtual geographical scene ontology, based on which we developed geographical spatiotemporal semantic rules and designed a dynamic annotation algorithm to enhance the representation of virtual geographical scenes. Finally, we investigated a real-time transmission method of panoramic video based on 5 G and used immersive virtual reality (IVR) to realize the user experience of remote immersion in geographical scenes. We selected a specific geographic environment containing multiple typical geographic entities to develop three prototype systems for experimental analyses. The results showed that the proposed method enabled users to view the virtual geographical scene on a VR device. The average latency for this process was 14.72 seconds. Compared with the virtual geographical scenes constructed by traditional methods, the experiments showed the proposed method advantageous in comprehensiveness, timeliness, and photorealism and abilities to enhance the user’s geographical scene perception.

Assessment and Application of Digital Museum Visitors' Emotional Experience Based on Virtual Reality Technology and Emotion Recognition Algorithm

Abstract Each person will have a different emotional experience for the same scene or exhibit, which reduces the accuracy of emotional recognition and leads to the complexity of the evaluation of visitors' emotional experience in digital museums. In order to improve the user experience of digital museums, the evaluation and optimization methods of visitors' emotional experience of digital museums based on virtual reality (VR) technology and emotion recognition algorithm are studied. A spectrogram is generated according to the voice sent by tourists when they visit the digital museum, which is based on the CSWNet_CRNN emotion recognition depth learning model input, which evaluates tourists' emotional experience results and draws their emotional responses to digital museums. The visual and auditory features of the digital museum scene with positive emotional experience are extracted, respectively. Using VR technology, the extracted features are applied to each link of the digital museum scene content design, optimizing the digital museum VR scene and improving the digital museum experience. The experiment shows that the tourist emotion recognition accuracy of this method is high, and the emotion recognition accuracy of 300 random tourists can reach 100%. In terms of generating new scenes, the feature extraction results of this scene are consistent with the feature estimation of positive emotions by ordinary people. The use of extracted features to optimize the digital museum scene has better realism and detail accuracy, which can be favored by most people and promotes the sustainable development of digital museums.

Research on the Aesthetic Orientation of Artistic Creation in Virtual Reality Scenes

With the advancement of science and technology, virtual reality technology has gradually penetrated into the field of artistic creation, providing artists with a new creative platform and aesthetic experience space. This paper explores the aesthetic orientation of artistic creation in virtual reality scenes and its influencing factors, so as to provide useful reference for researchers and practitioners in related fields. Through the discussion of immersive aesthetic experience, interactive aesthetic participation and diversified aesthetic expression, it reveals how virtual reality technology brings revolutionary changes to artistic creation. This change is not only reflected in the presentation of artworks, but also goes deeper into the audience's aesthetic psychology and the artist's creative concept. There are many factors that affect the aesthetic orientation of artistic creation in virtual reality scenes. Such as technical factors, cultural factors, and the artist's creative intentions. Typical virtual reality art creation cases are selected for analysis, which not only shows the wide application of virtual reality technology in artistic creation, but also reveals the formation process and influencing factors of different aesthetic orientations. With the continuous advancement of technology and the continuous innovation of art, artistic creation in virtual reality scenes will be richer.

Where was this thing again? Evaluating methods to indicate remembered object positions in virtual reality.

A current focus in sensorimotor research is the study of human perception and action in increasingly naturalistic tasks and visual environments. This is further enabled by the recent commercial success of virtual reality (VR) technology, which allows for highly realistic but well-controlled three-dimensional (3D) scenes. VR enables a multitude of different ways to interact with virtual objects, but only rarely are such interaction techniques evaluated and compared before being selected for a sensorimotor experiment. Here, we compare different response techniques for a memory-guided action task, in which participants indicated the position of a previously seen 3D object in a VR scene: pointing, using a virtual laser pointer of short or unlimited length, and placing, either the target object itself or a generic reference cube. Response techniques differed in availability of 3D object cues and requirement to physically move to the remembered object position by walking. Object placement was the most accurate but slowest due to repeated repositioning. When placing objects, participants tended to match the original object's orientation. In contrast, the laser pointer was fastest but least accurate, with the short pointer showing a good speed-accuracy compromise. Our findings can help researchers in selecting appropriate methods when studying naturalistic visuomotor behavior in virtual environments.

Attentional Bias, Pupillometry, and Spontaneous Blink Rate: Eye Characteristic Assessment Within a Translatable Nicotine Cue Virtual Reality Paradigm.

Incentive salience processes are important for the development and maintenance of addiction. Eye characteristics such as gaze fixation time, pupil diameter, and spontaneous eyeblink rate (EBR) are theorized to reflect incentive salience and may serve as useful biomarkers. However, conventional cue exposure paradigms have limitations that may impede accurate assessment of these markers. This study sought to evaluate the validity of these eye-tracking metrics as indicators of incentive salience within a virtual reality (VR) environment replicating real-world situations of nicotine and tobacco product (NTP) use. NTP users from the community were recruited and grouped by NTP use patterns: nondaily (n=33) and daily (n=75) use. Participants underwent the NTP cue VR paradigm and completed measures of nicotine craving, NTP use history, and VR-related assessments. Eye-gaze fixation time (attentional bias) and pupillometry in response to NTP versus control cues and EBR during the active and neutral VR scenes were recorded and analyzed using ANOVA and analysis of covariance models. Greater subjective craving, as measured by the Tobacco Craving Questionnaire-Short Form, following active versus neutral scenes was observed (F1,106=47.95; P<.001). Greater mean eye-gaze fixation time (F1,106=48.34; P<.001) and pupil diameter (F1,102=5.99; P=.02) in response to NTP versus control cues were also detected. Evidence of NTP use group effects was observed in fixation time and pupillometry analyses, as well as correlations between these metrics, NTP use history, and nicotine craving. No significant associations were observed with EBR. This study provides additional evidence for attentional bias, as measured via eye-gaze fixation time, and pupillometry as useful biomarkers of incentive salience, and partially supports theories suggesting that incentive salience diminishes as nicotine dependence severity increases.

Review of joint attention deficit intervention based on virtual reality for children with autism

Joint attention deficit is one of the core disorders in children with autism, which seriously affects the development of multiple basic skills such as language and communication. Virtual reality scene intervention has great potential in improving joint attention skills in children with autism due to its good interactivity and immersion. This article reviewed the application of virtual reality based social and nonsocial scenarios in training joint attention skills for children with autism in recent years, summarized the problems and challenges of this intervention method, and proposed a new joint paradigm for social scenario assessment and nonsocial scenario training. Finally, it looked forward to the future development and application prospects of virtual reality technology in joint attention skill training for children with autism.

Application of Graph Neural Network in Matching Intangible Cultural Heritage Models with Virtual Reality Scenes

Application of Graph Neural Network in Matching Intangible Cultural Heritage Models with Virtual Reality Scenes

Evaluation of Rural Healing Landscape DESIGN Based on Virtual Reality and Electroencephalography

From the user’s perspective, emotional elements are increasingly being used in design. Researchers have indicated that healing landscapes in rural areas play a positive role in soothing human emotions. In this study, a landscape with healing functions was designed, and 32 subjects experienced emotions in a virtual reality (VR) scene while their 32-channel electroencephalography (EEG) signals were collected. This study compared the brain responses with and without the presence of healing landscape elements and conducted correlation coefficient analysis using eight different regression prediction models to examine the relationship between security, comfort, positivity, and corresponding healing landscape elements. The results show significant improvements in emotions of security, comfort, and positivity post-exposure to the landscape design, especially with certain elements, such as seating, shrubs, and tree pools. EEG data indicate enhanced emotional and cognitive states, particularly relaxation, with increased activity in specific brain regions. The decision tree regression model is the most suitable for our data. It reveals strong correlations between specific healing landscape elements and emotional responses. In the comfort category, “shrubs” show the highest correlation (R² = 0.82), while in the security category, “trees” have the highest correlation (R² = 0.77). Similarly, in the positivity category, “trees” again exhibit the highest correlation (R² = 0.71) with EEG data, indicating their significant impact on these emotional dimensions. This study demonstrates the importance of using scientific methods, such as EEG technology, to validate the principles of emotional design and also underscores the role of green environments in enhancing psychological health and emotional comfort.

The impact of virtual reality scenes on stress response characteristics of individuals with different personality traits

Virtual reality based physical stress (VRPS) paradigms could eliminate the influence of social factors on participants, and it may be a desirable tool to explore the impact of personality traits on stress levels. In this study, we attempt to explore the effects of VRPS on stress response among individuals with different personality traits. Forty male participants with an average age of 22.79 ± 0.41 years were divided into two groups based on Harm Avoidance (HA) scores of Tridimensional Personality Questionnaire (TPQ), referred to as the Low-HA group and the High-HA group. The stress levels of the participants were assessed using salivary α-amylase (sAA) activity and heart rate variability (HRV) indices pre- and post-stress. The influence of personality traits on stress response among different groups was analyzed. VRPS significantly affected the sAA activity and HRV indicators of both groups. During and after stress, there were significant differences in sAA activity and HRV indicators between the two groups. The sAA levels and HRV indices of the Low-HA group were lower than those of the High-HA group. Furthermore, sAA levels and HRV indices were correlated with the scores of TPQ. VRPS scenarios elicit different stress responses on individuals with different harm avoidance personality traits. Stress evaluation based on VR scenarios presents potential in personality trait assessments, particularly for distinguishing between individuals with low and high HA tendencies.