Virtual Scene Research Articles

Animation Design and Virtual Reality Dynamic Scene Rendering Optimization Based on Transformer Model

Animation Design and Virtual Reality Dynamic Scene Rendering Optimization Based on Transformer Model

The impact of virtual reality scenes on stress response characteristics of individuals with different personality traits

Virtual reality based physical stress (VRPS) paradigms could eliminate the influence of social factors on participants, and it may be a desirable tool to explore the impact of personality traits on stress levels. In this study, we attempt to explore the effects of VRPS on stress response among individuals with different personality traits. Forty male participants with an average age of 22.79 ± 0.41 years were divided into two groups based on Harm Avoidance (HA) scores of Tridimensional Personality Questionnaire (TPQ), referred to as the Low-HA group and the High-HA group. The stress levels of the participants were assessed using salivary α-amylase (sAA) activity and heart rate variability (HRV) indices pre- and post-stress. The influence of personality traits on stress response among different groups was analyzed. VRPS significantly affected the sAA activity and HRV indicators of both groups. During and after stress, there were significant differences in sAA activity and HRV indicators between the two groups. The sAA levels and HRV indices of the Low-HA group were lower than those of the High-HA group. Furthermore, sAA levels and HRV indices were correlated with the scores of TPQ. VRPS scenarios elicit different stress responses on individuals with different harm avoidance personality traits. Stress evaluation based on VR scenarios presents potential in personality trait assessments, particularly for distinguishing between individuals with low and high HA tendencies.

A Wearable Fingertip Force Feedback Device System for Object Stiffness Sensing.

Virtual reality technology brings a new experience to human-computer interaction, while wearable force feedback devices can enhance the immersion of users in interaction. This paper proposes a wearable fingertip force feedback device that uses a tendon drive mechanism, with the aim of simulating the stiffness characteristics of objects within virtual scenes. The device adjusts the rotation angle of the torsion spring through a DC motor, and then uses a wire to convert the torque into a feedback force at the user's index fingertips, with an output force of up to 4 N and a force change rate of up to 10 N/s. This paper introduces the mechanical structure and design process of the force feedback device, and conducts a mechanical analysis of the device to select the appropriate components. Physical and psychological experiments are conducted to comprehensively evaluate the device's performance in conveying object stiffness information. The results show that the device can simulate different stiffness characteristics of objects, and users can distinguish objects with different stiffness characteristics well when wearing the force feedback device and interacting with the three-dimensional virtual environments.

Design and evaluation of an AR-based thermal imaging system for planning reconstructive surgeries.

Thermal imaging can be used for the non-invasive detection of blood vessels of the skin. However, mapping the results to the patient currently lacks user-friendliness. Augmented reality may provide a useful tool to superimpose thermal information on the patient. A system to support planning in reconstructive surgery using a thermal camera was designed. The obtained information was superimposed on the physical object using a Microsoft HoloLens. An RGB, depth, and thermal camera were combined to capture a scene of different modalities and reconstruct a virtual scene in real time. To register the different cameras and the AR device, an active calibration target was developed and evaluated. A Vuforia marker was used to register the hologram in the virtual space. The accuracy of the projected hologram was evaluated in a laboratory setting with participants by measuring the error between the physical object and the hologram. The AR-based system was evaluated by 21 participants in a laboratory setting. The mean projection error is 10.3 ± 9.4mm. The system is able to stream a three-dimensional scene with augmented thermal information in real time at 5 frames per second. The active calibration target can be used independently of the environment. The calibration target provides an easy-to-use method for the registration of cameras capturing the visible to long-infrared spectral range. The inside-out tracking of the HoloLens in combination with a Vuforia marker is not accurate enough for the intended clinical use.

Virtual geographical scene twin modeling: a combined data-driven and knowledge-driven method with bridge construction as a case study

Virtual geographical scene twin modeling: a combined data-driven and knowledge-driven method with bridge construction as a case study

Development of Digital Show Methodology

The article proposes a method for creating a digital display of a clothing collection using modern information technologies. The authors analyzed technologies that can be used to display the collection in a digital environment. Particular attention is paid to 3D modeling and animation, as well as the integration of virtual models and scenes into videos. An algorithm is proposed that includes the stages of creating the image of the collection and selecting avatars, animation of the process of passing models in designed clothes across the stage in three-dimensional space. The purpose of this work is to consider the features of using neural networks for generating images, online resources and three graphic editors: a general-purpose 3D editor Blender, a 3D editor for designing and visualizing clothing Clo3D and a program for color correction, editing and video processing DaVinci Resolve. The main task was to study the possibilities of these technological solutions for creating motion design for a virtual fashion show of a clothing collection. Development algorithms are described at different stages: from searching for an artistic image using neural networks and digital design to developing animation design and video post-processing. The research materials contribute to understanding the possibilities of digital design in the field of fashion. The described tools allow you to create realistic clothing models, demonstrate animation and fabric simulation, and present collections in a dynamic format. In conclusion, recommendations are offered for the use of computer graphics for the effective development of virtual screenings. The article is intended for specialists in the field of costume and fashion design, computer graphics, virtual reality, as well as for anyone interested in innovative approaches to the organization and presentation of digital products.

Design of Lumbar Rehabilitation Training System Based on Virtual Reality

A virtual reality-based lumbar rehabilitation training system is designed to address the increasing number of patients with low back pain (LBP) year by year. Attitude sensors are used to track lower back movement. In order to improve the effect of rehabilitation training, several virtual rehabilitation training games and assessment scenes are designed based on the Unity3D engine to complete different tasks from simple to complex. The goal is to increase patients’ interest in rehabilitation training. The experimental results verify the accuracy of rehabilitation data acquisition, real-time interactive communication, and the smooth operation of rehabilitation scenes.

EEG classification model for virtual reality motion sickness based on multi-scale CNN feature correlation

BackgroundVirtual reality motion sickness (VRMS) is a key issue hindering the development of virtual reality technology, and accurate detection of its occurrence is the first prerequisite for solving the issue. ObjectiveIn this paper, a convolutional neural network (CNN) EEG detection model based on multi-scale feature correlation is proposed for detecting VRMS. MethodsThe model uses multi-scale 1D convolutional layers to extract multi-scale temporal features from the multi-lead EEG data, and then calculates the feature correlations of the extracted multi-scale features among all the leads to form the feature adjacent matrixes, which converts the time-domain features to correlation-based brain network features, thus strengthen the feature representation. Finally, the correlation features of each layer are fused. The fused features are then fed into the channel attention module to filter the channels and classify them using a fully connected network. Finally, we recruit subjects to experience 6 different modes of virtual roller coaster scenes, and collect resting EEG data before and after the task to verify the model. Results: The results show that the accuracy, precision, recall and F1-score of this model for the recognition of VRMS are 98.66 %, 98.65 %, 98.68 %, and 98.66 %, respectively. The proposed model outperforms the current classic and advanced EEG recognition models. SignificanceIt shows that this model can be used for the recognition of VRMS based on the resting state EEG.

The Impact of WeChat Platform on Social Interaction Under the Perspective of Situational Theory

The new media environment will promote new interactions and relationships. This paper attempts to use the situation theory of media to explore the impact of the WeChat platform on social behavior and analyze the new communication scenes created by the WeChat platform, and the new communication scenes have changed the interaction behavior and communication relationship. These issues have become an important exploration of the current in-depth understanding of online social media. This essay combines and summarizes the new media situations that WeChat has constructed and discuss the influences of these situations on social interactions according to situation theory. This research also explores how WeChat scenario affects its clients’ communication behaviors and relations through the methodology of depth interview, questionnaires and participatory-observation. Through the combing of the media scenes and their influences on WeChat and expand the analysis of the two virtual scenes of group chat and circle of friends. The user realizes the common presence of the virtual body through the WeChat group chat scene.. In the circle of friends, the users have the autonomy to control the screen, and can switch the front and back areas in the scene of the circle of friends at will. WeChat makes the online and offline scenes continue to merge through the expansion of the offline acquaintance circle and the social interaction of strangers on the line at the same time. Using in-depth interviews and questionnaire surveys, the author finds that the interaction scene of WeChat construction has two effects on social interaction behavior. On one hand, some new methods of communications have emerged. The virtual scenario socialization creates the interaction between human and machine, and the open scene of WeChat makes the user's personality and ability become their shining point which can makes them attractive. The user uses "likes" as a way of communication in the situations. On the other hand, the expression package came into being in order to provide more scene information. Situations changes in the front, middle, and back areas affect the user's expression package usage behavior.

Generating common spaces through virtual reality telepresence and shared scene synthesis

This study aims to create a shared virtual space for remote telepresence, expanding the range of activities beyond traditional 2D screen-based communication. The challenge lies in seamlessly connecting the virtual and physical environments, enabling free movement and interaction. To address this, in the proposed approach advanced techniques in scene understanding, spatial mapping, and virtual environment generation are employed. Primary data is collected from each participant's space, analyzing crucial information like object placement, room layouts, and interactive elements. This forms the basis for generating a virtual scene that aligns with these features, creating a unified environment. The proposed approach ensures compatibility and meets participants' needs by utilizing a shared function optimization module. Additionally, it enhances the scene further through the use of deep-learning and conditional techniques. The created scene optimally supports shared functionalities for walking, sitting, and working, all of which mirror the physical objects present in the users' actual surroundings. Experiments using the MatterPort3D dataset evaluate the proposed approach, alongside a comparative user study. Results demonstrate the potential of the proposed approach in overcoming challenges of remote telepresence, enabling immersive and interactive experiences in the virtual space. In conclusion, this research tackles the problem of creating a shared virtual space for remote telepresence. Analyzing input data and employing advanced techniques generates a coherent virtual scene.

Artistic Virtual Reality Scene Analysis Taking into Account the Interactive Digital Media Pattern Generation Technology

With the development of science and technology and the continuous strength of Chinese spiritual civilization, digital media patterns have penetrated into various fields and played a central role in them. This article studies the endless emergence of various new media, and the channels of information dissemination are becoming more and more abundant. With the development of science and technology, new media are gradually derived, such as: IPTV, electronic magazines, etc., which have developed on the basis of traditional media. But there is a qualitative difference from traditional media. Digital media has a profound impact on mass society.This article proposes that digital media has become the most important means of communication in the information society. The digital media creative industry and the country's economic development have formed a close relationship of interdependence and mutual promotion. Based on the interactive digital media model generation technology, the research of artistic virtual and real scene analysis is the main issue today. The experimental results of this paper show that through the experimental investigation and analysis in 2019 and 2020, it is found that the application trend of company A and B in the virtual reality situation has increased significantly. The application rate of company A and B in 2019 reached about 43%, but the trend is not unstable. However, in 2020, it can be clearly seen that the application rate of companies A and B in applying digital media patterns to virtual and real scenarios has stabilized, exceeding 35% every month. It can be seen in the text below that when the four models are in different periods, the value of the optimal solution is different, and when the model is at 0.1 and 31, the optimal solution appears.Experiments show that digital media is widely used in the analysis of virtual and real scenarios. Based on the new research field formed by the fusion and intersection of digital media patterns, starting with the changes in the main body, objects, tools and forms of virtual and real scene communication in the context of digital media, it conducts in-depth research on the creation of artistic virtual and real scenes from many aspects.

SADNet: Generating immersive virtual reality avatars by real‐time monocular pose estimation

SummaryGenerating immersive virtual reality avatars is a challenging task in VR/AR applications, which maps physical human body poses to avatars in virtual scenes for an immersive user experience. However, most existing work is time‐consuming and limited by datasets, which does not satisfy immersive and real‐time requirements of VR systems. In this paper, we aim to generate 3D real‐time virtual reality avatars based on a monocular camera to solve these problems. Specifically, we first design a self‐attention distillation network (SADNet) for effective human pose estimation, which is guided by a pre‐trained teacher. Secondly, we propose a lightweight pose mapping method for human avatars that utilizes the camera model to map 2D poses to 3D avatar keypoints, generating real‐time human avatars with pose consistency. Finally, we integrate our framework into a VR system, displaying generated 3D pose‐driven avatars on Helmet‐Mounted Display devices for an immersive user experience. We evaluate SADNet on two publicly available datasets. Experimental results show that SADNet achieves a state‐of‐the‐art trade‐off between speed and accuracy. In addition, we conducted a user experience study on the performance and immersion of virtual reality avatars. Results show that pose‐driven 3D human avatars generated by our method are smooth and attractive.

Research on the design and application of 3D scene animation game entertainment system based on user motion sensing participation

Developed a 3D indoor virtual scene assembly system based on dual touch screen. The system adopts a combination of 2D views and 3D views. The 2D and 3D views are displayed on the two touch screens. We have comprehensively designed various interactive operations of 2D view and 3D view from the perspectives of simplicity, easy learning and easy to use, and reduced the complex 3D interactive operations to 2D interactive operations. Users can conduct two-dimensional interactive operation and 3D interactive operation based on two display screens, which can realize the efficient assembly of 3D virtual scene. The results indicate significant differences in the number of modifications of mode A and B (t = 2.982, p = 0.006 < 0.01), effective design time (t = 4.990, p < 0.001), first design satisfaction (t = 5.074, p = <0.001) and final design satisfaction (t = 2.787, p = 0.010 < 0.05). Participants in mode A had fewer modifications and shorter effective design times compared to mode B. Study and design a new workbench supporting the collaborative design of 3D virtual scenes. The workbench can support a three-dimensional indoor virtual scene assembly system based on dual touch screen, and designers and users can assemble the virtual scene together. The workbench mainly consists of two touch displays and a holographic display device. The holographic display device displays the three-dimensional scene designed by the user in real time, and supports the designer and the user to operate the three-dimensional scene through the touch film attached to it. Touch display enables designers and users to assemble scenes on 2D views through simple touch operations such as drag and drop. The face-to-face collaborative design method provided by the workbench is more conducive to the communication between designers and users, so as to improve the design efficiency of 3D scenes.

AI-enabled Rendering Technology for the Construction and Development of Dunhuang Virtual Pavilion

This paper focuses on the use of AI technology to assist in the construction of digital scenes in Dunhuang Research Institute, which realizes the roaming of tourists in virtual scenes through image recognition and semantic analysis of massive digital image data, and explores the possibility of AI-assisted rendering of virtual museums. Using AI technology, tourists can observe the uniqueness of each cave within the cavern and its rich content from any angle, and obtain an immersive travel experience. The study fully utilizes the new generation of information technology to build a virtual scene platform of Dunhuang Mogao Grottoes, which comprehensively presents the history, cultural value and artistic value of Dunhuang Mogao Grottoes through the comprehensive integration and display of a variety of digital resources. Focusing on digital media and virtual reality technology, through the use of artificial intelligence technology and the comprehensive integration of related digital resources, to build a complete set of virtual reality ecosystem, and explore the realization of multi-source data fusion of Dunhuang cultural communication.

Investigating cortical activity during cybersickness by fNIRS

This study investigated brain responses during cybersickness in healthy adults using functional near-infrared spectroscopy (fNIRS). Thirty participants wore a head-mounted display and observed a virtual roller coaster scene that induced cybersickness. Cortical activation during the virtual roller coaster task was measured using fNIRS. Cybersickness symptoms were evaluated using a Simulator Sickness Questionnaire (SSQ) administered after the virtual rollercoaster. Pearson correlations were performed for cybersickness symptoms and the beta coefficients of hemodynamic responses. The group analysis of oxyhemoglobin (HbO) and total hemoglobin (HbT) levels revealed deactivation in the bilateral angular gyrus during cybersickness. In the Pearson correlation analyses, the HbO and HbT beta coefficients in the bilateral angular gyrus had a significant positive correlation with the total SSQ and disorientation. These results indicated that the angular gyrus was associated with cybersickness. These findings suggest that the hemodynamic response in the angular gyrus could be a biomarker for evaluating cybersickness symptoms.

Visual behavior characteristics of historical landscapes based on eye-tracking technology

ABSTRACT The subjective encounter with architecture encompasses the particular presentation of architectural elements or the overall structure to individuals, taking into account their perception, cognition, and thought processes. The purpose of this research is to investigate the intricate relationship between visual attention directed towards historical architecture and the subjective experiences engendered during the process of observation. Utilizing eye-tracking technology within the realm of virtual reality, this study delves into the observation patterns exhibited by individuals when confronted with historical architecture, specifically focusing on the traditional courtyard residences found in Beijing’s hutongs. The panoramic images, meticulously modeled and rendered, are divided into six distinct areas of interest: base, ground, window&doors, walls, roof, and eaves. The eye-tracking data of 81 participants, who engaged with 10 virtual scenes through the employment of VR headsets, along with their responses to architectural style questionnaires, were systematically gathered. Through comprehensive analysis, encompassing the examination of total fixation duration (TFD), fixation count (FC), first fixation duration (FFD), and time to first fixation (TFF) across the areas of interest (AOIs), in addition to deviations in scores among different architectural areas and styles, notable insights emerged. Results indicate that both professionals and non-professionals allocate heightened attention, as evidenced by TFD, FC, TFF, and FFD, to elements that exhibit greater score deviations in the questionnaires as opposed to those with smaller discrepancies. Moreover, the TFD and FC metrics pertaining to the Windows&doors AOI collectively constitute 40.20% and 40.71%, respectively. Undoubtedly, these figures signify the preeminent focal points within all AOIs. This underscores the pivotal role played by doors and windows in shaping individuals’ judgments pertaining to historical architectural styles. This research provides designers with valuable understanding of the cognitive patterns exhibited by individuals in engaging with aspects of history, which enable them to identify and preserve key elements associated with historical styles. Additionally, it establishes a fundamental cornerstone for future quantitative research endeavors centered around the preservation of historical architectural styles.

Varifocal occlusion in an optical see-through near-eye display with a single phase-only liquid crystal on silicon

We propose a near-eye display optics system that supports three-dimensional mutual occlusion. By exploiting the polarization-control properties of a phase-only liquid crystal on silicon (LCoS), we achieve real see-through scene masking as well as virtual digital scene imaging using a single LCoS. Dynamic depth control of the real scene mask and virtual digital image is also achieved by using a focus tunable lens (FTL) pair of opposite curvatures. The proposed configuration using a single LCoS and opposite curvature FTL pair enables the self-alignment of the mask and image at an arbitrary depth without distorting the see-through view of the real scene. We verified the feasibility of the proposed optics using two optical benchtop setups: one with two off-the-shelf FTLs for continuous depth control, and the other with a single Pancharatnam–Berry phase-type FTL for the improved form factor.

Generating trajectory data without behavior modeling: An online virtual reality method in wayfinding performance evaluation for buildings

Wayfinding performance is one of the key issues in pre- or post-occupancy evaluation for large-scale public buildings and such evaluation typically revolves around participants’ wayfinding trajectories. Traditional evaluation methods, including laboratory virtual reality (VR) experiments and computer simulations, have their respective advantages in accuracy or cost-efficiency. However, they either suffer from high costs and low efficiency or sacrifice the diversity of behavioral data for higher efficiency and lower costs. Therefore, this paper proposes a novel method named “sampling as simulation” (SaS) via an online VR wayfinding experiment. This method allows for the rapid collection of behavioral data at a lower cost to generate highly accurate wayfinding trajectory data for evaluation purposes. It utilizes a model-image integrated approach to create realistic online virtual environments and employs a spatiotemporal replay algorithm to generate a crowd of true-trajectory-driven avatars, significantly enhancing the immersion and realism of the online virtual scenes. This method was demonstrated and validated with a wayfinding performance post-occupancy evaluation (POE) project at the International Departure of Satellite Terminal One (S1) in Shanghai Pudong International Airport (PVG). In this example, a total of 13,712 original trajectories were collected within a 72-h time window, serving as the original dataset for evaluation. Based on this dataset, the study effectively simulated wayfinding behavior in three different scenarios and evaluated wayfinding performance. Compared to existing methods, the SaS method not only improves data collection efficiency and reduces costs but also maintains the diversity of behavior by generating transformed trajectories directly from human data without behavior modeling.

Diversity of 3D APAR and LAI dynamics in broadleaf and coniferous forests: Implications for the interpretation of remote sensing-based products

Forests substantially mediate the water and carbon dioxide exchanges between terrestrial ecosystems and the atmosphere. The rate of this exchange, including evapotranspiration (ET) and gross primary production (GPP), depends mainly on the underlying vegetation type, health state, and the composition of abiotic environmental drivers. However, the complex 3D structure of forest canopies and the inherent top-view perspective of optical and thermal remote sensing complicate remote sensing-based retrievals of biotic and abiotic factors that eventually determine ET and GPP. This study investigates the sensitivity of remote sensing approaches to 3D variation of abiotic and biotic environmental drivers. We use 3D virtual scenes of two structurally different Swiss forests and the radiative transfer model DART to simulate the 3D distribution of solar irradiance and reflected radiance in the forest canopy. These simulations, in combination with LiDAR data, are used to derive the absorbed photosynthetic active radiation (APAR) and the leaf area index (LAI) in 3D space. The 3D variation of both parameters was quantified and analyzed. We then simulated images of the top-of-canopy bi-directional reflectance factor (BRF) and compared them with the hemispheric-conical reflectance factor (HCRF) data derived from HyPlant airborne imaging spectrometer measurements. The simulated BRF data was used to derive APAR and LAI, and the results were compared to their respective 3D representations. We unravel considerable spatial differences between both representations. We discuss possible reasons for the disagreement, including a potential insensitivity of the inherent top-of-canopy view for the real 3D product dynamics and limitations of the processing of remote sensing data, especially the approximation of effective surface irradiance. Our results can help understanding sources of uncertainties in remote sensing based gas exchange products and defining mitigation strategies.

A Pre-convolved Representation for Plug-and-Play Neural Illumination Fields

Recent advances in implicit neural representation have demonstrated the ability to recover detailed geometry and material from multi-view images. However, the use of simplified lighting models such as environment maps to represent non-distant illumination, or using a network to fit indirect light modeling without a solid basis, can lead to an undesirable decomposition between lighting and material. To address this, we propose a fully differentiable framework named Neural Illumination Fields (NeIF) that uses radiance fields as a lighting model to handle complex lighting in a physically based way. Together with integral lobe encoding for roughness-adaptive specular lobe and leveraging the pre-convolved background for accurate decomposition, the proposed method represents a significant step towards integrating physically based rendering into the NeRF representation. The experiments demonstrate the superior performance of novel-view rendering compared to previous works, and the capability to re-render objects under arbitrary NeRF-style environments opens up exciting possibilities for bridging the gap between virtual and real-world scenes.