Virtual Reality Scene Research Articles

Development of a Hybrid Method to Generate Gravito-Inertial Cues for Motion Platforms in Highly Immersive Environments.

Motion platforms have been widely used in Virtual Reality (VR) systems for decades to simulate motion in virtual environments, and they have several applications in emerging fields such as driving assistance systems, vehicle automation and road risk management. Currently, the development of new VR immersive systems faces unique challenges to respond to the user’s requirements, such as introducing high-resolution 360° panoramic images and videos. With this type of visual information, it is much more complicated to apply the traditional methods of generating motion cues, since it is generally not possible to calculate the necessary corresponding motion properties that are needed to feed the motion cueing algorithms. For this reason, this paper aims to present a new method for generating non-real-time gravito-inertial cues with motion platforms using a system fed both with computer-generated—simulation-based—images and video imagery. It is a hybrid method where part of the gravito-inertial cues—those with acceleration information—are generated using a classical approach through the application of physical modeling in a VR scene utilizing washout filters, and part of the gravito-inertial cues—the ones coming from recorded images and video, without acceleration information—were generated ad hoc in a semi-manual way. The resulting motion cues generated were further modified according to the contributions of different experts based on a successive approximation—Wideband Delphi-inspired—method. The subjective evaluation of the proposed method showed that the motion signals refined with this method were significantly better than the original non-refined ones in terms of user perception. The final system, developed as part of an international road safety education campaign, could be useful for developing further VR-based applications for key fields such as driving assistance, vehicle automation and road crash prevention.

Agricultural Machinery Virtual Assembly System Using Dynamic Gesture Recognitive Interaction Based on a CNN and LSTM Network

The agricultural machinery experiment is restricted by the crop production season. Missing the crop growth cycle will extend the machine development period. The use of virtual reality technology to complete preassembly and preliminary experiments can reduce the loss caused by this problem. To improve the intelligence and stability of virtual assembly, this paper proposed a more stable dynamic gesture cognition framework: the TCP/IP protocol constituted the network communication terminal, the leap motion-based vision system constituted the gesture data collection terminal, and the CNN-LSTM network constituted the dynamic gesture recognition classification terminal. The dynamic gesture recognition framework and the harvester virtual assembly platform formed a virtual assembly system to achieve gesture interaction. Through experimental analysis, the improved CNN-LSTM network had a small volume and could quickly establish a stable and accurate gesture recognition model with an average accuracy of 98.0% (±0.894). The assembly efficiency of the virtual assembly system with the framework was improved by approximately 15%. The results showed that the accuracy and stability of this model met the requirements, the corresponding assembly parts were robust in the virtual simulation environment of the whole machine, and the harvesting behaviour in the virtual reality scene was close to the real scene. The virtual assembly system under this framework provided technical support for unmanned farms and virtual experiments on agricultural machinery.

Virtual reality scene modeling in the context of Internet of Things

The Internet and the Internet of Things (IoT) connect the world inside and outside the network through online and offline communication, which forms the original single network virtual platform. The integration of IoT, virtual reality and artificial intelligence can provide intelligent WebVR interaction for various application fields. However, we need to solve the problems of data lightweight, real-time data transmission based on the IoT, and visualization of scene data on the Web. This paper uses the IoT and virtual reality technology to propose a virtual reality scene modeling method based on the IoT platform, as well as the database dynamic loading management method and device of the virtual reality system, which includes building the database, storing the involved data structures for various scenes and functions in the virtual reality system. The time–space IoT engine is connected with the integrated service sharing center. The IoT platform collects data through the IoT sensor system and provides support for the application platform through the integrated service sharing center. The proposed method can meet the needs of large-scale data acquisition and modeling, shorten the loading time, improve the efficiency, and save the wasted computing resources and memory.

A Study on Usability and Motion Sickness of Locomotion Techniques for Virtual Reality


 
 
 Virtual Reality (VR) is widely used today in both research and entertainment. The continuous growth of this technology makes VR consumer hardware now available for masses. The new trend in the next generation of VR devices is a VR headset and controllers with inside-out technology. These VR devices will become an important basis for the future of VR applications. Virtual travel or locomotion inside VR experiences is the important part in the VR application development, which affected to the users preference. The goal of this research is to study the difference of locomotion in VR with new trend devices consisting of VR headset and controllers without using other accessories. Three locomotion techniques: controller-based, motion-based and teleportation-based were used to analyze the differences. The VR scene with virtual environments was created to use in the experiment where the users have to move with different locomotion technique. The Usability Questionnaire (UQ) is used to evaluate the usability value of each locomotion technique, while Simulator Sickness Questionnaire (SSQ) is used to assess the motion sickness value. The results showed that the usability (p-value=0.02007) and motion sickness (p-value=0.00014) of all locomotion techniques are different and the usability affected to the user preference. The conclusions of VR locomotion studies were discussed with the limitations of the study and the future work for this research.
 
 

Move your virtual body: differences and similarities in brain activation patterns during hand movements in real world and virtual reality

Virtual reality (VR) is a promising tool for neurological rehabilitation, especially for motor rehabilitation. In the present study, we investigate whether brain activation patterns that are evoked by active movements are comparable when these movements are carried out in reality and in VR. Therefore, 40 healthy adults (20 men, mean age 25.31 years) performed hand movements and viewed these movements in a first-person view in reality, a VR scene showing realistic virtual hands, and a VR scene showing abstract virtual hands, in a randomized order. The VR conditions were presented via an immersive 3D head-mounted display system. EEG activity was assessed over the hand motor areas during and after movement execution. All three conditions led to typical EEG activation patterns over the motor cortex. Hence, brain activation patterns were largely comparable between conditions. However, the VR conditions, especially the abstract VR condition, led to a weaker hemispheric lateralization effect compared to the real-world condition. This indicates that hand models in VR should be realistic to be able to evoke activation patterns in the motor cortex comparable to real-world scenarios.

Relaxation Degree Analysis Using Frontal Electroencephalogram Under Virtual Reality Relaxation Scenes.

Increasing social pressure enhances the psychological burden on individuals, and the severity of depression can no longer be ignored. The characteristics of high immersion and interactivity enhance virtual reality (VR) application in psychological therapy. Many studies have verified the effectiveness of VR relaxation therapy, although a few have performed a quantitative study on relaxation state (R-state). To confirm the effectiveness of VR relaxation and quantitatively assess relaxation, this study confirmed the effectiveness of the VR sightseeing relaxation scenes using subjective emotion scale and objective electroencephalogram (EEG) data from college students. Moreover, some EEG features with significant consistent differences after they watched the VR scenes were detected including the energy ratio of the alpha wave, gamma wave, and differential asymmetry. An R-state regression model was then built using the model stacking method for optimization, of which random forest regression, AdaBoost, gradient boosting (GB), and light GB were adopted as the first level, while linear regression and support vector machine were applied at the second level. The leave-one-subject-out method for cross-validation was used to evaluate the results, where the mean accuracy of the framework achieved 81.46%. The significantly changed features and the R-state model with over 80% accuracy have laid a foundation for further research on relaxation interaction systems. Moreover, the VR relaxation therapy was applied to the clinical treatment of patients with depression and achieved preliminary good results, which might provide a possible method for non-drug treatment of patients with depression.

A direct classification approach to recognize stress levels in virtual reality therapy for patients with multiple sclerosis

AbstractMultiple sclerosis (MS) is a chronic, debilitating, and often progressive inflammatory disease of the nervous system. The disease is highly stressful, which accelerates the risk of depression in people diagnosed with it and possibly exacerbates MS activity. One approach to reducing stress levels in such patients is to utilize virtual reality (VR). Using VR technology and recording physiological signals before and after displaying different environments to the individual, this article proposed a novel therapy procedure for improving stress levels. In the first phase, by distinguishing the stress level obtained from each environment watched by the patient, their corresponding labels are determined by two psychiatrists. Accordingly, the automated model is designed based on the analysis of VR scenes and can accurately classify MS patients' stress levels after watching the 3D environment. The proposed model consists of a fractal descriptor and SVM‐RBF classifier to recognize VR scenes that can significantly reduce the stress level in MS patients. The accuracy of estimating MS patients' stress levels after watching different simulated VR environments is higher than 97%. By employing this method to classify VR scenes better and rehabilitate MS patients, it will be possible to significantly reduce their stress levels.

Design of VR application in detection line based on industrial robot for optical routers

In this study, an automated detection line for optical routers is designed. The output of the production line is evaluated and optimized on the basis of the product lifecycle management (PLM) data. The VR technology is used to build a 3D virtual reality scene to realize the layout design and planning of industry, while providing functions such as employee training, technological process roaming and interactive experience. Thus, a set of representative transformation schemes for flexible manufacturing cells can be obtained.

A chain navigation grid based on cellular automata for large-scale crowd evacuation in virtual reality

Large-scale crowd evacuation simulation in virtual reality is of great significance for emergency response. However, the existing large-scale crowd evacuation simulation methods are challenging to apply to the virtual reality scene. Thus, the efficiency of large-scale crowd rendering in virtual reality needs to be improved. Here we proposed a chain navigation grid for virtual reality large-scale crowd evacuation simulation. The objective of this study was to apply traditional cellular automata crowd evacuation simulation algorithms to virtual reality scenes through a chain navigation grid. Combined with the characteristics of crowd movement, we proposed the vertex raster rendering method to efficiently visualize large-scale crowds in virtual reality scenes. Finally, we constructed two evacuation scenes and carried out experimental analyses. The results show that the chain evacuation navigation grid can transform the discrete crowd movement of cellular automata into continuous crowd movement in virtual reality and can connect multiple cellular automata corresponding to multi-floors. Compared with the evacuation results of Pathfinder software, the chain evacuation navigation can add the factor of environmental familiarity. Therefore, this study improved evacuation simulation by increasing about 6% efficiency in time and computing process. Compared with the traditional crowd evacuation rendering method, the vertex raster rendering method increases the maximum number of people rendering from 2 times to 31 times.

Effect of a Virtual Reality-Based Restorative Environment on the Emotional and Cognitive Recovery of Individuals with Mild-to-Moderate Anxiety and Depression.

In this study, restorative environment theory and virtual reality (VR) technology were combined to build different 3D dynamic VR interactive scenes. We discuss the effects of a VR restorative environment on the emotional and cognitive recovery of individuals with mild-to-moderate anxiety and depression. First, we built a VR restorative garden scene, divided into four areas: forest, lawn, horticultural planting, and water features. The scene was verified to have a good recovery effect in 26 participants. Then, 195 participants with mild-to-moderate anxiety and depression were selected as experimental subjects. Through psychological testing and EMG (Electromyography) and EEG (Electroencephalography) data feedback, we further explored the differences in the sense of presence in VR restorative scenes and their effect on individual emotional and cognitive recovery. The results showed that (1) both the restorative environment images and the VR scenes had a healing effect (the reduction in negative emotions and the recovery of positive emotions and cognition), with no difference in the subjective feeling of recovery among the different scenes, but the recovery score of the VR urban environment was higher than that of the natural environment (differing from the results in real environments); (2) a high sense of presence can be experienced in different VR scenes, and interactive activities in VR scenes can provide a great presence experience; (3) the recovery effects of VR restorative environment on emotion and self-efficacy are realized through the presence of VR scenes; (4) a VR restorative environment is helpful for the emotional improvement and cognitive recovery of individuals with mild-to-moderate anxiety and depression. VR urban scenes also have good recovery effects. In terms of cognitive recovery, self-efficacy improved significantly. In addition, from the perspective of EEG indicators, the VR restorative scene experience activated the prefrontal lobe, which is conducive to cognitive recovery in individuals with mild-to-moderate anxiety and depression. In terms of emotional improvement, negative emotions were significantly reduced in the different VR scene groups. In conclusion, we further explored ways to help individuals with mild-to-moderate anxiety and depression, in order to promote the development and application of mental health.

A novel approach to diagnose ADHD using virtual reality

PurposeThe current golden standard for attention deficit hyperactivity disorder (ADHD) diagnosis is clinical diagnosis based on psychiatric interviews and psychological examinations. This is suboptimal, as clinicians are unable to view potential patients in multiple natural settings – a necessary condition for objective diagnosis. The purpose of this paper is to improve the objective diagnosis of ADHD by analyzing a quantified representation of the actions of potential patients in multiple natural environments.Design/methodology/approachThe authors use both virtual reality (VR) and artificial intelligence (AI) to create an objective ADHD diagnostic test. Diagnostic and statistical manual of mental disorders, 5th Edition (DSM-5) and ADHD Rating Scale are used to create a rule-based system of quantifiable VR-observable actions. As a potential patient completes tasks within multiple VR scenes, certain actions trigger an increase in the severity measure of the corresponding ADHD symptom. The resulting severity measures are input to an AI model, which classifies the potential patient as having ADHD in the form inattention, hyperactivity-impulsivity, combined or neither.FindingsThe result of this study shows that VR-observed actions can be extracted as quantified data, and classification of this quantified data achieves near-perfect sensitivity and specificity with a 98.3% accuracy rate on a convolutional neural network model.Originality/valueTo the best of the authors’ knowledge, this is the first study to incorporate VR and AI into an objective DSM-5-based ADHD diagnostic test. By including stimulation to the visual, auditory and equilibrium senses and tracking movement and recording voice, we present a method to further the research of objective ADHD diagnosis.

Multi-modal physiological signals based fear of heights analysis in virtual reality scenes

Fear of heights (FoH) analysis and its association to physiological signals can better help understand people’s emotion and quantify human’s behavior, which have been found important in many applications, such as disease analysis, affective computing, etc. Existing studies are mainly on how to alleviate FoH while little literature on FoH analysis has been reported in the past. In this paper, we present the studies of correlation on FoH to multi-modal physiological signals in the virtual reality (VR). To stimulate the FoH to participants, 4 types of VR scenarios that consist of the virtual scene of the VR game “Richie’s plank experience” and the realistic stimulus of hitting by basketball are adopted in the experiment. The synchronized eye movement (EMO), pupil, and electrocardiogram (ECG) of 17 healthy subjects with an even mix of men and women are recorded for FoH analysis. The observations on the FoH analysis carried out in the paper are two-fold: (1) The recognition ability of multi-modal physiological signals for HoF has been evaluated using ReliefF feature selection algorithm. The results show that pupil feature based model generally achieves the best classification performance for FoH. Multiple features consisted of pupil diameter, power spectral density (PSD) of EMO, pupil, ECG, and mean value of EMO can effectively quantify the variability. (2) The FoH analysis model built on the correlation analysis algorithms combined with multi-modal feature-level fusion and strategy-level fusion methods can effectively overcome the drawbacks of conventional statistical models.

The importance of peripheral vision when searching 3D real-world scenes: A gaze-contingent study in virtual reality.

Visual search in natural scenes is a complex task relying on peripheral vision to detect potential targets and central vision to verify them. The segregation of the visual fields has been particularly established by on-screen experiments. We conducted a gaze-contingent experiment in virtual reality in order to test how the perceived roles of central and peripheral visions translated to more natural settings. The use of everyday scenes in virtual reality allowed us to study visual attention by implementing a fairly ecological protocol that cannot be implemented in the real world. Central or peripheral vision was masked during visual search, with target objects selected according to scene semantic rules. Analyzing the resulting search behavior, we found that target objects that were not spatially constrained to a probable location within the scene impacted search measures negatively. Our results diverge from on-screen studies in that search performances were only slightly affected by central vision loss. In particular, a central mask did not impact verification times when the target was grammatically constrained to an anchor object. Our findings demonstrates that the role of central vision (up to 6 degrees of eccentricities) in identifying objects in natural scenes seems to be minor, while the role of peripheral preprocessing of targets in immersive real-world searches may have been underestimated by on-screen experiments.

Connecting virtual reality and ecology: a new tool to run seamless immersive experiments in R.

Virtual reality (VR) technology is an emerging tool that is supporting the connection between conservation research and public engagement with environmental issues. The use of VR in ecology consists of interviewing diverse groups of people while they are immersed within a virtual ecosystem to produce better information than more traditional surveys. However, at present, the relatively high level of expertise in specific programming languages and disjoint pathways required to run VR experiments hinder their wider application in ecology and other sciences. We present R2VR, a package for implementing and performing VR experiments in R with the aim of easing the learning curve for applied scientists including ecologists. The package provides functions for rendering VR scenes on web browsers with A-Frame that can be viewed by multiple users on smartphones, laptops, and VR headsets. It also provides instructions on how to retrieve answers from an online database in R. Three published ecological case studies are used to illustrate the R2VR workflow, and show how to run a VR experiments and collect the resulting datasets. By tapping into the popularity of R among ecologists, the R2VR package creates new opportunities to address the complex challenges associated with conservation, improve scientific knowledge, and promote new ways to share better understanding of environmental issues. The package could also be used in other fields outside of ecology.

Application of Virtual Reality and Artificial Intelligence Technology in Fitness Clubs

With the continuous development and progress of virtual reality technology in recent years, the application of virtual reality technology in all aspects of real life is no longer limited to the military field, medical, or film production fields, but it gradually appears in front of the public, into the lives of ordinary people. The human-computer interaction method in virtual reality and the presentation effect of the virtual scene are the two most important aspects of the virtual reality experience. How to provide a good human-computer interaction method for virtual reality applications and how to improve the final presentation effect of the virtual reality scene is also becoming an important research direction. This paper takes the virtual fitness club experience system as the application background, analyzes the function and performance requirements of the virtual reality experience system in the virtual reality environment, and proposes the use of Kinect as a video acquisition device to extract the user’s somatosensory operation actions through in-depth information to achieve somatosensory control. This article adopts a real human-computer interaction solution, uses Unity 3D game engine to build a virtual reality scene, defines shaders to improve the rendering effect of the scene, and uses Oculus Rift DK2 to complete an immersive 3D scene demonstration. This process greatly reduces resource consumption; it not only enables users to experience unprecedented immersion as users but also helps people create unprecedented scenes and experiences through virtual imagination. The virtual fitness club experience system probably reduces resource consumption by nearly 70%.

Application of Immersive Virtual Reality to Pragmatics Data Collection Methods

This exploratory study investigated the usability of immersive virtual reality (VR) as a way of creating a role-play task to examine pragmatic competence, specifically the ability to produce the speech act of request. The study created a closed role-play task in two versions. One was a standard computer-based version in which participants read a written scenario displayed on the screen and produced the target speech act for the computer. The other one was a VR version in which participants put on a VR headset and produced the target speech act for the interlocutor in the virtual space. Five native and five non-native speakers of English completed both versions and participated in a follow-up interview. The purpose of the interview was to examine similarities and differences in participants' perceptions of the two role-play tasks in four areas: (1) thought processes (what they were thinking during the task), (2) recall (what they remembered about the task), (3) difficulty (what made the task difficult), and (4) enjoyment (whether the task was fun). Results revealed that the participants attended to various audio-visual cues in the VR scene and used them to guide their actions. The VR version also evoked greater emotional reactions from the participants.

Ink Art Three-Dimensional Big Data Three-Dimensional Display Index Prediction Model

This paper starts with the study of realistic three-dimensional models, from the two aspects of ink art style simulation model and three-dimensional display technology, explores the three-dimensional display model of three-dimensional model ink style, and conducts experiments through the software development platform and auxiliary software. The feasibility of the model is verified. Aiming at the problem of real-time rendering of large-scale 3D scenes in the model, efficient visibility rejection method and a multiresolution fast rendering method were designed to realize the rapid construction and rendering of ink art 3D virtual reality scenes in a big data environment. A two-dimensional cellular automaton is used to simulate a brushstroke model with ink and wash style, and outlines are drawn along the path of the brushstroke to obtain an effect close to the artistic style of ink and wash painting. Set the surface of the model with ink style brushstroke texture patterns, refer to the depth map, normal map, and curvature map information of the model, and simulate the drawing effect of the method by procedural texture mapping. Example verification shows that the rapid visualization analysis model of ink art big data designed in this paper is in line with the prediction requirements of ink art big data three-dimensional display indicators. The fast visibility removal method is used to deal with large-scale three-dimensional ink art in a big data environment. High efficiency is achieved in virtual reality scenes, and the multiresolution fast rendering method better maintains the appearance of the prediction model without major deformation.

High Visual-Quality Scenes in Low-Cost Virtual Reality With Collisions and Irregular Surfaces

Mobile phones offer an excellent low-cost alternative for Virtual Reality. However, the hardware constraints of these devices restrict the displayable visual complexity of graphics. Image-based rendering techniques arise as an alternative to solve this problem, but usually, the support of collisions and irregular surfaces (i.e., any surface that is not flat or even) represents a challenge. In this work, we present a technique suitable for both virtual and real-world environments that handle collisions and irregular surfaces for an image-based rendering technique in low-cost virtual reality. We also conducted a user evaluation for finding the distance between images that presents a realistic and natural experience by maximizing the perceived virtual presence and minimizing the cybersickness effects. The results prove the benefits of our technique for both virtual and real-world environments.

Mitigating Psychological Trauma on Adult Burn Patients Based on Virtual Reality Technology of Smart Medical Treatment.

Virtual reality (VR) is one of the hot spots in the computer network world in recent years, which has attracted more and more people's attention. This study mainly explores the effect of mitigating the psychological trauma of adult burn patients based on the VR technology of smart medical treatment. First, the EEG data are sent to the data processing module through a wireless protocol; then, the data processing module denoises the EEG data and performs feature extraction and feedback parameter calculation; after that, these parameters will be sent to the VR interaction engine; based on this, these parameters change the VR scene to capture and reflect the physiological activities of the patient's brain in real time; finally, the patient uses the VR scene content presented by the real-time feedback of the captured EEG signal as a guide to making self-adjustment in time, and the electrical signal of captured brain at this time is again transmitted to the next work cycle and continues to feed back and present new VR interactive scenes to guide and intervene in the patient's self-regulation behavior. The VR feedback training module is responsible for receiving the characteristic data calculated from the EEG acquisition and processing module and converts it into parameter variables that control the VR intervention system. The system user adjusts the state according to the feedback information displayed in the VR scene and generates new EEG signals to promote the realization of self-adjustment. The biofeedback training based on EEG feeds back the intuitive EEG state to the patient, prompting them to learn how to realize self-regulation and achieve the purpose of adjusting the level of mental health. The degree of itching and pain in the VR treatment group was alleviated, and compared with the normal training treatment results, it was statistically significant (P > 0.05). This study has a positive effect on the psychological intervention of patients with facial injuries.

Smells Influence Perceived Pleasantness but Not Memorization of a Visual Virtual Environment.

The present study aimed to investigate whether the perception of still scenes in a virtual environment in congruent versus incongruent condition can be influenced by odors. Ninety healthy participants were divided into three groups, including two experimental virtual reality (VR) environments: a rose garden, an orange basket, and a control condition. In each VR condition, participants were exposed to a rose odor, an orange odor, or no odor, resulting in congruent, incongruent, and control conditions. Participants were asked to describe (a) the content of the VR scene and rate its overall pleasantness and (b) the smell and to rate its intensity and pleasantness. For each condition, participants were tested twice. During the second test, participants provided ratings and descriptions of the content of the VR scenes without being exposed to odors or VR environments. Virtual scenarios tended to be remembered as more pleasant when presented with congruent odors. Furthermore, participants used more descriptors in congruent scenarios than in incongruent scenarios. Eventually, rose odor appeared to be remembered as more pleasant when presented within congruent scenarios. These findings show that olfactory stimuli in congruent versus incongruent conditions can possibly modulate the perception of the pleasantness of visual scenes but not the memorization.