Immersive Virtual Reality Environment Research Articles

Nanotilus: Generator of Immersive Guided-Tours in Crowded 3D Environments.

Immersive virtual reality environments are gaining popularity for studying and exploring crowded three-dimensional structures. When reaching very high structural densities, the natural depiction of the scene produces impenetrable clutter and requires visibility and occlusion management strategies for exploration and orientation. Strategies developed to address the crowdedness in desktop applications, however, inhibit the feeling of immersion. They result in nonimmersive, desktop-style outside-in viewing in virtual reality. This article proposes Nanotilus-a new visibility and guidance approach for very dense environments that generates an endoscopic inside-out experience instead of outside-in viewing, preserving the immersive aspect of virtual reality. The approach consists of two novel, tightly coupled mechanisms that control scene sparsification simultaneously with camera path planning. The sparsification strategy is localized around the camera and is realized as a multi-scale, multi-shell, variety-preserving technique. When Nanotilus dives into the structures to capture internal details residing on multiple scales, it guides the camera using depth-based path planning. In addition to sparsification and path planning, we complete the tour generation with an animation controller, textual annotation, and text-to-visualization conversion. We demonstrate the generated guided tours on mesoscopic biological models - SARS-CoV-2 and HIV. We evaluate the Nanotilus experience with a baseline outside-in sparsification and navigational technique in a formal user study with 29 participants. While users can maintain a better overview using the outside-in sparsification, the study confirms our hypothesis that Nanotilus leads to stronger engagement and immersion.

Designing Immersive Virtual Reality Environments for Supporting Patients at Home: Translating Input From Home Care Nurse Experts to Design Requirements.

Immersive virtual reality computer programs provide new experimental and treatment interventions that hold great promise for nursing. Immersive virtual reality uses sensory cues to represent real-world environments in a way that makes participants feel as if they are in a physical space different from the one in which they currently exist. As the acceptance of immersive virtual reality as a clinical and experimental tool has grown, so has the need to ensure that the context depicted in the environment mirrors both the sensory and the task requirements of the real-world situation. Here, we describe the use of nurse expert key informant group interviews to generate requirements that must be fulfilled in immersive virtual reality environments designed to evoke and engage participants in self-management tasks. An expert panel of four home care nurses participated in three sessions designed to elicit details of common home care challenges, frequency of variation, and typical participants. More than 20 potential scenarios were identified. The design team later used this information to create design requirements for two key scenarios and subsequently develop immersive virtual reality environments for use in research studies, mapping sensory and functional expectations to immersive virtual reality implementations. Challenges in mapping from key informant group findings to requirements are addressed.

Designing Immersive Virtual Reality Environments for Supporting Patients at Home: Translating Input From Home Care Nurse Experts to Design Requirements.

CIN: Computers, Informatics, Nursing 41(3):p 183-184, March 2023. | DOI: 10.1097/01.NCN.0000904400.19657.5d

Examining the relationships between students’ perceptions of technology, pedagogy, and cognition: the case of immersive virtual reality mini games to foster computational thinking in higher education

Researchers are increasingly exploring educational games in immersive virtual reality (IVR) environments to facilitate students’ learning experiences. Mainly, the effect of IVR on learning outcomes has been the focus. However, far too little attention has been paid to the influence of game elements and IVR features on learners’ perceived cognition. This study examined the relationship between game elements (challenge, goal clarity, and feedback) as pedagogical approach, features of IVR technology (immersion and interaction), and learners’ perceived cognition (reflective thinking and comprehension). An experiment was conducted with 49 undergraduate students who played an IVR game-based application (iThinkSmart) containing mini games developed to facilitate learners’ computational thinking competency. The study employed partial least squares structural equation modelling to investigate the effect of educational game elements and learning contents on learner’s cognition. Findings show that goal clarity is the main predictor of learners’ reflective thinking and comprehension in an educational game-based IVR application. It was also confirmed that immersion and interaction experience impact learner’s comprehension. Notably, adequate learning content in terms of the organisation and relevance of the content contained in an IVR game-based application significantly moderate learners’ reflective thinking and comprehension. The findings of this study have implications for educators and developers of IVR game-based intervention to facilitate learning in the higher education context. In particular, the implication of this study touches on the aspect of learners’ cognitive factors that aim to produce 21st-century problem-solving skills through critical thinking.

Prior knowledge as a moderator between signaling and learning performance in immersive virtual reality laboratories.

The purpose of this study is to investigate the effects of signaling and prior knowledge on the cognitive loads, motivations, and learning of college students in an immersive virtual reality (IVR) environment. This study applied a 2 (signaling vs. no signaling) by 2 (high vs. low prior knowledge levels) between-subjects factorial design. The results revealed that signaling directed the attention of students with low prior knowledge levels, effectively helped them select relevant information and reduced their cognitive loads, whereas signaling had no significant effect on the cognitive loads, intrinsic motivation, and learning performance of learners with high levels of prior knowledge. These results suggest that IVR environments for students with low prior knowledge levels should reduce cognitive load and improve learning, and signals in the form of text annotations and color changes are recommended for additional support. Students with high prior knowledge levels do not require additional signals to support learning; therefore, the IVR environment needs to be designed in such a way as to be tailored to the individual differences of students.

Immersive role-playing with avatars leads to adoption of others’ personalities

In modern society, where nations and even individuals are divided, building bridges between people of different personalities and backgrounds is essential to achieve harmonious coexistence. In recent years, research has been conducted on the use of technologies to bridge this gap. In this study, the effectiveness of using immersive virtual reality (IVR) technology to play the role of a stranger in helping people learn about and empathize with others was investigated. Specifically, participants were asked to role-play a first-time stranger in an IVR environment after being given prior information about the stranger via a preparatory video. The effects of role-playing between acting as a target stranger through his or her avatar in an immersive environment, acting through a different avatar in an immersive environment, and acting through his or her avatars in a non-immersive interface were compared. The results showed that using IVR technology with an exact avatar to play the target person was found to have the greatest effect on the participants’ personalities and thoughts as well as increasing their empathy for that individual. This finding suggests that IVR technology may be an effective tool for bridging the gap between people from different backgrounds.

Auralisation of combined mitigation measures in railway pass-by noise

To reduce noise exposure along railway lines various combinations of noise mitigation measures can be considered. However, predicting and assessing their effects is non-trivial and the potential need for multiple measures is difficult to communicate to stakeholders. Auralisation is a promising tool that can help to support communication and decision-making, and enable psychoacoustic evaluations. This paper presents developments of a physics-based auralisation model for train pass-bys that allows various mitigation measures to be included. The work is conducted within the European research project SILVARSTAR. The proposed model includes contribution from rolling noise, impact noise, traction, auxiliary systems, and aerodynamic noise. It is physically based and allows a direct assessment of pass-by parameters such as speed, roughness, wheel flats and track design. Based on the TWINS model, five structural transfer paths for rolling noise are considered to integrate mitigation measures such as wheel and rail dampers. Shielding by noise barriers is simulated with analytical models. Reflection at different ground types is considered and can account for track embankments. The results can be coupled to an immersive Virtual Reality environment, by first panning the synthesised sounds to a small virtual speaker array and subsequently dynamic binaural rendering for headphones.

Cognitive and motivational benefits of a theory-based immersive virtual reality design in science learning

This study investigated the effects of an immersive virtual reality (IVR) nature-trail tour on participants’ science learning, self-efficacy, cognitive load, perceived enjoyment, and perceived usefulness, as compared to actual walking tours. The IVR tour was designed based on the Cognitive Theory of Multimedia Learning. In a between-subjects quasi-experiment, participants learned environmental science topics in one of three types of nature-trail tours, including an IVR tour, a business-as-usual walking tour, and an enhanced walking tour. Results of analyses of covariance indicated that the theory-based IVR design was effective in improving participants’ science learning and their self-efficacy perceptions. At the same time, the IVR tour was found to be as enjoyable as the walking tours and did not pose an unnecessary cognitive load during the learning process. The results have implications for designing IVR environments to (1) enhance cognitive and motivational outcomes in science learning and (2) increase the accessibility of nature-based sites.

Automatic Detection of Cognitive Impairment with Virtual Reality.

Cognitive impairment features in neuropsychiatric conditions and when undiagnosed can have a severe impact on the affected individual's safety and ability to perform daily tasks. Virtual Reality (VR) systems are increasingly being explored for the recognition, diagnosis and treatment of cognitive impairment. In this paper, we describe novel VR-derived measures of cognitive performance and show their correspondence with clinically-validated cognitive performance measures. We use an immersive VR environment called VStore where participants complete a simulated supermarket shopping task. People with psychosis (k=26) and non-patient controls (k=128) participated in the study, spanning ages 20-79 years. The individuals were split into two cohorts, a homogeneous non-patient cohort (k=99 non-patient participants) and a heterogeneous cohort (k=26 patients, k=29 non-patient participants). Participants' spatio-temporal behaviour in VStore is used to extract four features, namely, route optimality score, proportional distance score, execution error score, and hesitation score using the Traveling Salesman Problem and explore-exploit decision mathematics. These extracted features are mapped to seven validated cognitive performance scores, via linear regression models. The most statistically important feature is found to be the hesitation score. When combined with the remaining extracted features, the multiple linear regression model resulted in statistically significant results with R2 = 0.369, F-Stat = 7.158, p(F-Stat) = 0.000128.

VR for Pain Relief.

The present chapter explores how immersive virtual reality (VR) systems can be used for pain research and treatment. Pain is a universal, yet entirely subjective and multifaceted unpleasant experience. One of the earliest VR studies on pain highlighted the role of attention in pain modulation. However, the role of body representation in pain modulation has also been described as a crucial factor. Through virtual reality systems, it is possible to modulate both attention to pain and body representation. In this chapter, first we define how immersive VR can be used to create the illusion of being present in immersive VR environments and argue why VR can be an effective tool for distracting patients from acute pain. However, distraction seems to be less useful in chronic pain treatment. Chronic pain can be highly disabling and can significantly impact not only the sufferer's quality of life, but also their perceptions of the bodily self. Close neural connections between the body matrix and pain open a chance for influencing pain through bodily illusions. This chapter explores approaches to inducing body ownership illusions in VR and discusses how they have been applied in pain research. The present chapter also covers a set of practical indications and methodological caveats of immersive VR and solutions for overcoming them. Finally, we outline several promising future research directions and highlight several yet unexplored areas.

An assembly process simulation method in immersive virtual reality environment

An assembly process simulation method in immersive virtual reality environment

Novel Approach to Transfer of Coordination Skills to the Paretic Hand by Improving the Efficiency of Cortical Network.

We have previously shown that healthy subjects can transfer coordination skills to the unpracticed hand by performing a unimanual task with the other hand and visualizing a bimanual action using a game-like interactive system. However, whether this system could be used to transfer coordination skills to the paretic hand after stroke and its underlying neural mechanism remain unknown. Here, using a game-like interactive system for visualization during physical practice in an immersive virtual reality environment, we examined coordination skill improvement in the unpracticed/paretic hand after training in 10 healthy subjects and 13 chronic and sub-acute stroke patients. The bimanual movement task was defined as simultaneously drawing non-symmetric three-sided squares (e.g., U and C), while the training strategy was performing a unimanual task with the right/nonparetic hand and visualizing a bimanual action. We found large decreases in the intra-hand temporal and spatial measures for movement in the unpracticed/paretic hand after training. Furthermore, a substantial reduction in the inter-hand temporal and spatial interference was observed after training. Additionally, we examined the related cortical network evolution using EEG in both the healthy subjects and stroke patients. Our studies show that the cortical network became more efficient after training in the healthy subjects and stroke patients. These results demonstrate that our proposed method could contribute to the transference of coordination skill to the paretic/unpracticed hand by promoting the efficiency of cortical networks.

Digital Twin and Metaverse Technologies, Cognitive Mapping and Navigation Tools, and Image Processing Computational and Object Tracking Algorithms in Immersive 3D Virtual Reality Environments

Digital Twin and Metaverse Technologies, Cognitive Mapping and Navigation Tools, and Image Processing Computational and Object Tracking Algorithms in Immersive 3D Virtual Reality Environments

Design and Development of Eye Tracking System for Children with Autism

Autism Spectrum Disorder (ASD) is a neurodevelopmental condition impacting communication and social interaction in children. Eye-tracking technology has emerged as a valuable tool to investigate visual attention patterns in children with ASD. This paper presents the design and development of an eye-tracking system for children with ASD, employing a Raspberry Pi 4, Noir camera, gaze tracker, and LED display to create an immersive virtual reality (VR) environment for precise visual attention measurement. The hardware implementation involves integrating the Raspberry Pi circuitry and AMOLED screen into a VR box, with the Noir camera capturing eye images. A meticulous calibration process maps pupil coordinates to screen coordinates accurately. The system's software, employing OpenCV for image processing and Firebase for data storage, is rigorously tested on adults to ensure reliable gaze tracking prior to assessing children with autism. Successful tracking of gaze coordinates during stimulus display indicates its potential for accurate data collection. However, minor reading errors from the sensor slightly hinder accuracy, suggesting improvements via camera sensor and microcontroller upgrades. The eye-tracking system offers a promising avenue for early and accurate ASD diagnosis, offering critical insights into visual attention patterns, and facilitating effective interventions to enhance social and communication skills in children with ASD.

Effectiveness of an Immersive Virtual Reality Environment on Curricular Training for Complex Cognitive Skills in Liver Surgery: A Multicentric Crossover Randomized Trial

Effectiveness of an Immersive Virtual Reality Environment on Curricular Training for Complex Cognitive Skills in Liver Surgery: A Multicentric Crossover Randomized Trial

Visual Feedback and Guided Balance Training in the Immersive Virtual Reality Environment for Lower Extremity Rehabilitation

Visual Feedback and Guided Balance Training in the Immersive Virtual Reality Environment for Lower Extremity Rehabilitation

Determinants of Learning Anatomy in an Immersive Virtual Reality Environment - A Scoping Review.

The online version contains supplementary material available at 10.1007/s40670-022-01701-y.

The effects of visual cues in an immersive virtual reality environment on adaptation to internal and external errors

The effects of visual cues in an immersive virtual reality environment on adaptation to internal and external errors

Immersive virtual reality for upper limb rehabilitation: comparing hand and controller interaction

Virtual reality shows great potential as an alternative to traditional therapies for motor rehabilitation given its ability to immerse the user in engaging scenarios that abstract them from medical facilities and tedious rehabilitation exercises. This paper presents a virtual reality application that includes three serious games and that was developed for motor rehabilitation. It uses a standalone headset and the user's hands without the need for any controller for interaction. Interacting with an immersive virtual reality environment using only natural hand gestures involves an interaction that is similar to that of real life, which would be especially desirable for patients with motor problems. A study involving 28 participants (4 with motor problems) was carried out to compare two types of interaction (hands vs. controllers). All of the participants completed the exercises. No significant differences were found in the number of attempts necessary to complete the games using the two types of interaction. The group that used controllers required less time to complete the exercise. The performance outcomes were independent of the gender and age of the participants. The subjective assessment of the participants with motor problems was not significantly different from the rest of the participants. With regard to the interaction type, the participants mostly preferred the interaction using their hands (78.5%). All four participants with motor problems preferred the hand interaction. These results suggest that the interaction with the user’s hands together with standalone headsets could improve motivation, be well accepted by motor rehabilitation patients, and help to complete exercise therapy at home.

Virtual reality for psycho-education on self-stigma in depression: Design of a randomised controlled trial

Major Depressive Disorder (MDD) is a prevalent and disabling mental health condition. Patients with MDD often deal with self-stigma, which can lead to more depressive symptoms. Psychoeducation about depression has proven to be effective in reducing depressive symptomatology and self-stigma. Involving a significant other in psychoeducation for depression, might increase mutual understanding. Virtual reality (VR) offers the opportunity to experience the perspective of having or living with someone with a mental condition. For this study an immersive VR environment is developed. The main objective of this study is to test whether our VR psychoeducation intervention is more successful in reducing self-stigma than standard psychoeducation for MDD. In this randomised controlled trial (RCT), 80 couples of patients and their significant other will be included and randomly assigned to one of two conditions: the VR psychoeducation intervention and standard psychoeducation. Patients will be aged 18 to 65, diagnosed with MDD. The main study parameter is self-stigma, as measured by the Internalized Stigma of Mental Illness scale. Secondary parameters include depressive symptoms, loneliness and perceived social support for the patient and burden of care and quality of life for the significant other. No control for nonspecific factors, limited individual adjustment, patients are not able to participate without a significant other. VR might open up the opportunity to reduce self-stigma and thereby improve the efficacy of psychoeducation in MDD.