Immersive Virtual Reality System Research Articles

SONIA: an immersive customizable virtual reality system for the education and exploration of brain networks

While mastery of neuroanatomy is important for the investigation of the brain, there is an increasing interest in exploring the neural pathways to better understand the roles of neural circuitry in brain functions. To tackle the limitations of traditional 2D-display-based neuronavigation software in intuitively visualizing complex 3D anatomies, several virtual reality (VR) and augmented reality (AR) solutions have been proposed to facilitate neuroanatomical education. However, with the increasing knowledge on brain connectivity and the functioning of the sub-systems, there is still a lack of similar software solutions for the education and exploration of these topics, which demand more elaborate visualization and interaction strategies. To address this gap, we designed the immerSive custOmizable Neuro learnIng plAtform (SONIA), a novel, user-friendly VR software system with a multi-scale interaction paradigm that allowed flexible customization of learning materials. With both quantitative and qualitative evaluations through user studies, the proposed system was shown to have high usability, attractive visual design, and good educational value. As the first immersive system that integrated customizable design and detailed narratives of the brain sub-systems for the education of neuroanatomy and brain connectivity, SONIA showcased new potential directions and provided valuable insights regarding medical learning and exploration in VR.

Training using a commercial immersive virtual reality system on hand–eye coordination and reaction time in students: a randomized controlled trial

The implementation of VR games opens up a wide range of opportunities for the development of dexterity, speed and precision of movements among various professional groups. The aim of this study was to investigate the effects of a commercial immersive VR music game on hand–eye coordination and reaction time speed in students. This study enrolled 32 individuals, randomly assigned to the experimental or control group. The intervention consisted of a 15-min training session of the immersive music game “Beat Saber”, once a day for 5 consecutive days. The primary outcomes included reaction time measurements: the plate tapping test and the ruler-drop test (Ditrich's test), trial making test (TMT) A and TMT B to assess coordination and visual attention, likewise VR sickness assessment by Virtual Reality Sickness Questionnaire (VRSQ). The secondary outcome included an energy expenditure assessment (SenseWear Armband). The data analysis revealed a statistically significant improvement in hand–eye coordination in the experimental group, with no improvement in the control group. The results were similar in measurements of reaction time. Analysis of the VRSQ questionnaire results showed a statistically significant reduction in oculomotor domain symptoms and total score during successive training days. The immersive VR music game has the potential to improve reaction time and hand–eye coordination in students.Graphical abstract

VisionCoach: Design and Effectiveness Study on VR Vision Training for Basketball Passing.

Vision Training is important for basketball players to effectively search for teammates who has wide-open opportunities to shoot, observe the defenders around the wide-open teammates and quickly choose a proper way to pass the ball to the most suitable one. We develop an immersive virtual reality (VR) system called VisionCoach to simulate the player's viewing perspective and generate three designed systematic vision training tasks to benefit the cultivating procedure. By recording the player's eye gazing and dribbling video sequence, the proposed system can analyze the vision-related behavior to understand the training effectiveness. To demonstrate the proposed VR training system can facilitate the cultivation of vision ability, we recruited 14 experienced players to participate in a 6-week between-subject study, and conducted a study by comparing the most frequently used 2D vision training method called Vision Performance Enhancement (VPE) program with the proposed system. Qualitative experiences and quantitative training results are reported to show that the proposed immersive VR training system can effectively improve player's vision ability in terms of gaze behavior and dribbling stability. Furthermore, training in the VR-VisionCoach Condition can transfer the learned abilities to real scenario more easily than training in the 2D-VPE Condition.

Breaking social isolation for older people living alone with technology

ABSTRACT This paper is aiming to present two studies investigating attitudes of older people towards different technologies to break their social isolation and investigating the impacts of Immersive Virtual Reality (IVR) on physical and mental dimensions. The first study, based on focus groups conducted with 23 older people (aged from 65 to 93 years-old) allowed to collect attitudes and opinions of older people about different technologies to help them to break social isolation. The second study, based on an experiment conducted with 42 older people (aged from 63 to 85 years-old) allowed to investigate the impact of a specific Immersive Virtual Reality (IVR) on physical and mental dimensions. The first study confirm that Immersive Virtual Reality (IVR) can be a relevant digital tool for elderly people according to them, i.e. a digital tool perceived as useful, usable and acceptable. The second study demonstrates that use of a specific Immersive Virtual Reality (IVR) system can reduce the perception of social isolation and can have positive impacts on physical dimensions such as the weight and the waist circumference. Theoretical and applied implications are discussed.

Neurorehabilitation of the upper extremity - immersive virtual reality vs. electromechanically assisted training. A comparative study.

Severe paresis of the contralesional upper extremity is one of the most common and debilitating post-stroke impairments. The need for cost-effective high-intensity training is driving the development of new technologies, which can complement and extent conventional therapies. Apart from established methods using electromechanical devices, immersive virtual reality (iVR) systems hold promise to provide cost-efficient high-intensity arm training. We investigated whether iVR-based arm training yields at least equivalent effects on upper extremity function as compared to an electromechanically assisted training in stroke patients with severe arm paresis. 52 stroke patients with severe arm paresis received a total of ten daily group therapy sessions over a period of three weeks, which consisted of 20 min of conventional therapy and 20 min of either electromechanically assisted (ARMEOSpring®) or iVR-based (CUREO®) arm training. Changes in upper extremity function was assessed using the Action Research Arm Test (ARAT) and user acceptance was measured with the User Experience Questionnaire (UEQ). iVR-based training was not inferior to electromechanically assisted training. We found that 84% of patients treated with iVR and 50% of patients treated with electromechanically assisted arm training showed a clinically relevant improvement of upper extremity function. This difference could neither be attributed to differences between the groups regarding age, gender, duration after stroke, affected body side or ARAT scores at baseline, nor to differences in the total amount of therapy provided. The present study results show that iVR-based arm training seems to be a promising addition to conventional therapy. Potential mechanisms by which iVR unfolds its effects are discussed.

Head-Mounted Displays for Upper Limb Stroke Rehabilitation: A Scoping Review

Upper extremity (UE) paresis is one of the most frequent and disabling clinical consequences after stroke. Head-Mounted Displays (HMDs) are wearable virtual reality devices that seem effective in promoting the recovery of functional abilities by increasing adherence levels in this population. This scoping review is aimed at collecting available evidence on the use of HMD-based immersive virtual reality systems for UE rehabilitation treatment in stroke survivors. Four electronic bibliographic databases were consulted from inception until 18 January 2023. A total of 19 clinical trials in which HMDs were used as a clinical tool for increasing UE functioning, as a single intervention or in adjunct to other rehab treatments, were included; no restrictions were applied for UE paresis severity or stroke onset. The large majority of the clinical trials involved chronic stroke patients (15 out of 19), with a wide range of UE impairments. Overall, HMD use seemed to be well-tolerated and promising for increasing UE motor function in adult chronic stroke survivors, with benefits in subjects’ arm use and independence. The possibility of executing highly realistic and task-oriented movements appears to be promising in enhancing gesture relevance, thus promoting new motor strategies in a “virtual ecological way”. Across studies, we found a high heterogeneity in protocol design and a lack of reporting that prevents us drawing conclusions regarding potential subgroups of patients that could benefit more from HMD-based interventions or suggested treatment modalities.

The educational needs of virtual reality simulation training for novice nurses’ adaptation to clinical practice: A mixed methods study

Purpose: The purpose of this study is to identify the educational needs of virtual reality simulations that can be applied to novice nurses during the waiting period before starting work in a hospital.Methods: A convergent mixed methods was used. The survey data were collected from 230 novice nurses, and a focus group interview was conducted with 6 new nurses. The data were collected from November 2022 to January 2023. Descriptive statistics, a frequency analysis, independent t-test, and an Importance-Performance Analysis were performed using SPSS 24.0.Results: Appropriate topics for virtual reality simulation education were indicated to be medications and intravenous injections, which are high priority topics in quantitative and qualitative research. The novice nurses wanted group activity training three to four times a week for two weeks before beginning work in a hospital. They also wanted an immersive virtual reality system based on a real hospital environment.Conclusion: Based on the above results, this study provides basic data for the development of a virtual reality simulation education that can improve the adaptation of novice nurses to clinical practice. A strategy was suggested to utilize the waiting time before beginning work in a hospital as educational time.

Influence of Normal Aging and Multisensory Data Fusion on Cybersickness and Postural Adaptation in Immersive Virtual Reality

Immersive Virtual Reality (VR) systems are expanding as sensorimotor readaptation tools for older adults. However, this purpose may be challenged by cybersickness occurrences possibly caused by sensory conflicts. This study aims to analyze the effects of aging and multisensory data fusion processes in the brain on cybersickness and the adaptation of postural responses when exposed to immersive VR. Methods: We repeatedly exposed 75 participants, aged 21 to 86, to immersive VR while recording the trajectory of their Center of Pressure (CoP). Participants rated their cybersickness after the first and fifth exposure. Results: The repeated exposures increased cybersickness and allowed for a decrease in postural responses from the second repetition, i.e., increased stability. We did not find any significant correlation between biological age and cybersickness scores. On the contrary, even if some postural responses are age-dependent, a significant postural adaptation occurred independently of age. The CoP trajectory length in the anteroposterior axis and mean velocity were the postural parameters the most affected by age and repetition. Conclusions: This study suggests that cybersickness and postural adaptation to immersive VR are not age-dependent and that cybersickness is unrelated to a deficit in postural adaptation or age. Age does not seem to influence the properties of multisensory data fusion.

Literature Survey: The Potential of Integrating Immersive Experience and Aesthetic Experience in Virtual Reality Historical Event

This study intends to explore the potential of integrating of two major theories that stemmed from two entirely different disciplines, namely the human-computer interaction and the arts. Indeed, there are vast gaps when two different theories, such as technology and art, are to be unified to develop a new element that complements to both disciplines. Empirically, virtual reality as immersive technological innovations are progressively developed and evaluated among various disciplines. Virtual reality technically is customarily to immersive experience as a measurement tool while virtual three-dimensional animated environment in virtual reality is an aesthetic object that requires an aesthetic-friendly method as its measurement tool. The proposition of evaluation is to measure the user immersive experience and aesthetic experience when dealing with an art object that imbues with digital technology. Therefore, an integrative multidimensional evaluation is required to evaluate virtual reality environment. This study uses scientific literature survey which research papers are retrieved from Google Scholar, ScienceDirect, and Scopus. Thus, this paper reports on the potentials of integrating the immersive experience and aesthetic experience theories to evaluate a virtual reality historical event. It explains the concept of immersive experience and aesthetic experience in immersive virtual reality applications in general and museum digital preservation practices in particular. Then focuses on the concept idea of how to integrate immersive experience and aesthetic experience in virtual reality historical event application by discovering the construct and element from both theories. Thus, the finding unveils the possibilities of integration immersive experience and aesthetic experience in terms of integrating the construct and elements, contributing to contemplate new possibilities and discoveries in both fields and become catalysts for future research. This initial concept idea could become fundamental conceptual design reference of developing virtual reality system in museum context used by designers, developers and museum professionals. In the conclusion, this study concluded that there is a significant and lucrative potential to integrate both theory in developing immersive virtual reality systems. Hence recommend to a new integrative framework development that is applicable to immersive product development.

Exploring the Perception of the Effect of Three-Dimensional Interaction Feedback Types on Immersive Virtual Reality Education

Immersive virtual reality (VR) systems are becoming widely used for education with three-dimensional (3D) information. Specifically, three-dimensional spaces to create virtual environments can help increase students’ learning interest and ability with spatial interaction. Also, with the use of multimodal interaction, VR systems can provide highly effective ways to solve problems through natural experiences. Additionally, immersive environments can bring together people in remote locations, which has been increasingly applied in education applications with the use of technology to simulate real situations. However, effective interaction methods that improve the learning ability of people participating in educational activities in 3D immersive environments are yet to be well defined. In this study, we investigated the effect of the interaction feedback types on the perception of students participating in VR environments. We conducted the experiment on three types of interaction responses, and our study was designed as a virtual chemistry class. Our experimental study showed that the interaction feedback type of a deformable object had a greater educational effect than other types of visual or audio feedback, and our results are expected to provide guidelines on how to create effective immersive education content and interaction methods.

Inclusive Learning through Immersive Virtual Reality and Semantic Embodied Conversational Agent: A case study in children with autism

Sustainable Development Goal 4 from the United Nations 2030 Agenda, focus on ensuring inclusive and equitable quality education and promoting lifelong learning opportunities for all. Inclusive education is a key component of sustainable development goal 4, and assistive technology is a critical factor in achieving it. In this context, this paper introduces the development of an immersive virtual reality system with conversational skills for inclusive learning processes. The idea of this work is to provide an assistive technology to motivate, educate, and train students with disabilities in a more inclusive way. In order to improve the user experience, the system was developed to work in a CAVE-like immersive environment using Natural Language Processing through a Semantic Embodied Conversational Agent. This work highlights that assistive technology can promote educational, psychological, and social benefits for students with disabilities. The use of assistive technology can enable academic engagement and social participation and be transformative from a psychological perspective. A case study was conducted on children with autism, which showed encouraging results of the system as an assistive technology for teaching and learning.

Development of an immersive virtual reality system for learning about plants in primary education: evaluation of teachers’ perceptions and learners’ flow experiences and learning attitudes

Development of an immersive virtual reality system for learning about plants in primary education: evaluation of teachers’ perceptions and learners’ flow experiences and learning attitudes

Exploring the user experience of hands-free VR interaction methods during a Fitts’ task

Despite advancements in interaction with immersive Virtual Reality (VR) systems, using hand gestures for all interactions still imposes some challenges, especially in interactions with graphical user interfaces that are usually performed with point-and-click interfaces. Therefore, exploring the use of alternative hands-free methods for selection is essential to overcome usability problems and provide natural interaction for users. The results and insights gained from this exploration can lead to enhanced user experiences in VR applications. This study aims to contribute to the literature with the evaluation of the usability of the most commonly used hands-free methods for selection and system control tasks in immersive VR and their impact on standard and validated experience and usability metrics, namely the sense of presence, cybersickness, system usability, workload, and user satisfaction. A Fitts’ selection task was performed using a within-subjects design by nine participants experienced in VR. The methods evaluated were the handheld controllers, the head gaze, eye gaze, and voice commands for pointing at the targets, and dwell time and voice commands to confirm the selections. Results show that the methods provide similar levels of sense of presence and low cybersickness while showing low workload values and high user satisfaction, matching the experience of traditional handheld controllers for non-multimodal approaches. The assisted eye gaze with dwell was the preferred hands-free method and the one with the highest values of usability. Still, developers should minimize the number of gaze movements to reduce fatigue. The evaluation also showed that using a multimodal approach for selections, especially using the voice, decreases user satisfaction and increases users’ frustration.

Vocabulary Acquisition in New and Learned Contexts Using Immersive Virtual Reality

Vocabulary acquisition is important for learning a language. Language learners need multiple encounters with a word to add unknown word aspects to their word knowledge or to strengthen their knowledge of previously learned word aspects. Combining vocabulary acquisition with immersive virtual reality (IVR) makes it possible to have such multiple encounters. In IVR, objects representing target words can be shown in the same virtual environment as they were first encountered (learned context), but also in a new virtual environment (new context). To investigate learning words in either a new or learned context in IVR we built Wics (words in context system). We used Wics to carry out an experiment in which participants could learn 32 Japanese words, represented by 3D objects, in three sessions. In the second and third learning session all words were shown in either a new context, with a changed environment and a changed visual representation of the word objects, or a learned context, where only the placement of the objects was changed. Performance scores for an immediate and a delayed post-test (one week later) revealed no significant differences in performance between conditions. However, there was a tendency towards participants spending more time on learning the words in the learned context condition. We believe Wics to be the first IVR system to provide learners with multiple learning sessions to provide multiple encounters with the same words in different contexts, where learners have control over their own explorative learning, and where the post-tests are inside IVR.

Restoring Dunhuang Murals: Crafting Cultural Heritage Preservation Knowledge into Immersive Virtual Reality Experience Design

With the development of digital technology, tourists have higher expectations for their experiences with digital cultural heritage (CH). However, there is limited research and design guidelines on how to effectively convey expert knowledge and transfer the value and connotations of digital CH to users in a virtual environment. Based on existing literature, we proposed a systematic design strategy for immersive virtual reality (IVR) systems, incorporating multimodal interaction, gamification, and storytelling for CH presentation, development, and promotion to the public. Accordingly, taking the virtual restoration of the Dunhuang murals as an example, we developed an IVR system that provided users with learning, interactive, and entertaining experiences, facilitating their transition from mere visitors to active learners while engaging with digital CH. We conducted a between-subjects user study involving 42 participants. The results demonstrated that our IVR system motivated users interests in CH, promoted CH preservation awareness, and could be applied in the domain of digital CH experiences.

Design of immersive virtual reality training for endoscopic surgery

Endoscopic surgery training is limited by a lack of specialized instructors and access to technological equipment. Virtual reality training offers a solution to this issue, as it allows for accessible and efficient practice. This study conducts a literature review to investigate the current state of virtual reality training systems in endoscopic surgery and subsequently designs and tests an immersive virtual reality system through a proof of concept with endoscopic surgery specialists.

Fully immersive virtual reality for skull-base surgery: surgical training and beyond.

A virtual reality (VR) system, where surgeons can practice procedures on virtual anatomies, is a scalable and cost-effective alternative to cadaveric training. The fully digitized virtual surgeries can also be used to assess the surgeon's skills using measurements that are otherwise hard to collect in reality. Thus, we present the Fully Immersive Virtual Reality System (FIVRS) for skull-base surgery, which combines surgical simulation software with a high-fidelity hardware setup. FIVRS allows surgeons to follow normal clinical workflows inside the VR environment. FIVRS uses advanced rendering designs and drilling algorithms for realistic bone ablation. A head-mounted display with ergonomics similar to that of surgical microscopes is used to improve immersiveness. Extensive multi-modal data are recorded for post-analysis, including eye gaze, motion, force, and video of the surgery. A user-friendly interface is also designed to ease the learning curve of using FIVRS. We present results from a user study involving surgeons with various levels of expertise. The preliminary data recorded by FIVRS differentiate between participants with different levels of expertise, promising future research on automatic skill assessment. Furthermore, informal feedback from the study participants about the system's intuitiveness and immersiveness was positive. We present FIVRS, a fully immersive VR system for skull-base surgery. FIVRS features a realistic software simulation coupled with modern hardware for improved realism. The system is completely open source and provides feature-rich data in an industry-standard format.

EEG-Based Brain-Computer Interactions in Immersive Virtual and Augmented Reality: A Systematic Review

Brain-computer interactions allow humans to passively or actively control computer systems using their brain activity. For more than a decade now, these interactions have been implemented and evaluated in immersive virtual environments where they prompt novel means of human interaction with systems. In this paper, we present a systematic review of 76 studies published over the last 10 years that develop and evaluate immersive virtual reality or augmented reality systems with electroencephalography-based interactions. The aim of the review is to summarize and highlight trends in technology design, research methods, current practices, techniques used in systems of this kind, challenges and opportunities that present direction for future research in this area. Our analysis uncovers useful insights, limitations, and highlights of the trends, innovations, and usability and technical challenges at the intersection of brain-computer interfaces and immersive technologies, as well as recommendations for future research.

Digital Therapeutics (DTx) Expand Multimodal Treatment Options for Chronic Low Back Pain: The Nexus of Precision Medicine, Patient Education, and Public Health.

Digital therapeutics (DTx, software as a medical device) provide personalized treatments for chronic diseases and expand precision medicine beyond pharmacogenomics-based pharmacotherapies. In this perspective article, we describe how DTx for chronic low back pain (CLBP) can be integrated with pharmaceutical drugs (e.g., NSAIDs, opioids), physical therapy (PT), cognitive behavioral therapy (CBT), and patient empowerment. An example of an FDA-authorized DTx for CLBP is RelieVRx, a prescription virtual reality (VR) app that reduces pain severity as an adjunct treatment for moderate to severe low back pain. RelieVRx is an immersive VR system that delivers at-home pain management modalities, including relaxation, self-awareness, pain distraction, guided breathing, and patient education. The mechanism of action of DTx is aligned with recommendations from the American College of Physicians to use non-pharmacological modalities as the first-line therapy for CLBP. Herein, we discuss how DTx can provide multimodal therapy options integrating conventional treatments with exposome-responsive, just-in-time adaptive interventions (JITAI). Given the flexibility of software-based therapies to accommodate diverse digital content, we also suggest that music-induced analgesia can increase the clinical effectiveness of digital interventions for chronic pain. DTx offers opportunities to simultaneously address the chronic pain crisis and opioid epidemic while supporting patients and healthcare providers to improve therapy outcomes.

Virtual Art Therapy: Application of Michelangelo Effect to Neurorehabilitation of Patients with Stroke.

In neurorehabilitation, some studies reported the effective use of art therapy for reducing psychological disorders and for enhancing physical functions and cognitive abilities. Neuroaesthetical studies showed that seeing an art masterpiece can spontaneously elicit a widespread brain arousal, also involving motor networks. To combine contemplative and performative benefits of art therapy protocols, we have developed an immersive virtual reality system, giving subjects the illusion that they are able to paint a copy of famous artistic paintings. We previously observed that during this virtual task, subjects perceived less fatigue and performed more accurate movements than when they were asked to color the virtual canvas. We named this upshot the Michelangelo effect. The aim of this study was to test the rehabilitative efficacy of our system. Ten patients with stroke in the subacute phase were enrolled and trained for one month with virtual art therapy (VAT) and physiotherapy. Their data were compared with those of ten patients matched for pathology, age and clinical parameters, trained only with conventional therapy for the same amount of time. The VAT group showed a significantly higher improvements in the Barthel Index score, a measure of independency in activities of daily living (66 ± 33% vs. 31 ± 28%, p = 0.021), and in pinching strength (66 ± 39% vs. 18 ± 33%, p = 0.008), with respect to the group treated with conventional rehabilitation.