Virtual Reality Setup Research Articles

Designing a virtual reality system for clinical education and examination

The COVID-19 pandemic clearly showed how much medical and clinical students’ education relied on physical instruction and examination. The work presented here was born by the fact that medical and clinical education had to be suspended during the social distancing phase of the pandemic. This work presents the lessons learned from creating a virtual reality system (VTRACS) for educating and assessing clinical students on a limited budget. Our work showed that clinical education could be done in a virtual space with a minimal virtual reality setup and a low-cost virtual reality headset.

Feeling senseless sensations: a crossmodal EEG study of mismatched tactile and visual experiences in virtual reality

Objective.To create highly immersive experiences in virtual reality (VR) it is important to not only include the visual sense but also to involve multimodal sensory input. To achieve optimal results, the temporal and spatial synchronization of these multimodal inputs is critical. It is therefore necessary to find methods to objectively evaluate the synchronization of VR experiences with a continuous tracking of the user.Approach.In this study a passive touch experience was incorporated in a visual-tactile VR setup using VR glasses and tactile sensations in mid-air. Inconsistencies of multimodal perception were intentionally integrated into a discrimination task. The participants' electroencephalogram (EEG) was recorded to obtain neural correlates of visual-tactile mismatch situations.Main results.The results showed significant differences in the event-related potentials (ERP) between match and mismatch situations. A biphasic ERP configuration consisting of a positivity at 120 ms and a later negativity at 370 ms was observed following a visual-tactile mismatch.Significance.This late negativity could be related to the N400 that is associated with semantic incongruency. These results provide a promising approach towards the objective evaluation of visual-tactile synchronization in virtual experiences.

Head-area sensing in virtual reality: future visions for visual perception and cognitive state estimation

Biosensing techniques are progressing rapidly, promising the emergence of sophisticated virtual reality (VR) headsets with versatile biosensing enabling an objective, yet unobtrusive way to monitor the user’s physiology. Additionally, modern artificial intelligence (AI) methods provide interpretations of multimodal data to obtain personalised estimations of the users’ oculomotor behaviour, visual perception, and cognitive state, and their possibilities extend to controlling, adapting, and even creating the virtual audiovisual content in real-time. This article proposes a visionary approach for personalised virtual content adaptation via novel and precise oculomotor feature extraction from a freely moving user and sophisticated AI algorithms for cognitive state estimation. The approach is presented with an example use-case of a VR flight simulation session explaining in detail how cognitive workload, decrease in alertness level, and cybersickness symptoms could be modified in real-time by using the methods and embedded stimuli. We believe the envisioned approach will lead to significant cost savings and societal impact and will thus be a necessity in future VR setups. For instance, it will increase the efficiency of a VR training session by optimizing the task difficulty based on the user’s cognitive load and decrease the probability of human errors by guiding visual perception via content adaptation.

Age-related changes in time perception: Effects of immersive virtual reality and spatial location of stimuli

The perception of time is subject to various environmental influences and exhibits changes across the lifespan. Studies on time perception have often been conducted using abstract stimuli and artificial scenarios, and recent claims for more naturalistic paradigms and realistic stimuli pose the question as to whether immersive virtual reality set-ups differently affect the timing abilities of older versus younger adults. Here, we tested the hypotheses that naturalistic 3D stimuli presented in immersive virtual reality (as opposed to abstract 2D stimuli presented on a computer screen) and the spatial location of those stimuli (left vs. right) affect the perceived time point of their occurrence.Our results demonstrate that a naturalistic presentation of stimuli leads to a bias towards earlier time points in younger, but not older participants. Furthermore, this bias was associated with lower scores of memory capacity. Contrary to our hypothesis that right-sided stimuli are perceived as later than left-sided stimuli, no spatial influences on temporal processing were observed. These results show that older and younger adults are differently affected by an increase in the realism and the immersiveness of experimental paradigms, and highlight the importance of task design in studies on human time perception.

Smart ArM: a customizable and versatile robotic arm prosthesis platform for Cybathlon and research

BackgroundIn the last decade, notable progress in mechatronics paved the way for a new generation of arm prostheses, expanding motor capabilities thanks to their multiple active joints. Yet, the design of control schemes for these advanced devices still poses a challenge, especially with the limited availability of command signals for higher levels of arm impairment. When addressing this challenge, current commercial devices lack versatility and customizing options to be employed as test-beds for developing novel control schemes. As a consequence, researchers resort to using lab-specific experimental apparatuses on which to deploy their innovations, such as virtual reality setups or mock prosthetic devices worn by unimpaired participants.MethodsTo meet this need for a test-bed, we developed the Smart Arm platform, a human-like, multi-articulated robotic arm that can be worn as a trans-humeral arm prosthesis. The design process followed three principles: provide a reprogrammable embedded system allowing in-depth customization of control schemes, favor easy-to-buy parts rather than custom-made components, and guarantee compatibility with industrial standards in prosthetics.ResultsThe Smart ArM platform includes motorized elbow and wrist joints while being compatible with commercial prosthetic hands. Its software and electronic architecture can be easily adapted to build devices with a wide variety of sensors and actuators. This platform was employed in several experiments studying arm prosthesis control and sensory feedback. We also report our participation in Cybathlon, where our pilot with forearm agenesia successfully drives the Smart Arm prosthesis to perform activities of daily living requiring both strength and dexterity.ConclusionThese application scenarios illustrate the versatility and adaptability of the proposed platform, for research purposes as well as outside the lab. The Smart Arm platform offers a test-bed for experimenting with prosthetic control laws and command signals, suitable for running tests in lifelike settings where impaired participants wear it as a prosthetic device. In this way, we aim at bridging a critical gap in the field of upper limb prosthetics: the need for realistic, ecological test conditions to assess the actual benefit of a technological innovation for the end-users.

The Usefulness of a Virtual Environment-Based Patient Setup Training System for Radiation Therapy.

In radiation therapy, patient setup is important for improving treatment accuracy. The six-axis couch semi-automatically adjusts the patient's position; however, adjusting the patient to twist is difficult. In this study, we developed and evaluated a virtual reality setup training tool for medical students to understand and improve their patient setup skills for radiation therapy. First, we set up a simulated patient in a virtual space to reproduce the radiation treatment room. A gyro sensor was attached to the patient phantom in real space, and the twist of the phantom was linked to the patient in the virtual space. Training was conducted for 24 students, and their operation records were analyzed and evaluated. The training's efficacy was also evaluated through questionnaires provided at the end of the training. The total time required for patient setup tests before and after training decreased significantly from 331.9 s to 146.2 s. As a result of the questionnaire regarding the usability of training to the trainee, most were highly evaluated. We found that training significantly improved students' understanding of the patient setup. With the proposed system, trainees can experience a simulated setup that can aid in deepening their understanding of radiation therapy treatments.

The Role of Virtual Reality in Enhancing Motor Skills in Children: A Systematic Review

This research aims to analyze how virtual reality (VR) can enhance movement skills in children. We employed a systematic review model, searching for relevant articles in databases such as PubMed, Web of Science, and ScienceDirect. The search keywords included "virtual reality," "child," "social," and "motor skills." This study's inclusion criteria were articles focused on "virtual reality," "children with motor disorders," and "motor skills," published within the last five years (as of 2019). Articles from less well-known journals were excluded. We identified eight articles that met the inclusion criteria and were suitable for systematic review according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards. The findings indicate that VR plays a significant role in improving motor coordination abilities in children. Responses to VR interventions were overwhelmingly positive, showing reductions in physical, cognitive, and emotional stress. Therefore, VR proves to be a valuable tool for physiotherapists and physicians treating children with motor delays. However, some considerations need to be addressed. A few parents have raised concerns about the high costs and the substantial space requirements associated with VR setups. Keywords: virtual reality, motion disorders, motor, technology, children

Assessing the well-being benefits of VR nature experiences on group: Heart rate variability insights from a cross-over study

Recent trends show a rise in stress and psychological disorders, necessitating innovative strategies for recovery. Our study examines the psychophysiological effects of virtual reality (VR) nature, based on the biophilia hypothesis, within real-life group settings. A randomized, acute cross-over design was employed, involving 57 participants who experienced a 10-min session in either a VR nature or a reference room, separated by a 2-min interval. Participants were measured in groups of 7–10 individuals. The VR setup featured a landscape video projected around the participants, creating a captivating 3D environment, whereas the reference room utilized the same space but without the VR elements, maintaining a basic meeting room atmosphere with white walls and a quiet setting. The primary outcome measured was heart rate variability (HRV), complemented by exploratory outcomes including the heart and respiratory rates, and questionnaires on affective well-being, creativity, and sense of belonging. Findings revealed that VR nature exposure resulted in higher HRV and reduced heart rate, indicative of enhanced parasympathetic activity. Participants reported decreased feelings of anxiety and depression, with an increase in comfort, enthusiasm, creativity, and belonging. These results present the VR nature's physiological and psychological benefits, demonstrating its effectiveness in promoting relaxation and recovery in a real-life group setting. In conclusion, the VR nature room can emerge as a versatile tool for stress alleviation and well-being improvement.

Bodily and Visual-Cognitive Navigation Aids to Enhance Spatial Recall in Mild Cognitive Impairment.

Individuals with mild cognitive impairment (MCI) syndrome often report navigation difficulties, accompanied by impairments in egocentric and allocentric spatial memory. However, studies have shown that both bodily (e.g., motor commands, proprioception, vestibular information) and visual-cognitive (e.g., maps, directional arrows, attentional markers) cues can support spatial memory in MCI. We aimed to assess navigation cues for innovative spatial training in aging. Fifteen MCI patients were recruited for this study. Their egocentric and allocentric memory recall performances were tested through a navigation task with five different virtual reality (VR) assistive encoding navigation procedures (bodily, vision only, interactive allocentric map, reduced executive load, free navigation without cues). Bodily condition consisted of an immersive VR setup to engage self-motion cues, vision only condition consisted of passive navigation without interaction, in the interactive allocentric map condition patients could use a bird-view map, in the reduced executive load condition directional cues and attentional markers were employed, and during free navigation no aid was implemented. Bodily condition improved spatial memory compared to vision only and free navigation without cues. In addition, the interactive allocentric map was superior to the free navigation without cues. Surprisingly, the reduced executive load was comparable to vison only condition. Moreover, a detrimental impact of free navigation was observed on allocentric memory across testing trials. These findings challenge the notion of an amodal representation of space in aging, suggesting that spatial maps can be influenced by the modality in which the environment was originally encoded.

Urban Green Spaces and Mental Well-Being: A Systematic Review of Studies Comparing Virtual Reality versus Real Nature

Increasingly, urban planners are adopting virtual reality (VR) in designing urban green spaces (UGS) to visualize landscape designs in immersive 3D. However, the psychological effect of green spaces from the experience in VR may differ from the actual experience in the real world. In this paper, we systematically reviewed studies in the literature that conducted experiments to investigate the psychological benefits of nature in both VR and the real world to study nature in VR anchored to nature in the real world. We separated these studies based on the type of VR setup used, specifically, 360-degree video or 3D virtual environment, and established a framework of commonly used standard questionnaires used to measure the perceived mental states. The most common questionnaires include Positive and Negative Affect Schedule (PANAS), Perceived Restorativeness Scale (PRS), and Restoration Outcome Scale (ROS). Although the results from studies that used 360-degree video were less clear, results from studies that used 3D virtual environments provided evidence that virtual nature is comparable to real-world nature and thus showed promise that UGS designs in VR can transfer into real-world designs to yield similar physiological effects.

Zero-Shot Segmentation of Eye Features Using the Segment Anything Model (SAM)

The advent of foundation models signals a new era in artificial intelligence. The Segment Anything Model (SAM) is the first foundation model for image segmentation. In this study, we evaluate SAM's ability to segment features from eye images recorded in virtual reality setups. The increasing requirement for annotated eye-image datasets presents a significant opportunity for SAM to redefine the landscape of data annotation in gaze estimation. Our investigation centers on SAM's zero-shot learning abilities and the effectiveness of prompts like bounding boxes or point clicks. Our results are consistent with studies in other domains, demonstrating that SAM's segmentation effectiveness can be on-par with specialized models depending on the feature, with prompts improving its performance, evidenced by an IoU of 93.34% for pupil segmentation in one dataset. Foundation models like SAM could revolutionize gaze estimation by enabling quick and easy image segmentation, reducing reliance on specialized models and extensive manual annotation.

Multimodal emotion classification using machine learning in immersive and non-immersive virtual reality

Affective computing has been widely used to detect and recognize emotional states. The main goal of this study was to detect emotional states using machine learning algorithms automatically. The experimental procedure involved eliciting emotional states using film clips in an immersive and non-immersive virtual reality setup. The participants’ physiological signals were recorded and analyzed to train machine learning models to recognize users’ emotional states. Furthermore, two subjective ratings emotional scales were provided to rate each emotional film clip. Results showed no significant differences between presenting the stimuli in the two degrees of immersion. Regarding emotion classification, it emerged that for both physiological signals and subjective ratings, user-dependent models have a better performance when compared to user-independent models. We obtained an average accuracy of 69.29 ± 11.41% and 71.00 ± 7.95% for the subjective ratings and physiological signals, respectively. On the other hand, using user-independent models, the accuracy we obtained was 54.0 ± 17.2% and 24.9 ± 4.0%, respectively. We interpreted these data as the result of high inter-subject variability among participants, suggesting the need for user-dependent classification models. In future works, we intend to develop new classification algorithms and transfer them to real-time implementation. This will make it possible to adapt to a virtual reality environment in real-time, according to the user’s emotional state.

A Canvas of Air and Signs: Integrating Voice Activated Hand Sign Recognition and Air Canvas for Hearing Impaired and Non-Verbal People

Moving hands in air and watching your screen writes for you and at the same time making different Signs with hands and the system displays it as well as speak what Sign you are making.This seems to be very futuristic approach towards the realm of Image Processing and Gesture Recognition . In this paper, we present a very interesting and a novel approach towards an interactive learning platform where one can draw the content on screen while moving their hand in air and can also use hand sign language to communicate with an ease with the Hearing Impaired and Dumb Community. Our system combines both technologies to create a smooth and engaging experience for users. It can be used in interactive art setups or virtual reality setups. Air canvas enables users to draw and manipulate digital content in mid-air with object tracking using Computer Vision and Mediapipe framework, while hand gesture recognition allows for real-time interpretation of Hand Signs to perform actions or commands within the system. This Model not only recognizes the sign but also speaks it loud using pyttsx3 a text-to-speech conversion Library, ensuring a good communication between a normal human and people with Non-Verbal and Hearing Impaired disability. To enhance the performance of the model We validate the model with a real dataset trained by us. This training was essential for refining the accuracy and efficiency of the model.

The synergy complement control approach for seamless limb-driven prostheses

Limb-driven control allows for direct control by using residual limb movements rather than unnatural and complex muscle activation. Existing limb-driven methods simultaneously learn a variety of possible motions, ranging from a residual limb to entire arm motions, from human templates by relying on linear or nonlinear regression techniques. However, the map between a low-dimensional residual limb movement and high-dimensional total limb movement is highly underdetermined. Therefore, this complex, high-dimensional coordination problem cannot be accurately solved by treating it as a data-driven black box problem. Here we address this challenge by introducing the residual limb-driven control framework synergy complement control. Firstly, the residual limb drives a one-dimensional phase variable to simultaneously control the multiple joints of the prosthesis. Secondly, the resulting prosthesis motion naturally complements the movement of the residual limb by its synergy components. Furthermore, our framework adds information on contextual tasks and goals and allows for seamless transitions between these. Experimental validation was conducted using subjects with preserved arms employing an exo-prosthesis setup, and studies involving participants with and without limb differences in a virtual reality setup. The findings affirm that the restoration of lost coordinated synergy capabilities is reliably achieved through the utilization of synergy complement control with the prosthesis.

Exploring Virtual Environments to Assess the Quality of Public Spaces

Human impression plays a crucial role in effectively designing infrastructures that support active mobility such as walking and cycling. By involving users early in the design process, valuable insights can be gathered before physical environments are constructed. This proactive approach enhances the attractiveness and safety of designed spaces for users. This study conducts an experiment comparing real street observations with immersive virtual reality (VR) visits to evaluate user perceptions and assess the quality of public spaces. For this experiment, a high-resolution 3D city model of a large-scale neighborhood was created, utilizing Building Information Modeling (BIM) and Geographic Information System (GIS) data. The model incorporated dynamic elements representing various urban environments: a public area with a tramway station, a commercial street with a road, and a residential playground with green spaces. Participants were presented with identical views of existing urban scenes, both in reality and through reconstructed 3D scenes using a Head-Mounted Display (HMD). They were asked questions related to the quality of the streetscape, its walkability, and cyclability. From the questionnaire, algorithms for assessing public spaces were computed, namely Sustainable Mobility Indicators (SUMI) and Pedestrian Level of Service (PLOS). The study quantifies the relevance of these indicators in a VR setup and correlates them with critical factors influencing the experience of using and spending time on a street. This research contributes to understanding the suitability of these algorithms in a VR environment for predicting the quality of future spaces before occupancy.

Towards the Legibility of Multi-Robot Systems

Communication is crucial for human-robot collaborative tasks. In this context, legibility studies movement as the means of implicit communication between robotic systems and a human observer. This concept has been explored mostly for manipulators and humanoid robots. In contrast, little information is available in the literature about legibility of multi-robot systems or swarms, where simplicity and non-anthropomorphism of robots, along with the complexity of their interactions and aggregated behavior impose different challenges that are not encountered in single-robot scenarios. This paper investigates legibility of multi-robot systems. Hence, we extend the definition of legibility, incorporating information about high-level goals in terms of the coordination objective of the group of robots, to previous results that focused solely on the legibility of spatial goals. A set of standard multi-robot algorithms corresponding to different coordination objectives are implemented and their legibility is evaluated in a user study, where participants observe the behavior of the multi-robot system in a virtual reality setup and are asked to identify the system’s spatial goal and coordination objective. The results of the study confirmed that coordination objectives are discernible by the users, hence multi-robot systems can be controlled to be legible, in terms of spatial goal and coordination objective.

Topographic mapping of the sensorimotor qualities of empathic reactivity: A psychophysiological study in people with spinal cord injuries.

The experience of empathy for pain is underpinned by sensorimotor and affective dimensions which, although interconnected, are at least in part behaviorally and neurally distinct. Spinal cord injuries (SCI) induce a massive, below-lesion level, sensorimotor body-brain disconnection. This condition may make it possible to test whether sensorimotor deprivation alters specific dimensions of empathic reactivity to observed pain. To explore this issue, we asked SCI people with paraplegia and healthy controls to observe videos of painful or neutral stimuli administered to a hand (intact) or a foot (deafferented). The stimuli were displayed by means of a virtual reality set-up and seen from a first person (1PP) or third person (3PP) visual perspective. A number of measures were recorded ranging from explicit behaviors like explicit verbal reports on the videos, to implicit measures of muscular activity (like EMG from the corrugator and zygomatic muscles that may represent a proxy of sensorimotor empathy) and of autonomic reactivity (like the electrodermal response and Respiratory Sinus Arrhythmia that may represent a general proxy of affective empathy). While no across group differences in explicit verbal reports about the pain stimuli were found, SCI people exhibited reduced facial muscle reactivity to the stimuli applied to the foot (but not the hand) seen from the 1PP. Tellingly, the corrugator activity correlated with SCI participants' neuropathic pain. There were no across group differences in autonomic reactivity suggesting that SCI lesions may affect sensorimotor dimensions connected to empathy for pain.

Improving Learners’ Self-Efficacy in Performing Design Reviews with Virtual Reality

Research on the use of innovative technology, such as virtual reality (VR), in Architecture, Engineering, and Construction (AEC) education, has been growing in the past twenty years. However, such research still requires robust investigation, as few studies have adopted educational psychology theories and rigorous methods. Furthermore, most of the research has focused on the cognitive effects of VR and additional research is also needed to investigate the affective effects, such as motivation, engagement, and self-efficacy. This study aims to evaluate the effects of technology decision-making between immersive virtual reality (IVR) and non-immersive virtual reality (nIVR) setups for a learning activity on AEC learners. For this, three hypotheses are formulated and tested on 165 UK students. Based on the results, both the designed IVR and nIVR learning activities had significant positive effects on learners’ self-efficacy and user experience with no significant difference between IVR and nIVR delivery. With this research, the authors contribute to the growing literature on VR implementation in AEC classrooms by showcasing a study founded on educational psychology theory and by using a rigorous research methodology. Furthermore, this study illustrates the effects that IVR and nIVR have on students’ affective learning and opens the possibility of new research in the field.

Measuring trust with the Wayfinding Task: Implementing a novel task in immersive virtual reality and desktop setups across remote and in-person test environments.

Trust is a key feature of social relationships. Common measures of trust, questionnaires and economic games, lack ecological validity. Hence, we sought to introduce an immersive, virtual reality (VR) measure for the behavioral assessment of trust across remote and in-person settings, building on the maze task of Hale et al. (2018). Our 'Wayfinding Task' consists of an interconnected urban environment for participants to navigate on the advice of two characters of differing trustworthiness. We present four studies implementing the Wayfinding Task in remote and in-person testing environments and comparing performance across head-mounted display (HMD)-based VR and desktop setups. In each study, the trustworthiness of two virtual characters was manipulated, through either a fact sheet providing trustworthiness information, or a behavior-based trustworthiness manipulation task termed the Door Game, based on Van der Biest et al., 2020. Participants then completed the Wayfinding Task. Overall, we found that participant behavior in the Wayfinding Task reflected the relative trustworthiness of the two characters; in particular, the trustworthy character was approached more often for advice, reflecting data from our Door Game. We found mostly null results for our novel outcome measure, interpersonal distance. Remote testing successfully achieved these effects. While HMD-based VR and desktop setups both showed these effects, there was a stronger effect of trustworthiness in the HMD VR version of the task. These results have implications for the measurement of trust in behavioral settings and the use of remote and VR-based testing in social experiments.

Multisensory input modulates memory-guided spatial navigation in humans

Efficient navigation is supported by a cognitive map of space. The hippocampus plays a key role for this map by linking multimodal sensory information with spatial memory representations. However, in human navigation studies, the full range of sensory information is often unavailable due to the stationarity of experimental setups. We investigated the contribution of multisensory information to memory-guided spatial navigation by presenting a virtual version of the Morris water maze on a screen and in an immersive mobile virtual reality setup. Patients with hippocampal lesions and matched controls navigated to memorized object locations in relation to surrounding landmarks. Our results show that availability of multisensory input improves memory-guided spatial navigation in both groups. It has distinct effects on navigational behaviour, with greater improvement in spatial memory performance in patients. We conclude that congruent multisensory information shifts computations to extrahippocampal areas that support spatial navigation and compensates for spatial navigation deficits.