Interactive Virtual Reality Research Articles

Digital twin-driven dynamic monitoring system of the upper limb force

Digital twin represents the core technology to realize the dynamic monitoring of complex industrial systems. However, the human body, as the most complex system in the physical world, digital twin is rarely applied in it. In this study, we successfully demonstrated a digital twin in the human biomedical application by proposing a dynamic monitoring system of the upper limb force. In this system, the real upper limb drives the motion of the virtual one in real-time and dynamically updates the force. Meanwhile, the virtual upper limb feeds back the monitoring-results of the force to the controller of the real upper limb via immersive virtual reality interaction. Experimental results of the typical motions of the upper limb revealed that the proposed system functioned interactively in real-time in a non-invasive manner, while ensuring the accurate solving of the muscle force. In conclusion, our digital twin-driven system is of great importance for rehabilitation medicine, biomechanical scientific research and physical training, promoting the application of the digital twin in the human biomedical field.

Manipulation Guidance Field for Collaborative Object Manipulation in VR

Object manipulation is a fundamental interaction in virtual reality (VR). Efficient and accurate manipulation is important for many VR applications, especially collaborative VR applications. We introduce a collaborative method based on the manipulation guidance field (MGF) to improve manipulation accuracy and efficiency. MGF aims to guide users of different manipulation types to different manipulation viewpoints to efficiently and collaboratively manipulate objects. First, we introduce the concept of MGF and its construction method. Two strategies are offered to accelerate the MGF updating process. A collaborative manipulation method to manipulate objects using the guidance of MGF is then proposed. Finally, a user study (n = 36 participants) was conducted to evaluate the efficiency and accuracy of our MGF-based collaborative object manipulation method in three scenes: (1) Livingroom scene; (2) WaveHouse scene; (3) Pipe scene. Compared to a control method without MGF, the results show that our MGF-based method has significantly reduced task completion time, position error, rotation error, and task load.

DinosaurVR: Using Virtual Reality to Enhance a Museum Exhibition

Museums featuring dinosaur fossils have always attracted the attention of the crowd. However, sustaining public interest in science becomes more challenging year after year, even for popular attractions, calling for changes in how exhibits appear in the face of new technologies. To enrich the user’s experience and enhance the exhibition’s attractiveness, we developed and evaluated an immersive and interactive Virtual Reality (VR) experience to integrate the Paleontology exhibition at the Câmara Cascudo Museum in Natal, Brazil. Experienced VR users were interviewed and reported, through software tests, that the system was stable and performed as intended without any noticeable issues that could affect the user experience. They also found the content adequate, except for the suggestion to include more information about Paleoichnology and the finding of fossil tracksites. The graphical quality received mixed reviews, with some participants suggesting improvements to the terrain, such as adding vegetation and enhancing lighting. They also noted that the experience would appeal to younger audiences, thanks to the novelty of VR technology and the accessibility offered by the museum. Improvements were made to the environment based on their feedback, including changes to the terrain and lighting. Additionally, we developed two alternative versions of the experience, one for multi-projection in a room and another without interactive elements. Our results indicate that the VR experience can be successfully integrated with the exhibition and has the potential to enhance museum visits. It can connect the audience with the actual fossils on display by using a dinosaur in the exhibition. It allows one to visualize how their region changed in the past million years. Finally, this experiment helped bridge the population from an unprivileged region with the science that is produced upon the fossils found in their context. Such knowledge broadens the public’s imagination and triggers a whole chain of development for the community around the museum. Our VR exhibition prototype showed great potential to amplify this mission.

Social stress in an interaction with artificial agents in virtual reality: Effects of ostracism and underlying psychopathology

Social stress can emerge from situational as well as dispositional factors. Here we tested in direct juxtaposition how ostracism in a Cyberball game and underlying psychopathology levels both influence subjective and objective stress markers in an interaction with artificial agents in virtual reality (VR) in 80 participants from a student population. Ostracism led to moderately enhanced subjective stress and negative mood but not to alterations on objective markers of stress. By contrast, underlying psychopathology levels were associated with substantially stronger alterations on subjective stress markers and were additionally associated with reduced eye gaze at virtual agents' heads, larger pupil size, larger high-frequency pupil-size variability, higher heart rate and reduced high-frequency heart-rate variability. Effects for social anxiety, general anxiety and depression levels were overall similar with largest effects on objective stress markers linked to general anxiety. These findings contest the suitability of the Cyberball game as a model for real-life ostracism but demonstrate the utility of gaze and physiological data in predicting psychopathology levels during interactions in VR. These findings furthermore highlight the need for data security solutions when using social VR applications where rich data streams are exposed publicly.

Development of an interactive VR flight simulation with ArcGIS SDK for Unity and a hang glider

Development of an interactive VR flight simulation with ArcGIS SDK for Unity and a hang glider

Wearable teleoperation controller with 2-DoF robotic arm and haptic feedback for enhanced interaction in virtual reality.

Teleoperation is an essential component in fields such as medicine and manufacturing, enabling human operators to interact with remote robotic systems. A wearable device has been designed and manufactured to enhance sensitivity, wearability, and synchronization, providing users with the ability to experience the sensation of grasping virtual objects. The device incorporates a 2-DoF robotic arm, haptic sensors for finger gestures, and a Unity-powered virtual scene system. Its effectiveness was evaluated through user experiments, where participants were asked to rank the weights of three virtual balls and identify the direction of force applied to a virtual ball in separate tests. Additionally, the device's ability to render various shapes was also examined. The experiments showed that 73.3% of participants accurately ranked the balls by mass, and an overall correctness rate of 87.3% was achieved for force direction identification. For shape rendering, the device yielded more accurate results for simple objects like spheres, whereas rendering more complex objects such as cups and cones was challenging. The findings indicate that this wearable device has potential applications in haptic feedback and virtual reality contexts. Despite the challenges with complex shape rendering, the device shows promising capability in enhancing the user's immersive experience in virtual environments.

Application of teaching mode based on VR display and interaction

In the context of rapid social and economic development, traditional classroom teaching mode has been unable to meet the needs of educational guidance. In order to continuously expand students' thinking consciousness and imagination ability, and scientifically solve the problems of single teaching method and limited learning content in the past, the education field proposes to realize a teaching system based on VR display and interactive design. Actively cultivate students' ability of independent learning and independent thinking. Therefore, after understanding the teaching system architecture based on VR display and interaction, this paper deeply discusses the practical teaching model and evaluation system according to the application characteristics of the system, so as to clarify the direction and measures for the application of micro-R technology in the future education field.

Analysis of Cartographic Symbols as Visual Support in Interactive VR Geovisualizations

Abstract. Game engines and head-mounted displays (HMD) can create interactive virtual reality (VR) cartographies. Many VR geovisualizations try to depict places as close to reality as possible. The scaling is 1:1 for most VR geovisualizations. If VR geovisualizations are too close to reality or complex, users can navigate through them better, but the amount of detail makes it more challenging to target information. With the help of cartographic elements, a more targeted focus can again be acquired. The focus and style of VR cartography depend on the topic. The topics selected were "Floods and heavy rainfall events", "Noise propagation analysis", and "Lighting simulation". These are three topic examples in which 3D city models are used. CityGML models and fictitious 3D objects were used. Different rendering techniques and point signatures were implemented for each of the themes. All three topics were implemented in an application as a serious game. With the help of an interactive interface and motion controllers, users can change and navigate all three themes. Solution-oriented objects can be placed on all three topics as an additional interactive element. These options allow users to learn targeted information through various presentation options and implement solutions.

Touching the future of training: investigating tangible interaction in virtual reality

Virtual reality offers exciting new opportunities for training. This inspires more and more training fields to move from the real world to virtual reality, but some modalities are lost in this transition. In the real world, participants can physically interact with the training material; virtual reality offers several interaction possibilities, but do these affect the training’s success, and if yes, how? To find out how interaction methods influence the learning outcome, we evaluate the following four methods based on ordnance disposal training for civilians: 1) Real-World, 2) Controller-VR, 3) Free-Hand-VR, and 4) Tangible-VR in a between-subjects experiment (n = 100). We show that the Free-Hand-VR method lacks haptic realism and has the worst training outcome. Training with haptic feedback, e.g., Controller-VR, Tangible-VR, and Real-World, lead to a better overall learning effect and matches the participant’s self-assessment. Overall, the results indicate that free-hand interaction is improved by the extension of a tracked tangible object, but the controller-based interaction is most suitable for VR training.

Changes in Navigation Controls and Field-of-View Modes Affect Cybersickness Severity and Spatiotemporal Gait Patterns After Exposure to Virtual Environments.

To examine the effects of navigation controls and field-of-view modes on cybersickness severity and gait dynamics after cessation of exposure to a virtual environment (VE). The applications of virtual reality are increasing in various fields; however, whether changes in interaction techniques and visual contents could mitigate the potential gait disturbance following VE exposure remains unclear. Thirty healthy adults wore a head-mounted display to complete six sessions of 12-min run-and-gun tasks using different navigation controls (gamepad, head, natural) and field-of-view modes (full, restricted). Forward and backward walking tasks were performed before and after VE exposure. The degrees of cybersickness and presence were evaluated using questionnaires, along with the in-session task performance. Spatiotemporal gait measures and their variabilities were calculated for each walking task. The participants experienced less cybersickness with the head and natural controls than with the gamepad. Natural control, based on matching body movements, was associated with the highest degree of presence and best performance. VE navigation using the gamepad showed reduced cadences and increased stride times during postexposure forward-walking tasks. When the VE was presented via the restricted field-of-view mode, increased gait variabilities were observed from backward-walking tasks after VE exposure. Body movement-based navigation controls may alleviate cybersickness. We observed gait adaptation during both ambulation tasks, which was influenced by the navigation control method and field-of-view mode. This study provides the first evidence for gait adaptation during balance-demanding tasks after VE exposure, which is valuable for designing guidelines for virtual reality interactions.

Using Literary Texts in Developing Intercultural Competence of Foreign English Language Learners in Virtual Space

The current study provides a broad overview of intercultural interaction in virtual reality (VR) settings made feasible by technology innovations, cultural components, and the usage of literary texts. The extensive usage of technology and social media by Generation Z molds their individuality and creates a new cultural realm. This study intends to analyze intercultural communication and language use in virtual spaces. It examines how the literary-based-cyber-culture influences, especially Generation Z. Additionally, the study delineates social media and literary texts usage to develop intercultural communication and how the students’ engagement in VR or digital space shapes their perceptions, attitudes, behavior, identities, and culture. A qualitative descriptive technique as a methodological instrument is used to collect information to compare it to other approaches and conceptualize it from the standpoints of different EFL learners. The pedagogical potential of using literary texts in the digital world offers to develop intercultural discourse, foreign language acquisition, and instruction in the context of technology-integrated educational practices in foreign language education. Finally, the study suggests educational institutions and think tanks adopt an innovative and constructive view to improve global acceptance, tolerance, and communication.

Speaking in front of cartoon avatars: A behavioral and psychophysiological study on how audience design impacts on public speaking anxiety in virtual environments

Public speaking anxiety is defined as a strong difficulty in speaking in front of an audience and has been shown to impair work performance and social relationships. Virtual Reality (VR) offers an efficient tool for modulating public speaking anxiety through a wide range of customizations concerning environmental settings. However, scientific research needs to understand better what features of the simulated environment are more important to increase or reduce participants’ perceived discomfort. The present study investigates the role of visual (human vs cartoon characters) and acoustic (human vs robotic voice) audience features on perceived anxiety, sense of presence, and perceived realism in an interactive VR public speaking scenario. 42 participants (mean age = 24 y.o; Females = 30) performed four public speaking sessions, characterized by different levels (high vs. low) of graphic and acoustic audience design. Both explicit (questionnaires) and implicit physiological measures (Electrodermal activity-EDA) collected during audience interaction were used to assess the participants’ experiences.The results showed that the features of the simulated audience played a crucial role in perceived anxiety during a virtual public speaking. Specifically, the more realistic level of graphic and acoustic stimuli resulted in higher levels of self-reported anxiety as compared to the lower realism level. However, the experienced realism and the sense of presence seem more affected by the graphical than acoustic features of the virtual environment. By contrast, the acoustic features impact on the interaction realism with the virtual audience. Interestingly, the robotic voice (lower acoustic realism) increased electrodermal response during the interaction with the audience, interpreted as a break in the sense of presence. A positive correlation between anxiety, sense of presence, and experienced realism was found. As well, perceived anxiety is correlated with electrodermal activity during the performance. Nevertheless, physiological activity is more affected by the first experience than the realism features, suggesting the presence of a habituation effect across the repeated sessions. Taken together, the results of our study showed that multisensory features (graphical and acoustic) of the virtual environment play a fundamental role in creating realistic public speaking experiences and might be used within gamification strategies for soft skill training (e.g., for improving public speaking anxiety).

Online view enhancement for exploration inside medical volumetric data using virtual reality

Background and Objective:Medical image visualization is an essential tool for conveying anatomical information. Ray-casting-based volume rendering is commonly used for generating visualizations of raw medical images. However, exposing a target area inside the skin often requires manual tuning of transfer functions or segmentation of original images, as preset parameters in volume rendering may not work well for arbitrary scanned data. This process is tedious and unnatural. To address this issue, we propose a volume visualization system that enhances the view inside the skin, enabling flexible exploration of medical volumetric data using virtual reality. Methods:In our proposed system, we design a virtual reality interface that allows users to walk inside the data. We introduce a view-dependent occlusion weakening method based on geodesic distance transform to support this interaction. By combining these methods, we develop a virtual reality system with intuitive interactions, facilitating online view enhancement for medical data exploration and annotation inside the volume. Results:Our rendering results demonstrate that the proposed occlusion weakening method effectively weakens obstacles while preserving the target area. Furthermore, comparative analysis with other alternative solutions highlights the advantages of our method in virtual reality. We conducted user studies to evaluate our system, including area annotation and line drawing tasks. The results showed that our method with enhanced views achieved 47.73% and 35.29% higher accuracy compared to the group with traditional volume rendering. Additionally, subjective feedback from medical experts further supported the effectiveness of the designed interactions in virtual reality. Conclusions:We successfully address the occlusion problems in the exploration of medical volumetric data within a virtual reality environment. Our system allows for flexible integration of scanned medical volumes without requiring extensive manual preprocessing. The results of our user studies demonstrate the feasibility and effectiveness of walk-in interaction for medical data exploration.

Dual-Gain Mode of Head-Gaze Interaction Improves the Efficiency of Object Positioning in a 3D Virtual Environment

Head-gaze interaction is an integral mode of interaction in virtual reality (VR) applications, demonstrating high precision in fine manipulation tasks but low efficiency in large-scale object movements. To enhance the efficiency of head-gaze interaction, this study adjusted the control-display gain to compensate for the weaknesses of head-gaze interaction in a long-distance object-positioning task. We investigated the effect of the control-display gain of head-gaze interaction on movement time (MT) using a cohort of participants (n = 24) to perform experiments. The results showed that the MT first decreased as the gain increased from 1 to 1.5 and then increased afterwards. Further analysis showed that a high gain improved the interaction efficiency in the ballistic phase, but reduced the interaction efficiency in the corrective phase. To be able to obtain higher efficiency of interaction, we designed a dual-gain mode which set different gains in the ballistic and corrective phases. Evaluated using an additional experimental cohort (n = 24), our results showed that the dual-gain mode was more efficient than the mono-gain mode. Moreover, the dual-gain mode with optimal gains did not induce a more serious perception of inconsistency, confusion, nonacceptance and motion sickness, while it had a tendency to reduce the total workload compared to the interaction with normal gain. Our findings provide potential valuable design insights and guidance contributing to improving the efficiency of head-gaze interaction in virtual spaces.

Virtual Reality (VR) - The Effectiveness in Interior Architecture Technical Drafting

Technical drafting knew as drafting in detailed drawing. It is essential for communicating ideas in the industry by using familiar symbols, units of measurement, notation systems, visual styles, and page layouts using Virtual Reality (VR). Interior Architecture students spend most of their time creating those physical models or digital models instead of creating and developing ideas. However, lack of research and knowledge on technical drawing is one of the reasons that hard to understand and explore for the beginner level student. This research aims to develop a new interactive virtual reality application that can be used in beginner-level technical drafting. Therefore, this project uses the ADDIE model to develop the application and collect data at the early and final stages. This study concludes that virtual reality applications for technical drafting can guide and become a source of information for students understanding technical drafting and developing space planning in interior architecture.

Enhancing Virtual and Augmented Reality Interactions with a MediaPipe-Based Hand Gesture Recognition User Interface

Enhancing Virtual and Augmented Reality Interactions with a MediaPipe-Based Hand Gesture Recognition User Interface

Harnessing Artificial Intelligence for Early and Evolution of Alzheimer’s Disease Detections and Enhancing Senior Mental Health through Innovative Art-Singing Therapies: A Multidisciplinary Approach

The well-documented therapeutic potential of group singing for patients living with Alzheimer’s disease (PLAD) has been hindered by COVID-19 restrictions, exacerbating loneliness and cognitive decline among seniors in residential and long-term care centers (CHSLDs). Addressing this challenge, the multidisciplinary study aims to develop a patient-oriented virtual reality (XR) interaction system facilitating group singing for mental health support during confinement and enhancing the understanding of the links between Alzheimer’s disease, social interaction, and singing. The researchers also propose to establish an early AD detection system using voice, facial, and non-invasive biometric measurements and validate the efficacy of selected intervention practices. The methodology involves co-designing an intelligent environment with caregivers to support PLAD mental health through online group singing, addressing existing constraints in CHSLDs. The researchers will engage volunteers in remote singing interactions and validate the impact of voice stimulation for PLADs using a control group. The primary expected outcome is the development of an “Intelligent Learning Health Environment,” fostering interactions while adapting to individual PLAD situations and incrementally accumulating knowledge on AD signs. This environment will facilitate the transfer of knowledge and technologies to promote non-verbal interactions via singing, enabling intervention at the first symptoms. Additionally, the research will contribute to transforming CHSLDs’ living environments, informed by neuroscience insights, and potentially extend the “collaborative self-care” approach to support seniors in aging safely and healthily at home.

3D Interactive Virtual Reality Media to Improve Learning Outcomes in Thematic Subjects

At the elementary level of school, learning activities are encouraged to prioritise student participation and interaction in the learning process. This is done in order to maximise the overall educational experience. Utilising different kinds of educational media that are based on technology is one of them. It is with media that is based on virtual reality (VR), which gives users the ability to interact with goods in a virtual setting that not only appears real but also has the sensation of being real. This study aims to develop 3D Virtual Reality Interactive Media to improve student learning outcomes in elementary schools. A quasi-experimental approach was utilised for the methodology of this research, and pre- and post-test control groups were included. The learning achievement test is the tool that is being utilised in this research. Students in fourth grade from elementary schools participated in the research project. The following are some of the findings that emerged from this research: (1) the development of a model for 3D virtual interactive multimedia; (2) the implementation of learning through 3D virtual reality interactive media utilising a systems perspective; (3) Varieties of learning assessment (assessment) that make use of authentic assessment; (4) Data demonstrating a good response from students towards the implementation of 3D virtual reality interactive media. It has been demonstrated that the implementation of interactive media in the form of 3D virtual reality has been successful in enhancing the learning outcomes of students in Thematic Subjects, particularly Science in Elementary Schools.

Social presence as a training resource: comparing VR and traditional training simulations

From immersive simulations to interactive tutorials, Virtual Reality (VR) is transforming the way we learn and practise new skills. Especially for social skills training, a growing number of simulations have been designed in which trainees learn to master difficult communicative situations. One of the factors to which the effectiveness of VR as a learning technology is attributed to is the users’ feeling of social presence during the simulated interaction. This paper presents the evaluation of (1) a role play training, (2) a learning app and (3) a VR training application in a workshop series. Social presence was perceived as equally convincing and engaging for the prototypical VR scene as for the traditional form of role play, although the course of the interaction in VR was highly determined compared to the interaction dynamics of a human role play. In our interpretation, this confirms social presence as a valuable resource for training social interaction, which spans across various learning settings and methods in increasingly blended or hybrid learning and working contexts.

Crafting Attention in Immersive and Interactive Practices

This creation is part of the ongoing project “Positioning the Audience in Cinematic Virtual Reality”, supported by the Scientific and Technological Research Council of Turkey. The project engages with the attentional and corporeal resonance between spectators and visual experiences in participatory and immersive practices. In the continuing first phase of the project, we are designing and producing a VR creation that will then enable us to conduct audience research to focus on the attention practices of the viewers in the second phase. Hence, as a critical research-creation project it has two complementary and symmetrical foci that employ both theory and practice, and that are intertwined with one another. In tandem with making an interactive and photorealistic (cinematic) VR film where spectator crafts the very narrative unfolding by voluntary and involuntary attentional participation in a given scene, we are also developing a conceptual framework on how immersive and interactive technologies are conditioned by the freedom given to the spectator. As interdisciplinary project it brings together researchers from film studies, new media studies and psychology to develop a holistic understanding on the attention of spectator in moving image technologies. In making, instead of using game-based interactivity technologies enabled by the controllers, the VR film uses gaze detection techniques and the attentional economy of the spectator as active defining element of the narrative unfolding. Combining both the bottom-up and the top-down attentional procedures it allows spectators to craft a circular experience and aims to problematize the craft of the spectator. In theory it aims to develop an historical and critical understanding of visual practices where spectator is the active producer of the optical experience in certain media ecology including technologies from proto-cinematic devices to the post-cinematic media constellation comprising video games, Virtual Reality etc. Therefore we aim to problematize and rend sensible the craft and the autonomy of the viewer by combining conventional academic research practices with artistic experimentation. The creation undertaken in the first phase of the research is aimed to result in multiple alternative designs, which will form the basis of our presentation at IFM2023.