Interactive Virtual Reality Research Articles

Bridging theory and practice: CFD simulation and interactive VR for conduction heat transfer learning

AbstractSoftware simulation programs and virtual reality (VR) have become powerful tools for several educational purposes, and recently, they were used in a wide range of applications. In cases of inaccessibility to labs, workshops, or industries, as happened before in the coronavirus disease 2019 pandemic, these tools could be effectively integrated with practical lab experiments. In this study, a computational fluid dynamics (CFD) simulation and a VR module were utilized to simulate heat transfer by conduction through various experimental cases. The CFD simulation results were obtained using ANSYS/FLUENT software. Meantime, the experimental data were obtained by carrying out three experiments of heat conduction with different heat transfer rates through simple, composite, and different cross‐sectional area bars. At last, the experimental procedure and devices used were virtually constructed using SolidWorks software as three‐dimensional models, which were then extruded into VR and augmented reality models. It was found that the simulation results closely align with the experimental ones, and the temperature profile in both cases has the same behavior with small differences, which indicates the validity of the developed module to be used as a simulation of the actual experiments. In addition to improving knowledge of heat transfer principles, this combination of simulation and VR technology advances many Sustainable Development Goals (SDGs), including advancing quality education (SDG 4) and innovation in higher education (SDG 9). Additionally, this method assists in achieving the course learning objectives by mimicking real‐world lab experiments, guaranteeing that students graduate from the course with the required information and abilities.

, virtual reality embodying philosophical and conceptual heritage

ABSTRACT In this paper, we introduce the <Sunghaksipdo VR> project, which is based on Sunghaksipdo (聖學十圖). Known as the Ten Diagrams on Sage Learning, Sunghaksipdo was a heritage created by Toegye, the Korean Confucian scholar of the Joseon Dynasty. While conveying philosophical concepts such as Li (理) and Gyung (敬) and their spiritual values of mindfulness from the heritage to VR, we have focused on reconnecting the original visual diagrams and texts into perceivable VR experiences. We approached this task through the following: (1) by constructing conceptual and philosophical VR space and environments based on the original diagrams, (2) by creating conceptual objects inside VR, and (3) by designing diverse user interactions with such virtual conceptual objects. Particularly the interactions allow the user’s embodied experience and performative engagement, which become a way of understanding the meaning of the philosophical message. While explaining these with diverse exemplary cases, we hope to show how new media and interactive VR technologies can be used to realize philosophical concepts and spiritual value. Through this, we hope to discuss how VR could be imagined as a tangible philosophy discovering the plane of immanence.

The Aural Eye

Human aural experience can be equally considered along spatial and temporal continuums. We hear at all times of the day and night, and within all places and spaces: built, natural, public, private and virtual. The territory – both physical and philosophical – between music composition for interior listening and traditional sound based research within environmental acoustics is gradually being occupied by the listener-centred approach of soundscape studies. Soundscape design is emerging as an interdisciplinary field within design education, and one that not only challenges the ocular-centric nature of most design education, but one that could provide a useful mode through which to investigate the coincidences between different design disciplines. This paper draws on the author’s own practice as a sound designer in a variety of spatial sound projects in built and virtual contexts to discuss ideas of landscape, interiority, space and place as experienced through listening. This will include aspects of Canopies: chimerical acoustic environments for the Southgate soundscape system, Ecstasis: human presence in digital environments for an interactive VR system and stereoscopic projections, and The Occupation of Space: Soundsites project with the Melbourne blind community.2 The ideas and technologies underpinning these projects also form the basis of a new pedagogy of sound and listening housed in the Spatial Information Architecture Laboratory’s (SIAL) Sound Studios at RMIT University. The place and role of the Sound Studio’s program in providing an aural perspective that compliments the visual methodologies of co-located design disciplines is discussed.

Augmenting collaborative interaction with shared visualization of eye movement and gesture in VR

AbstractVirtual Reality (VR)‐enabled multi‐user collaboration has been gradually applied in academic research and industrial applications, but it still has key problems. First, it is often difficult for users to select or manipulate objects in complex three‐dimesnional spaces, which greatly affects their operational efficiency. Second, supporting natural communication cues is crucial for cooperation in VR, especially in collaborative tasks, where ambiguous verbal communication cannot effectively assign partners the task of selecting or manipulating objects. To address the above issues, in this paper, we propose a new interaction method, Eye‐Gesture Combination Interaction in VR, to enhance the execution of collaborative tasks by sharing the visualization of eye movement and gesture data among partners. We conducted user experiments and showed that using dots to represent eye gaze and virtual hands to represent gestures can help users complete tasks faster than other visualization methods. Finally, we developed a VR multi‐user collaborative assembly system. The results of the user study show that sharing gaze points and gestures among users can significantly improve the productivity of collaborating users. Our work can effectively improve the efficiency of multi‐user collaborative systems in VR and provide new design guidelines for collaborative systems in VR.

Design of a lightweight and easy‐to‐wear hand glove with multi‐modal tactile perception for digital human

AbstractWithin the field of human–computer interaction, data gloves play an essential role in establishing a connection between virtual and physical environments for the realization of digital human. To enhance the credibility of human‐virtual hand interactions, we aim to develop a system incorporating a data glove‐embedded technology. Our proposed system collects a wide range of information (temperature, bending, and pressure of fingers) that arise during natural interactions and afterwards reproduce them within the virtual environment. Furthermore, we implement a novel traversal polling technique to facilitate the streamlined aggregation of multi‐channel sensors. This mitigates the hardware complexity of the embedded system. The experimental results indicate that the data glove demonstrates a high degree of precision in acquiring real‐time hand interaction information, as well as effectively displaying hand posture in real‐time using Unity3D. The data glove's lightweight and compact design facilitates its versatile utilization in virtual reality interactions.

PetPresence: Investigating the Integration of Real-World Pet Activities in Virtual Reality.

For VR interaction, the home environment with complicated spatial setup and dynamics may hinder the VR user experience. In particular, pets' movement may be more unpredictable. In this paper, we investigate the integration of real-world pet activities into immersive VR interaction. Our pilot study showed that the active pet movements, especially dogs, could negatively impact users' performance and experience in immersive VR. We proposed three different types of pet integration, namely semitransparent real-world portal, non-interactive object in VR, and interactive object in VR. We conducted the user study with 16 pet owners and their pets. The results showed that compared to the baseline condition without any pet-integration technique, the approach of integrating the pet as interactive objects in VR yielded significantly higher participant ratings in perceived realism, joy, multisensory engagement, and connection with their pets in VR.

Privacy-Preserving Gaze Data Streaming in Immersive Interactive Virtual Reality: Robustness and User Experience.

Eye tracking is routinely being incorporated into virtual reality (VR) systems. Prior research has shown that eye tracking data, if exposed, can be used for re-identification attacks [14]. The state of our knowledge about currently existing privacy mechanisms is limited to privacy-utility trade-off curves based on data-centric metrics of utility, such as prediction error, and black-box threat models. We propose that for interactive VR applications, it is essential to consider user-centric notions of utility and a variety of threat models. We develop a methodology to evaluate real-time privacy mechanisms for interactive VR applications that incorporate subjective user experience and task performance metrics. We evaluate selected privacy mechanisms using this methodology and find that re-identification accuracy can be decreased to as low as 14% while maintaining a high usability score and reasonable task performance. Finally, we elucidate three threat scenarios (black-box, black-box with exemplars, and white-box) and assess how well the different privacy mechanisms hold up to these adversarial scenarios. This work advances the state of the art in VR privacy by providing a methodology for end-to-end assessment of the risk of re-identification attacks and potential mitigating solutions. f.

Enhancing surgical training through immersive and interactive virtual reality experiences in robotic surgery

Enhancing surgical training through immersive and interactive virtual reality experiences in robotic surgery

Multimodal Physiological Analysis of Impact of Emotion on Cognitive Control in VR.

Cognitive control is often perplexing to elucidate and can be easily influenced by emotions. Understanding the individual cognitive control level is crucial for enhancing VR interaction and designing adaptive and self-correcting VR/AR applications. Emotions can reallocate processing resources and influence cognitive control performance. However, current research has primarily emphasized the impact of emotional valence on cognitive control tasks, neglecting emotional arousal. In this study, we comprehensively investigate the influence of emotions on cognitive control based on the arousal-valence model. A total of 26 participants are recruited, inducing emotions through VR videos with high ecological validity and then performing related cognitive control tasks. Leveraging physiological data including EEG, HRV, and EDA, we employ classification techniques such as SVM, KNN, and deep learning to categorize cognitive control levels. The experiment results demonstrate that high-arousal emotions significantly enhance users' cognitive control abilities. Utilizing complementary information among multi-modal physiological signal features, we achieve an accuracy of 84.52% in distinguishing between high and low cognitive control. Additionally, time-frequency analysis results confirm the existence of neural patterns related to cognitive control, contributing to a better understanding of the neural mechanisms underlying cognitive control in VR. Our research indicates that physiological signals measured from both the central and autonomic nervous systems can be employed for cognitive control classification, paving the way for novel approaches to improve VR/AR interactions.

Assessing Learning in an Immersive Virtual Reality: A Curriculum-Based Experiment in Chemistry Education

Despite the recent advances in Virtual Reality technology and its use in education, the review of the literature shows several gaps in research on how immersive virtual environments impact the learning process. In particular, the lack of curriculum-specific experiments along with investigations of the effects of different content, activity, and interaction types in the current VR studies has been identified as a significant shortcoming. This has been more significant in STEM fields, where VR has the potential to offer engaging experiential learning opportunities. The study reported here was designed to address this gap by assessing the effect of authentic visualization and interaction types on learning a particular scientific concept. A use case scenario of “orbital hybridization” in chemistry education was selected to create this experiment and to collect data for analysis. We collected data on learning outcomes, task-completion efficiency, accuracy, and subjective usability. A combination of learning content and tasks designed based on the relevant educational theories was presented to three groups: 2D, VR interaction type 1 (hand gestures), and VR interaction type 2 (ray casting). The results showed that VR could improve learning and that interaction type could influence efficiency and accuracy depending on the task.

Impact of interactive learning elements on personal learning performance in immersive virtual reality for construction safety training

To improve construction safety through proactive prevention strategies, it is essential to leverage virtual reality (VR) training programs that foster active learning through dynamic interaction between VR systems and learners. Addressing this need, this study proposed an interactive immersive VR (IVR)-based construction safety training framework, incorporating four interactive learning elements (ILEs): immediate feedback, basic interaction with objects, assembling objects, and knowledge testing. This study divided sixty trainees into two groups: one experienced the proposed novel interactive IVR-based training approach, while the other underwent traditional, non-interactive IVR-based training. To accurately evaluate the impact of the four ILEs on individual learning performance, this study conducted t-test analyses and utilized machine learning-based SHAP (Shapley Additive exPlanations) analysis to compare the results between both groups. The findings indicated that the proposed method significantly enhanced active learning, positively influencing the sensory and knowledge domains of trainees more effectively than the existing methods. Notably, 'ILE-(1). Immediate feedback' and 'ILE-(2). Basic interaction with objects' were identified as key factors in improving personal learning outcomes during training. This study contributes significantly to the field of management in engineering, demonstrating that interactive IVR-based training can offer a superior learning experience and enhance safety performance, thereby reducing accident rates and creating safer construction sites.

Immersive Storytelling in Social Virtual Reality for Human-Centered Learning about Sensitive Historical Events

History is a subject that students often find uninspiring in school education. This paper explores the application of social VR metaverse platforms in combination with interactive, nonlinear web platforms designed for immersive storytelling to support learning about a sensitive historical event, namely the Asia Minor Catastrophe. The goal was to design an alternative method of learning history and investigate if it would engage students and foster their independence. A mixed-methods research design was applied. Thirty-four (n = 34) adult participants engaged in the interactive book and VR space over the course of three weeks. After an online workshop, feedback was collected from participants through a custom questionnaire. The quantitative data from the questionnaire were analyzed statistically utilizing IBM SPSS, while the qualitative responses were coded thematically. This study reveals that these two tools can enhance historical education by increasing student engagement, interaction, and understanding. Participants appreciated the immersive and participatory nature of the material. This study concludes that these technologies have the potential to enhance history education by promoting active participation and engagement.

SlowR50-SA: A Self-Attention Enhanced Dynamic Facial Expression Recognition Model for Tactile Internet Applications

Emotion recognition from facial expressions is a challenging task due to the subtle and nuanced nature of facial expressions. Within the framework of Tactile Internet (TI), the integration of this technology has the capacity to completely transform real-time user interactions, by delivering customized emotional input. The influence of this technology is far-reaching, as it may be used in immersive virtual reality interactions and remote tele-care applications to identify emotional states in patients. In this paper, a novel emotion recognition algorithm is presented that integrates a Self-Attention (SA) module into the SlowR50 backbone (SlowR50-SA). The experiments on the DFEW and FERV39K datasets demonstrate that the proposed model achieves good performance in terms of both Unweighted Average Recall (UAR) and Weighted Average Recall (WAR) metrics, achieving a UAR (WAR) of 57.09% (69.87%) on the DFEW dataset, and UAR (WAR) of 39.48% (49.34%) on the FERV39K dataset. Notably, SlowR50-SA operates with only eight frames of input at low temporal resolution, highlighting its efficiency. Furthermore, the algorithm has the potential to be integrated into Tactile Internet applications, where it can be used to enhance the user experience by providing real-time emotion feedback. SlowR50-SA can also be used to enhance virtual reality experiences by providing personalized haptic feedback based on the user’s emotional state. It can also be used in remote tele-care applications to detect signs of stress, anxiety, or depression in patients.

Decoding Realism of Virtual Objects: Exploring Behavioral and Ocular Reactions to Inaccurate Interaction Feedback

Achieving temporal synchrony between sensory modalities is crucial for natural perception of object interaction in virtual reality. While subjective questionnaires are currently used to evaluate users’ VR experiences, leveraging behavior and psychophysiological responses can provide additional insights. We investigated motion and ocular behavior as discriminators between realistic and unrealistic object interactions. Participants grasped and placed a virtual object while experiencing sensory feedback that either matched their expectations or occurred too early. We also explored visual-only feedback vs. combined visual and haptic feedback. Due to technological limitations, a condition with delayed feedback was added post-hoc. Gaze-based metrics revealed discrimination between high and low feedback realism. Increased interaction uncertainty was associated with longer fixations on the avatar hand and temporal shifts in the gaze-action relationship. Our findings enable real-time evaluation of users’ perception of realism in interactions. They facilitate the optimization of interaction realism in virtual environments and beyond.

Evaluations of Virtual and Augmented Reality Technology-Enhanced Learning for Higher Education

Virtual reality (VR) has good potential to promote technology-enhanced learning. Students can benefit from immersive visualization and intuitive interaction in their learning of abstract concepts, complex structures, and dynamic processes. This paper is interested in evaluating the effects of VR learning games in a Virtual and Augmented Reality Technology-Enhanced Learning (VARTeL) environment within an engineering education setting. A VARTeL flipped classroom is established in the HIVE learning hub at Nanyang Technological University (NTU) Singapore for the immersive and interactive learning. Experiments are designed for the university students conducting the learning, with three interactive and immersive VR games related to science, technology, engineering and mathematics (STEM), i.e., virtual cells, a virtual F1 racing car, and vector geometry. These VR games are a part of the VARTeL apps designed in-house at NTU for STEM education. Quantitative and qualitative analyses are performed. A total of 156 students from Mechanical Engineering participated in the experiment. There are 15 participants selected for an interview after the experiment. Pre-tests and post-tests are performed using two different models, the developed VARTeL and the modified Technology-Rich Outcome-Focused Learning Environment Inventory (TROFLEI), in order to measure the efficiency of the VARTeL environment in Higher Education. Significant improvements of about 24.8% are observed for the post-tests over the pre-tests, which illustrate the effectiveness of the VARTeL for Engineering education. Details of the VR simulation games, methods of data collection, data analyses, as well as the experiment results are discussed. It is observed from the results that all the underlying scales of the modified TROFLEI are above the threshold for the ‘Good’ category, indicating that a very reliable questionnaire is designed in this research. The mean ‘Ideal’ values are about 0.7–2.6% higher than the mean ‘Actual’ values. The limitations of the experiment and future works with recommendations are also presented in this paper.

Immersive presentations of real-world medical equipment through interactive VR environment populated with the high-fidelity 3D model of mobile MRI unit

The primary goal behind the system presented in this paper is to investigate the efficacy of using virtual reality (VR) for showcasing sizable medical equipment. Specifically, we focused on a mobile magnetic resonance imaging (MRI) scanner mounted on a truck trailer. The latter is integral to the mobile MRI setup and must be presented as part of the immersive experience. Therefore, we not only have to depict the medical apparatus but also provide the means of understanding its surroundings. This is especially important to radiologists and other medical personnel to ascertain if a given mobile medical facility fulfills their needs and wants. Furthermore, despite such MRI devices being designed for mobility, their long-distance transportation can be time-consuming, troublesome and expensive. Therefore, we can observe the need for showcasing such mobile MRI units without additional cost and burden related to transportation. To achieve this, we designed an immersive environment in which the users can interact with the real-life scale 3D model of a mobile MRI. In addition, we also verified the usability and expressiveness of our system using established heuristical approaches.

Revolutionizing construction safety: introducing a cutting-edge virtual reality interactive system for training US construction workers to mitigate fall hazards

The construction industry is fraught with hazards, chief among them being the risk of falls from elevated positions, which are a leading cause of both fatalities and injuries among workers. Despite the prevalence of traditional safety training methods, their effectiveness in reducing fall risks remains limited. To solve this issue, this study proposes the adoption of interactive VR technologies to provide construction workers with immersive training experiences in the critical domain of fall safety. This approach not only ensures adherence to the Occupational Safety and Health Administration (OSHA) requirements but also leverages VR’s immersive capabilities to create a comprehensive and effective learning tool. To evaluate the proposed system’s efficacy, the researchers conducted an empirical assessment involving eighty-two construction workers from two small enterprises. Participants were divided into two groups: one receiving traditional training and the other undergoing VR-based instruction. Both groups underwent pre- and post-training evaluations comprising six targeted questions designed to measure the impact of each training method on their understanding and awareness of fall safety practices. The comparative analysis revealed no significant differences in baseline knowledge between the two groups prior to the training interventions. However, post-training evaluations demonstrated a notable improvement in the VR group, with a significant decrease in the number of incorrect responses, in stark contrast to the group subjected to traditional training methods. Statistical analysis further confirmed the superiority of VR training in enhancing participants’ knowledge. This was quantified by a p-value of 0.0016, indicating a high level of statistical significance well below the conventional threshold of 0.05. This study highlights the significant advantages of VR technology in construction safety training, demonstrating its superiority over traditional training methods in terms of knowledge retention and practical application. The results strongly support the wider adoption of VR in safety training, indicating its potential to enhance safety outcomes in the construction sector.

Action capture and VR interactive system for online experimental teaching

The development of technology has intensified people’s interest in virtual reality technology. Virtual reality technology is also widely used in fields such as teaching and healthcare. To improve the real-time and operability of virtual reality interaction systems, a motion capture and virtual reality interaction system design for online experimental teaching is proposed. The active apparent model is used for facial feature point localization, and the joint capture method is improved by combining threshold segmentation and forward kinematics algorithms. Experimental data confirms that compared to Kinect method, the improved hand joint detection method has higher joint positioning accuracy, with a loss rate of less than 40 % for hand joints. The average judgment accuracy of the six facial expression tests is 82 %, 81 %, 79 %, 78 %, 80 %, and 81 %, respectively. The comprehensive recognition accuracy of facial expression recognition is 80.17 %. The read, write, and update times for each frame of the virtual reality interaction system are 102.6 ms and 427 ms, respectively. The system memory usage is 920.5 M, and the CPU and GPU usage rates are 25.2 % and 4.4 %, respectively. The system can run smoothly. The virtual reality system can run continuously for 24 h, with strong operability and low difficulty in expanding system functions, achieving design goals.

A Wearable Haptic Device for the Hand With Interchangeable End-Effectors.

This article presents a 4-degrees-of-freedom (4-DoF) hand wearable haptic device for Virtual Reality (VR). It is designed to support different end-effectors, that can be easily exchanged so as to provide a wide range of haptic sensations. The device is composed of a static upper body, secured to the back of the hand, and the (changeable) end-effector, placed in contact with the palm. The two parts of the device are connected by two articulated arms, actuated by four servo motors housed on the upper body and along the arms. The article summarizes the design and kinematics of the wearable haptic device and presents a position control scheme able to actuate a broad range of end-effectors. As a proof of concept, we present and evaluate three representative end-effectors during interactions in VR, rendering the sensation of interacting (E1) with rigid slanted surfaces and sharp edges having different orientations, (E2) with curved surfaces having different curvatures, and (E3) with soft surfaces having different stiffness characteristics. A few additional end-effector designs are discussed. A human-subjects evaluation in immersive VR shows the broad applicability of the device, able to render rich interactions with a diverse set of virtual objects.

Perceived Realism of Virtual Textures Rendered by a Vibrotactile Wearable Ring Display.

Wearable haptic displays that relocate feedback away from the fingertip provide a much-needed sense of touch to interactions in virtual reality, while also leaving the fingertip free from occlusion for augmented reality tasks. However, the impact of relocation on perceptual sensitivity to dynamic changes in actuation during active movement remains unclear. In this work, we investigate the perceived realism of virtual textures rendered via vibrations relocated to the base of the index finger and compare three different methods of modulating vibrations with active finger speed. For the first two methods, changing finger speed induced proportional changes in either frequency or amplitude of vibration, and for the third method did not modulate vibration. In psychophysical experiments, participants compared different types of modulation to each other, as well as to real 3D-printed textured surfaces. Results suggest that frequency modulation results in more realistic sensations for coarser textures, whereas participants were less discerning of modulation type for finer textures. Additionally, we presented virtual textures either fully virtually in midair or under augmented reality in which the finger contacted a flat surface; while we found no difference in experimental performance, participants were divided by a strong preference for either the contact or non-contact condition.