Interactive Virtual Reality Research Articles

End-user development for interactive web-based virtual reality scenes

Recent advances in Virtual Reality (VR) technology and the increased availability of VR-equipped devices enable a wide range of consumer-oriented applications. For novice developers, however, creating interactive scenes for VR applications is a complex and cumbersome task that requires high technical knowledge which is often missing. This hinders the potential of enabling novices to create, modify, and execute their own interactive VR scenes. Although recent authoring tools for interactive VR scenes are promising, most of them focus on experts as the target group and neglect the novices with low programming knowledge. To lower the entry barrier, we provide an open-source web-based End-User Development (EUD) tool, called VREUD, that supports the rapid construction and execution of interactive VR scenes. To address the aspect of construction, VREUD enables the specification of the VR scene including interactions and tasks. Furthermore, our tool supports the execution and immersive experience of the created interactive VR scenes on VR head-mounted displays. Based on a user study, we have analyzed the effectiveness, efficiency, and user satisfaction of VREUD which shows promising results to empower novices in creating their interactive VR scenes.

Manufacturing service network characteristics analysis of digital twin system under edge computing and cloud edge collaborative computing environment

In order to optimise the configuration of manufacturing services in the virtual and real space and improve the efficiency of the collaboration of manufacturing service resources, we investigate the digital twin systems that provide various services under different service computing modes, analyse their basic network structure from the perspective of complex networks, and reveal internal interaction methods and feedback control process to optimise virtual and real resource allocation and improve physical system stability. On this basis, the characteristics of the manufacturing service network in the digital twin system are further analysed and explained, and the mechanism of manufacturing service collaboration in the virtual-real interaction fusion mode is described.

Evaluation of Hands-Free VR Interaction Methods During a Fitts’ Task: Efficiency and Effectiveness

Evaluation of Hands-Free VR Interaction Methods During a Fitts’ Task: Efficiency and Effectiveness

Comparison Analysis of Usability Using Controllers and Hand Tracking in Virtual Reality Gamelan (Sharon) Based On User Experience

Gamelan is one of the traditional musical instruments in Indonesia. Gamelan is also one of the cultures that must be preserved. To preserve gamelan, one way is by combining gamelan with technology that is currently developing. The technology that is currently trending is virtual reality. Virtual reality is often used for games in three-dimensional form. Gamelan can be something interesting in VR. In VR interactions, there are two interactions that are often used, namely controllers and hand tracking. Hand gestures are usually tracked using the built-in camera of the VR. Hand tracking can be used in VR applications to control the virtual object just like a controllers. This study aims to compare the use of controllers and hand tracking that involve interactions, namely collision, pressing, grabbing, and release. Thirty two Atma Jaya Yogyakarta University students who had tried virtual reality and played gamelan or had seen gamelan performances explore the differences between VR hand tracking and VR controllers in VR gamelan sharon. SUS (System Usability Scale) and USEQ (Use Questionnaire) were used to measure user usability and satisfaction. The result of the SUS score showed that users were more interested in using controllers than hand tracking. But the USEQ results showed that there were no difference in usefulness, ease of use, and satisfication. Based on the results of the interaction, using controllers looked more comfortable, but using hand tracking, playing gamelan looked more realistic.

Interactive Storytelling for Immersive Media, Augmented Manufacturing, and Digital Healthcare

The article explores interactive storytelling in augmented and virtual reality. It is based on the research project Immersive Media Lab, bridges the gap between technological developments and their application, and aims to combine art, economy, and science. Due to their increased availability, virtual and immersive technologies have not only seen a resurgence in their popularity but also offer intriguing new perspectives regarding their use in different contexts. Consequently, the notions of interactivity and immersion also play significant roles in the study of storytelling in conjunction with virtual and augmented reality. The Immersive Media Lab has employed state-of-the-art AR/VR technology in five use cases: VR Interaction, Artistic Motion Tracking in AR/VR, Audio AR for Industry, AR/VR Interfaces for Industry, and AR Education for Patients. All of them, but one, developed prototypical applications related to the central concept of interactive storytelling. Furthermore, each use case addresses three distinct dimensions of storytelling: the production of a coherent story, the collection and transfer of knowledge, and the specificity of the technology used. Covering the areas of Creative Media, Smart Manufacturing, and Healthcare leads to a particular research design, as each area is connected to different research contexts and comes with appliances of immersive media. The article focuses on various theoretical inputs regarding interactive and immersive storytelling in general and relates them to the academic background of the five use cases. Furthermore, each use case defined a story in its context, the purpose of its narration, and its media-specific context. Finally, the paper reflects on the outcomes of the use cases and outlines their potential for future applications of interactive storytelling in immersive media.

Assessment of user-interaction strategies for neurosurgical data navigation and annotation in virtual reality

While virtual-reality (VR) has shown great promise in radiological tasks, effective user-interaction strategies that can improve efficiency and ergonomics are still under-explored and systematic evaluations of VR interaction techniques in the context of complex anatomical models are rare. Therefore, our study aims to identify the most effective interaction techniques for two common neurosurgical planning tasks in VR (point annotation and note-taking) from the state-of-the-arts, and propose a novel technique for efficient sub-volume selection necessary in neuroanatomical navigation. We assessed seven user-interaction methods with multiple input modalities (gaze, head motion, controller, and voice) for point placement and note-taking in the context of annotating brain aneurysms for cerebrovascular surgery. Furthermore, we proposed and evaluated a novel technique, called magnified selection diorama (Maserama) for easy navigation and selection of complex 3D anatomies in VR. Both quantitative and semi-quantitative (i.e., NASA Task Load Index) metrics were employed through user studies to reveal the performance of each interaction scheme in terms of accuracy, efficiency, and usability. Our evaluations demonstrated that controller-based interaction is preferred over eye-tracking-based methods for point placement while voice recording and virtual keyboard typing are better than freehand writing for note-taking. Furthermore, our new Maserama sub-volume selection technique was proven to be highly efficient and easy-to-use. Our study is the first to provide a systematic assessment of existing and new VR interaction schemes for neurosurgical data navigation and annotation. It offers valuable insights and tools to guide the design of future VR systems for radiological and surgical applications.

Data Glove with Self-Compensation Mechanism Based on High-Sensitive Elastic Fiber-Optic Sensor

With the development of virtual reality (VR) interaction technology, data glove has become one of the most popular devices for human-computer interaction. It's valuable to design high-sensitive and flexible sensor for data glove. Therefore, a low-cost data glove based on self-compensating elastic optical fiber sensor with self-calibration function is proposed. The tunable and stretchable elastic fiber was fabricated by a simple, economical and controllable method. The fiber has good flexibility and high stability under stretching, bending and indentation deformation. The optical fibers are installed in the sensor in a U shape with a bending radius of 5 mm. Compared with the straight fiber, the response sensitivity of the U-shaped fiber to deformation is increased by about 7 times at most. The reference optical fiber is connected to the sensor, which effectively improves the stability and accuracy of the sensor system. In addition, the sensors are easy to install so that the data gloves can be customized for different hand shapes. In the gesture capture test, it can respond quickly and guide the manipulator to track the gesture. This responsive and stable data glove has broad development potential in motion monitoring, telemedicine and human-computer interaction.

A Preliminary Field Work on Digital Heritage and the Use of Virtual Reality for Cultural Heritage Management

This paper presents the results of a multidisciplinary research project carried on during the Covid-19 Pandemic and supported by TÜBİTAK (The Scientific and Technological Research Council of Turkey). The results are gathered from six months of field and office work, as the project was limited with this period. The archaeological site of Letoon in Muğla/Turkey has been chosen as the test area, specifically the triple temples of Leto, Apollo, and Artemis. Photogrammetric reconstruction of the current situation, as well as archaeologically accurate 3D models, have been created and converted into interactive immersive VR content to measure consumer behaviour and experience. These two different types of 3D content are integrated into the VR environment both separately and as a single content with switching from one to the other. After the creation process, the content was experienced by the visitors with different demographic characteristics and a survey was conducted to measure this experience.

Introducing and Familiarising Older Adults Living with Dementia and Their Caregivers to Virtual Reality.

Virtual Reality (VR) is increasingly being applied in dementia care across a range of applications and domains including health and wellbeing. Despite the commercial availability of VR, informants of design are not always aware of its functionality and capabilities, to meaningfully contribute to VR design. In designing VR applications for people living with dementia, it is recommended that older adults living with dementia and their support persons be involved in the design process using participatory approaches, thereby giving them a voice on the design of technology from the outset. A VR technology probe is a useful means of familiarising older adults living with dementia and their informal caregivers with the knowledge and understanding of interactive VR to employ technology that supports them to maintain their social health. This paper charts the implementation and evaluation of a VR technology probe, VR FOUNDations. To explore their experiences, nine older adults living with dementia and their nine informal caregivers trialled VR FOUNDations and completed semi-structured interviews after its use. Overall, older adults living with dementia and their informal caregivers perceived VR FOUNDations to achieve its aim of increasing understanding and inspiring future design decisions. The findings also identified promising positive experiences using a VR technology probe which may be indicative of its applicability to social health and wellbeing domains. This paper advocates for the structured design and implementation of VR technology probes as a pre-requisite to the participatory design of VR applications for the health and wellbeing of people living with dementia. The use of such technology probes may afford older adults living with dementia and their informal caregivers the best opportunity to contribute to design decisions and participate in technology design to support their health and wellbeing.

Expressiveness of real-time motion captured avatars influences perceived animation realism and perceived quality of social interaction in virtual reality

Using motion capture to enhance the realism of social interaction in virtual reality (VR) is growing in popularity. However, the impact of different levels of avatar expressiveness on the user experience is not well understood. In the present study we manipulated levels of face and body expressiveness of avatars while investigating participant perceptions of animation realism and interaction quality when disclosing positive and negative experiences in VR. Moderate positive associations were observed between perceptions of animation realism and interaction quality. Post-experiment questions revealed that many of our participants (approximately 40%) indicated the avatar with the highest face and body expressiveness as having the most realistic face and body expressions. The same proportion also indicated the avatar with the highest face and body expressiveness as being the most comforting and enjoyable avatar to interact with. Our results suggest that higher levels of face and body expressiveness are important for enhancing perceptions of realism and interaction quality within a social interaction in VR using motion capture.

Context-Consistent Generation of Indoor Virtual Environments Based on Geometry Constraints.

In this article, we propose a system that can automatically generate immersive and interactive virtual reality (VR) scenes by taking real-world geometric constraints into account. Our system can not only help users avoid real-world obstacles in virtual reality experiences, but also provide context-consistent contents to preserve their sense of presence. To do so, our system first identifies the positions and bounding boxes of scene objects as well as a set of interactive planes from 3D scans. Then context-consistent virtual objects that have similar geometric properties to the real ones can be automatically selected and placed into the virtual scene, based on learned object association relations and layout patterns from large amounts of indoor scene configurations. We regard virtual object replacement as a combinatorial optimization problem, considering both geometric and contextual consistency constraints. Quantitative and qualitative results show that our system can generate plausible interactive virtual scenes that highly resemble real environments, and have the ability to keep the sense of presence for users in their VR experiences.

A Qualitative Case Study on Deconstructing Presence for Young Adults and Older Adults

Abstract In this paper, we present the results of an exploratory qualitative case study on presence experienced by groups of young adults and older adults during their use of an interactive virtual reality application mixing realistic and fantasy elements. In contrast to most previous studies, we do not focus on a set of predetermined factors but instead adopt an open-ended qualitative approach to identify emerging factors from the users' experiences. We then analyze these factors against the place illusion/plausibility illusion (PI/PSI) framework of Skarbez, Neyret, et al. (2017) to investigate whether PI and PSI, as well as their contributing factors, can be separated. According to our findings, a user can experience PI and PSI independently from each other; however, they often appeared intermixed when investigated on the scope of the whole experience. Breaks in presence, as well as breaks in plausibility, could mostly, but not entirely, be attributed to immersion and coherence factors, respectively. An interesting finding is that both participant groups turned out to have two subgroups interpreting their experience with a particular frame of reference of differing expectations. These frames of reference affected not only PSI, as expected, but PI as well, suggesting that coherence could be a contributing factor to both PI and PSI. Our contribution adds to the relatively small body of research investigating the separation of PI and PSI. Our exploratory findings can be utilized as directions for designing future confirmatory studies.

EFFECTIVENESS ON TRAINING METHOD USING VIRTUAL REALITY AND AUGMENTED REALITY APPLICATIONS IN AUTOMOBILE ENGINE ASSEMBLY

Training and education have become increasingly crucial in obtaining new skills in a variety of fields, especially in assembly and disassembly operations. The main issue in mechanical engineering, particularly in the assembly department, was that automobile engine components assembly was found to be complicated and challenging to assemble using an existing method, where they only rely on a video-based method. The purpose of this paper is to create interactive Virtual Reality (VR) and Augmented Reality (AR) applications that allow users to efficiently assist and complete the assembly tasks. In this work, the authors designed and developed a fully immersive VR application using an HTC Vive headset and two AR applications (marker-less AR application and marker-based AR application) using EPSON MOVERIO BT-300 (AR Smart Glasses). Fourteen engineering students from Universiti Putra Malaysia were selected for the experiment. They were divided into four groups: video-based group, VR-based group, marker-less AR group, and marker-based AR group. They are required to complete all four experiments (video-based experiment, VR-based experiment, marker-less AR experiment, and marker-based AR experiment). The results showed that the marker-less AR application is the best impressive method (37% better), the VR application is the second impressive method (23% better) followed by the marker-based AR application is the third impressive method (3% better) compared to the existing video-based guideline. Therefore, the students favored AR and VR applications rather than the existing method to be used in automobile engine assembly tasks.

Virtual-real interaction control of hybrid load system for low-carbon energy services

Effectively aggregating demand-side loads is an essential way to promote the adjustment of energy structure. The electric water heaters (EWHs) and air conditioners (ACs) with great regulation potential make up a large part of demand-side loads. A system consisting of many EWHs and ACs presents hybrid characteristics of random and disorder due to the heterogeneity of physical parameters, which increases the difficulty of control. In this paper, a control system in layers based on virtual-real interaction is proposed to solve the problem. The control system consists of four layers: embedded control, aggregation control, cluster control and object control, which form a closed-loop. Starting from the embedded control layer, the bottom-up aggregation model is constructed to form a stable and controllable response system with considerable power transfer capacity. And from the object control layer, the top-down control model is built to stably adjust heterogeneous loads without influence on the comfort of users in a short-time scale. During the aggregation and control, aggregation characteristics of EWHs and ACs are modeled as the stochastic battery to describe the aggregation flexibility, and the complementarity of aggregation responses between EWHs and ACs is relied on to coordinate the load clusters. Moreover, the switching control model is constructed to manage the EWH cluster and develop the aggregation characteristic of EWHs. With the extensibility, the control algorithm is not limited to the application of EWHs and ACs. Finally, a tracking simulation for smoothing the clean energy output is provided to verify the effectiveness of the virtual-real interaction algorithm in layers.

An interactive virtual reality model for enhancing safety training in construction education

AbstractOne of the main factors leading to fatalities in construction sites is the inefficiency of safety training programs. This article presents an immersive and interactive virtual reality (VR) model for safety training, which is an advanced and more comprehensive version of previously developed VR models in the literature. The novelty of this research is present in the proposed general safety training model that is fully immersive and interactive, that is built based on the concepts of experiential learning; where users learn from their mistakes rather than being passive learners in a virtual environment. The model, which was developed using the Unity® platform and C#, contains 12 scenario‐based hazards with programmed interactions and responses to the actions of the trainees. The developed model was tested for two aspects: (1) the ability to achieve learning outcomes and supersede traditional training, and (2) user experience and usability. Using an experimental setting and statistical tests, the developed model has shown statistical significance when it comes to its efficiency in enhancing hazard identification and mitigation skills as opposed to traditional training. The model also performed well in terms of usability and user experience, and important lessons were extracted for future VR development in construction education applications. This research strengthens the utility of VR as an effective tool for experiential learning; which enables future research in exploring how computer applications can improve experiential learning even further.

Wireless Multiplayer Interactive Virtual Reality Game Systems With Edge Computing: Modeling and Optimization

Wireless multiplayer interactive virtual reality (VR) game has the high computing workload of VR and unpredictable interaction among players, which brings severe challenges to the design of wireless communication systems. In this paper, we propose a wireless multiplayer interactive VR game transmission framework based on mobile edge computing (MEC) that is able to model the interaction among players and compute the post-processing procedures at the MEC server or the mobile VR device. In the framework, the absolute delay of each player is used to avoid VR vertigo and the inter-player delay among players is used to model the fairness of the interactive game. Aiming to minimize the average inter-player delay, we optimize the computing resource allocation of the MEC server, the wireless bandwidth allocation and the post-processing decision policy subject to the constraints of the absolute delay requirements, the local energy limits of players, the total bandwidth limit and the computing resources limit. To tackle the non-convex problem efficiently, we design an iterative algorithm based on the NESTT-G algorithm which iteratively optimizes the truncated first-order Taylor approximation of the objective. Numerical results demonstrate the proposed algorithm can reduce the average inter-player delay significantly with lower complexity, and also reveal the impact of different parameters and the channel state conditions on the post-processing decision and edge resource allocation.

Digital Twin and Data-Driven Quality Prediction of Complex Die-Casting Manufacturing

Digital twin and data-driven technologies provide an idea for realizing complex product quality prediction. Aiming at the issues of real-time visual monitoring, operating status analysis, and quality prediction in complex die-casting intelligent manufacturing, a digital twin and data-driven quality prediction architecture is proposed. The virtual-real interaction digital twin of die-casting manufacturing cells is established. The collaborative working mode of physical cells, virtual cells, and real-time monitoring is constructed to predict product quality. The learning method of die-casting parameter data and appearance defect data is proposed to realize the real-time quality prediction in die-casting process and the appearance defect quality prediction after processing, respectively. The data preprocessing and XGBoost-based learning method is proposed for real-time quality prediction of die-casting process. A single-shot refinement neural network for aluminum casting tiny defects detection (Refine-ACTDD) based on deep learning is proposed to solve the high-precision defect detection problems of small appearance defects of complex castings and large interference of complex background. Taking the complex aluminum die-casting as an example, the applications of quality prediction are verified. The method provides a new technical approach for high-precision quality prediction of complex die-casting manufacturing.

Children's perceptions of interactive virtual-reality interventions implemented before and after intravenous cannulation.

Although some studies have reported evidence of the effectiveness of virtual-reality interventions implemented for children undergoing intravenous (IV) cannulation, children's perceptions of virtual-reality interventions implemented during IV cannulation warrant further exploration. To explore the school-aged children's perceptions of interactive virtual-reality interventions implemented before and after IV cannulation. A qualitative descriptive study was adopted. Sixty-nine children aged 6-12 years from two medical centers were recruited and interviewed from June to September 2020. After the completion of the immersive virtual-reality scene of IV cannulation before undergoing actual IV cannulation and the emotionally cathartic virtual-reality play after the placement process, individual interviews were conducted with the children in the paediatric wards. Inductive content analysis was performed to analyse children's perceptions. The study complied with the Consolidated Criteria for Reporting Qualitative Research. Three categories related to children's perception of interactive virtual-reality interventions implemented before and after IV cannulation were identified: (1) feelings towards and coping strategies employed during IV cannulation; (2) mental preparation through immersion in the virtual-reality scene; and (3) healing effects of immersive cathartic play. The findings indicate that interactive virtual-reality interventions can help hospitalised children mentally prepare for medical procedures, obtain knowledge regarding such procedures, and overcome their fear of needles. The children's reported perceptions of the virtual-reality interventions indicated that the interventions were age-appropriate, safe and fun. The results of this study highlight the need to more thoroughly understand the perceptions of hospitalised children and may serve as a reference for designing child-friendly care interventions for nursing practice.

PROCEDURAL DIGITAL TWIN GENERATION FOR CO-CREATING IN VR FOCUSING ON VEGETATION

Abstract. An early-stage development of a Digital Twin (DT) in Virtual Reality (VR) is presented, aiming for civic engagement in a new urban development located in an area that is a forest today. The area is presently used for recreation. For the developer, it is important both to communicate how the new development will affect the forest and allow for feedback from the citizen. High quality DT models are time-consuming to generate, especially for VR. Current model generation methods require the model developer to manually design the virtual environment. Furthermore, they are not scalable when multiple scenarios are required as a project progresses. This study aimed to create an automated, procedural workflow to generate DT models and visualize large-scale data in VR with a focus on existing green structures as a basis for participatory approaches. Two versions of the VR prototype were developed in close cooperation with the urban developer and evaluated in two user tests. A procedural workflow was developed for generating DT models and integrated into the VR application. For the green structures, efforts focused on the vegetation, such as realistic representation and placement of different types of trees and bushes. Only navigation functions were enabled in the first user test with practitioners (9 participants). Interactive functions were enabled in the second user test with pupils (age 15, 9 participants). In both tests, the researchers observed the participants and carried out short reflective interviews. The user test evaluation focussed on the perception of the vegetation, general perception of the VR environment, interaction, and navigation. The results show that the workflow is effective, and the users appreciate green structure representations in VR environments in both user tests. Based on the workflow, similar scenes can be created for any location in Sweden. Future development needs to concentrate on the refinement of buildings and information content. A challenge will be balancing the level of detail for communication with residents.

Transforming e-participation: VR-dialogue – building and evaluating an AI-supported framework for next-gen VR-enabled e-participation research

PurposeThe purpose of this study is to explore whether immersive virtual reality (VR) can complement e-participation and help alleviate some major obstacles that hinder effective communication and collaboration. Immersive virtual reality (VR) can complement e-participation and help alleviate some major obstacles hindering effective communication and collaboration. VR technologies boost discussion participants' sense of presence and immersion; however, studying emerging VR technologies for their applicability to e-participation is challenging because of the lack of affordable and accessible infrastructures. In this paper, the authors present a novel framework for analyzing serious social VR engagements in the context of e-participation.Design/methodology/approachThe authors propose a novel approach for artificial intelligence (AI)-supported, data-driven analysis of group engagements in immersive VR environments as an enabler for next-gen e-participation research. The authors propose a machine-learning-based VR interactions log analytics infrastructure to identify behavioral patterns. This paper includes features engineering to classify VR collaboration scenarios in four simulated e-participation engagements and a quantitative evaluation of the proposed approach performance.FindingsThe authors link theoretical dimensions of e-participation online interactions with specific user-behavioral patterns in VR engagements. The AI-powered immersive VR analytics infrastructure demonstrated good performance in automatically classifying behavioral scenarios in simulated e-participation engagements and the authors showed novel insights into the importance of specific features to perform this classification. The authors argue that our framework can be extended with more features and can cover additional patterns to enable future e-participation immersive VR research.Research limitations/implicationsThis research emphasizes technical means of supporting future e-participation research with a focus on immersive VR technologies as an enabler. This is the very first use-case for using this AI and data-driven infrastructure for real-time analytics in e-participation, and the authors plan to conduct more comprehensive studies using the same infrastructure.Practical implicationsThe authors’ platform is ready to be used by researchers around the world. The authors have already received interest from researchers in the USA (Harvard University) and Israel and run collaborative online sessions.Social implicationsThe authors enable easy cloud access and simultaneous research session hosting 24/7 anywhere in the world at a very limited cost to e-participation researchers.Originality/valueTo the best of the authors’ knowledge, this is the very first attempt at building a dedicated AI-driven VR analytics infrastructure to study online e-participation engagements.