Virtual Training System Research Articles

VR-Based Job Training System Using Tangible Interactions

Virtual training systems are in an increasing demand because of real-world training, which requires a high cost or accompanying risk, and can be conducted safely through virtual environments. For virtual training to be effective for users, it is important to provide realistic training situations; however, virtual reality (VR) content using VR controllers for experiential learning differ significantly from real content in terms of tangible interactions. In this paper, we propose a method for enhancing the presence and immersion during virtual training by applying various sensors to tangible virtual training as a way to track the movement of real tools used during training and virtualizing the entire body of the actual user for transfer to a virtual environment. The proposed training system connects virtual and real-world spaces through an actual object (e.g., an automobile) to provide the feeling of actual touch during virtual training. Furthermore, the system measures the posture of the tools (steam gun and mop) and the degree of touch and applies them during training (e.g., a steam car wash.) User-testing is conducted to validate the increase in the effectiveness of virtual job training.

A Deep Learning Framework of Autonomous Pilot Agent for Air Traffic Controller Training

In this work, a deep learning-based framework is proposed to implement an autonomous pilot agent (APA), which serves as a human pseudo-pilot to assist air traffic controller (ATCO) training. A novel paradigm, including speech recognition, language understanding, pilot repetition generation (PRG), and text-to-speech (TTS), is designed to formulate the framework pipeline, which also incorporates a simulation system interface. We mainly focus on the PRG and TTS models to address the ATC specificities in this work. The neural architecture is proposed to generate the text repetition instruction by using a sequence-to-sequence text mapping. The Transformer block is improved to implement a high-efficient TTS model, in which the nonautoregressive mechanism is applied to achieve the parallel synthesis. A dedicated phoneme vocabulary is designed to cope with the multilingual issue in the ATC domain and address the out-of-vocabulary problem. With the APA framework, a virtual training mode is proposed to complete the training task without the limitation of time and location. Experimental results on a real-world dataset show that the proposed APA framework replaces the human pilot with considerable high confidence in a real-time manner during the simulation training. Most importantly, the APA framework and the virtual training system are able to cope with the dilemma of physical attendance (like COVID-19) and improve the equipment utilization capacity for the ATCO training.

A Virtual Reality Algorithm for the Study of Clinical Efficacy of Sports Injury Rehabilitation Training.

This paper aims to explore the influence of virtual reality technology interactive rehabilitation training system and PT and OT operation therapy on the exercise function, daily life activity ability (ADL), and the quality of life in patients with a sports injury. In this context, this paper mainly designed three experiments to test the virtual rehabilitation system: two action experiments (experiment 1), two experiments with actions in 3 different positions (experiment 2), and five different actions (experiment 3), and the motion intention recognition rate, average total time, and task completion degree of the three experiments were calculated. The virtual scene and hardware equipment were kept stable, and the human-machine interaction effect was good. The effectiveness of the proposed virtual reality rehabilitation training system is demonstrated from other aspects. The results showed that the average completion time of 5 volunteers was 57.72 seconds, with an average offline accuracy of 89.03%. In experiment 2, the five volunteers averaged 54.98 seconds, with an average offline accuracy of 91.73%. The average recognition accuracy of the training system reached 90%, demonstrating the effectiveness of the virtual reality rehabilitation training system in terms of motor intention recognition rate, average total use time, and task completion.

AGRMTS: A virtual aircraft maintenance training system using gesture recognition based on PSO‐BPNN model

AbstractThe quality and efficiency of aircraft maintenance are the key to ensure flight safety and on‐time rate, which mainly depend on the techniques and experience of maintenance engineer. Generally, exercises on physical prototypes are used to improve the maintenance capability of engineers, but this will waste a lot of consumables and easily cause safety accidents. With the development of computer technology, maintenance training in a virtual environment has become an advanced and reliable solution. In this paper, a virtual training system of aircraft maintenance based on gesture recognition interaction is established. Leap Motion is used as a sensor to construct a hybrid machine learning gesture recognition model, so as to obtain natural human–computer interaction experience. In the recognition model, the initial weight matrix and the number of hidden layer nodes in the back propagation neural network are jointly optimized by the Particle Swarm Optimization algorithm with self‐adaption inertial weight. This optimization algorithm achieved a recognition rate of 81.26% in the dynamic gesture database constructed in this paper, which is higher than other available algorithms. A preliminary usability evaluation in university classrooms shows that the teaching system in this paper can achieve a better interactive experience.

A novel virtual reality simulation training system with haptic feedback for improving lateral ventricle puncture skill

A novel virtual reality simulation training system with haptic feedback for improving lateral ventricle puncture skill

Practical skills training in enclosure fires: An experimental study with cadets and firefighters using CAVE and HMD-based virtual training simulators

Training in enclosure fires is essential because it is one of the most dangerous tasks for firefighters. The training simulators should consider the specificity of the tasks performed to achieve the best training outcomes. Additionally, the end-users may not accept even the best technical solutions. Due to the availability of many different interfaces and methods (e.g., for image presentation), end-users' opinions are essential in choosing the best possible technologies. In this research, the aim is to create and compare two realistic virtual training systems for fighting enclosure fires, CAVE (Cave Automated Virtual Environment) with projectors and VR (Virtual Reality) with HMD (Head Mounted Display). Both focused on nozzle person training to fill the gap between theoretical training and real-life training with fires (e.g., drills with class A fuels). Efforts were made to minimize the disadvantages of training simulators while maintaining high simulation realism and low simulator cost. The study was conducted with the participation of 67 cadets and seven active firefighters. The method of reducing the limitations of VR training simulators while maintaining relatively low costs is discussed. The simulator enables multisensory experience with passive haptic and multi-user interaction considering the sense of touch. The VR simulator was rated better than the CAVE simulator, but both were accepted and rated as very useful. The paper also discusses the issues of simulation sickness symptoms, simulation realism (spatial presence), usability, acceptance of technology, and stress associated with using a training simulator.

Design and Development of Water Supply and Drainage Virtual Simulation Training System

Traditional practical teaching of water supply and drainage is often faced with problems such as limited faculty and course time, easy worn-out of equipment and safety risk in practical training, which brings many inconveniences to schools, teachers and students. In order to make up for the shortage of traditional water supply and drainage practical training teaching, the teaching environment and the process was simulated by virtual reality technology, and a virtual simulation training system for water supply and drainage was developed. Taking the course project of steel pipe threading as an example, this paper shows the construction situation of manual and electric steel pipe threading project, systematically explains the principle involved in the virtual training system of water supply and drainage, emphatically expounds the key technologies, and the implementation scheme was given.

Evaluation on applicationof virtual reality technology in dental implant process training

Objective: To explore the application effect and evaluation of virtual reality technology in oral implant therapy training. Methods: In November 2018, one adult patient (female, 36 years old) with missing right mandibular first molar was treated in the Department of Implantology, School and Hospital of Stomatology, Fujian Medical University. The three-dimensional virtual models of mandible and implant surgery tools were established, and the virtual reality software (Unity 3D 5.5.1) was imported. Combined with the virtual reality head mounted display, a virtual reality training system simulating the dental implant treatment process was independently developed. Ten refresher doctors and 20 graduate students in Department of Implantology, School and Hospital of Stomatology, Fujian Medical University from September 2018 to December 2019 were recruited as the experimental objects (no clinical experience was found). According to the level and seniority of doctors, they were randomly divided into virtual training group and conventional training control group, which made the two groups comparable, with 15 in each group. Subjective scores (including anatomical structure, surgical field of vision, cavity preparation, implant placement and process mastery) were given after the corresponding training in the two groups, and the virtual reality training system was used to test. The mesial and distal direction, buccolingual direction, depth and angle deviation of implants before and after the training were analyzed, and the differences between the two groups were compared. Results: The subjective scores of five dimensions in the virtual training group were significantly higher than those in the conventional training control group (P<0.05). In the virtual training group, the mesial and distal, buccolingual, depth and angle deviation of implants were (0.73±0.33), (0.78±0.41), (0.61±0.32) mm and 6.66°±3.87°. All of them were significantly lower than those in the control group [(0.85±0.32), (1.12±0.38), (0.89±0.24) mm and 9.68°±3.74°] (P<0.05). Conclusions: The self-developed virtual reality system of oral implant has good application effect, good operability and predictability. It can be effectively carried out in implant education and training, and it can strengthen skills of doctors, and is conducive to the practical operation.

Human-Computer Interaction Virtual Training System for College Students’ Innovation and Entrepreneurship Education

A recent trend in human-computer interaction has facilitated content production, such as innovation and information growth through intelligent systems, enabling unqualified users to describe a machine’s behaviour through visual programming. However, the difficulty could become similar to the standard coding techniques as the number of elements to be managed increases. Entrepreneurial education is defined as a collection of formalized lessons that advise, train, and educate all people who want to participate in socio-economic development through a project that promotes knowledge of entrepreneurship to advance the business. Hence, in this paper, the Human-computer interaction-based Entrepreneurship learning platform (HCI-ELP) has been proposed to boost student quality of life by enhancing self-confidence and changing the learning timeframe. HCI provides several promising implications for integrating knowledge management approaches, taking advantage of computer-supported interactive learning from entrepreneurship learning platforms. The suggested approach recommends collaboration to learn for students’ opinions on learning developed through a questionnaire’s tasks. Thus, the experimental results show that HCI-ELP achieves the highest performance rate to 97.66% and enhances student interaction in completing the task to improve curriculum development.

The training experience of nursing educators by virtual training system – results and reflection

NA

Development of augmented virtual reality-based operator training system for accident prevention in a refinery

Development of augmented virtual reality-based operator training system for accident prevention in a refinery

C-Hg

Computer-assisted systems can provide efficient and engaging ASD intervention environments for children with Autism Spectrum Disorder (ASD). However, most existing computer-assisted systems target only one skill deficit (e.g., social conversation skills) and ignore the importance of other areas, such as motor skills, that could also impact social interaction. This focus on a single domain may hinder the generalizability of learned skills to real-world scenarios, because the targeted teaching strategies do not reflect that real-world tasks often involve more than one skill domain. The work presented in this article seeks to bridge this gap by developing a Collaborative Haptic-gripper virtual skill training system (C-Hg). This system includes individual and collaborative games that provide opportunities for simultaneously practicing both fine motor skills (hand movement and grip control skills) as well as social skills (communication and collaboration) and investigating how they relate to each other. We conducted a usability study with 10 children with ASD and 10 Typically Developing (TD) children (8–12 years), who used C-Hg to play a series of individual and collaborative games requiring differing levels of motor and communication skill. Results revealed that participant performance significantly improved in both individual and collaborative fine motor skill training tasks, including significant improvements in collaborative manipulations between partners. Participants with ASD were found to conduct more collaborative manipulations and initiate more conversations with their partners in the post collaborative tasks, suggesting more active collaboration and communication of participants with ASD in the collaborative tasks. Results support the potential of our C-Hg system for simultaneously improving fine motor and social skills, with implications for impacts of improved fine motor skills on social outcomes.

A Technique for Mimicking Soft Tissue Manipulation from Experimental Data to a Wave Equation Model for a New Laparoscopic Virtual Reality Training System

Background: The difficulty of laparoscopic procedures and the specific psychomotor skills required support the need for a training system for intensive and repetitive practice to acquire the specific skills. The present VR training systems have some limitations with respect to the soft tissue models in the training system. This is associated with the need for a real-time simulation, which requires a balance between computational cost and accuracy. Objective: The primary objective of the study is to develop a two dimensional wave equation model that closely mimics the soft tissue manipulation in a laparoscopic procedure for a VR training system. Methods: A novel mathematical model based on the wave equation is prepared to represent the interaction between the laparoscopic tool and the soft tissue. The parameters within the model are determined through experimental analysis of a soft tissue phantom. The experimental setup involves a linear actuator applying force to the soft tissue phantom to generate deformation. Data acquisition is conducted through a camera and a robotic force acquisition system which measures force, displacement of the linear actuator and records a video. Through image processing, the displacements of the markers on the phantom’s x-y plane during its deformation are determined and these parameters are used to develop the model, which finally is validated through a comparative analysis. Results: The results from the developed model are observed and compared statistically as well as graphically with the finite element model based on deformation data. The results show that the deformation data between the developed model and the available model is significantly similar. Conclusion: This study demonstrates the adaptability of the wave equation to meet the needs of the specific surgical procedure through modification of the model based on the experimental data. Moreover, the comparative analysis further corroborates the relevance and validity of the model for the surgical training system.

Development of an 360-degree virtual reality video-based immersive cycle training system for physical enhancement in older adults: a feasibility study

BackgroundRecently, there is an increased number of studies that use 360° virtual reality (VR) video for medical and rehabilitative purposes. However, the 360° VR video experience for older adults has not yet been investigated. This study aimed to examine the validity of an 360° VR video-based immersive cycling training system (360° VRCTS) for older adults and to provide preliminary evidence of efficacy.MethodsWe developed a new virtual reality training system using an immersive environment 360° VRCTS. Five healthy older adults (2 males and 3 females) participated in this study. The system was tested in a single training session (biking for 20 min while viewing a 360° VR video scene through a large curved screen) to identify its strengths and weakness. The usability and acceptability of our system were measured using the system usability scale (SUS) and the simulator sickness questionnaire (SSQ).ResultsAll participants successfully completed the session without any discomfort. The average score for the SUS was 94.60 (range, 90–100), indicating high usability of the technology. The average score for the SSQ was 2.24 (standard deviation = 2.05), indicating that the system is well tolerated and has few side effects.ConclusionsThe 360° VRCTS may be a useful indoor training system for older adults due to its easy manipulation, high usability, and limited cybersickness.Trial registration numberClinical Research Information Services (CRiS), KCT0003555, Registered February 25, 2019, https://cris.nih.go.kr/cris/index/index.do.

Effects of Level of Immersion on Virtual Training Transfer of Bimanual Assembly Tasks

The availability of consumer-facing virtual reality (VR) headsets makes virtual training an attractive alternative to expensive traditional training. Recent works showed that virtually trained workers perform bimanual assembly tasks equally well as ones trained with traditional methods. This paper presents a study that investigated how levels of immersion affect learning transfer between virtual and physical bimanual gearbox assembly tasks. The study used a with-in subject design and examined three different virtual training systems i.e., VR training with direct 3D inputs (HTC VIVE Pro), VR training without 3D inputs (Google Cardboard), and passive video-based training. 23 participants were recruited. The training effectiveness was measured by participant’s performance of assembling 3D-printed copies of the gearboxes in two different timings: immediately after and 2 weeks after the training. The result showed that participants preferred immersive VR training. Surprisingly, despite being less favourable, the subjects’ performance of video-based training were similar to training on HTC VIVE Pro. However, video training led to a significant performance decrease in the retention test session 2 weeks after the training.

Effect of digital virtual simulation system for preclinical teaching of access and coronal cavity preparation

Objective: To investigate the effectiveness of a digital virtual simulation training system applied in the preclinical teaching of access and coronal cavity preparation. Methods: Twenty dental undergraduate students from Peking University School and Hospital of Stomatology were recruited and divided randomly and equally into two groups according to the random number method after being unified with theory training, including access and coronal cavity preparation skills assessment form and Simodont system operation manual. Tests for access and coronal cavity preparation skills by using standard simulation plastic teeth were performed and the scores were recorded as baseline for each student. Students in group of virtual simulation priority were trained using Simodont virtual simulation system, while those in group of phantom-simulator priority were trained using conventional phantom-simulator system. Access and coronal cavity preparation skills of standard simulation plastic teeth were assessed once again and recorded as the second scores for the two groups. Furthermore, the two groups of students exchanged training systems and were assessed and graded once more as the third scores. Finally, all students were asked to fill up a Teaching Questionnaire after the training. The data were then collected and analyzed. Results: For the group of virtual simulation priority, after the training by using Simodont virtual simulation system and conventional phantom-simulator system, the mean score of access and coronal cavity preparation (16.00±1.49) was significantly higher than the baseline score (13.30±1.41) (P<0.05). For the group of phantom-simulator priority, after the training by using conventional phantom-simulator system and Simodont virtual simulation system, the mean score of access and coronal cavity preparation (15.60±1.26) was also significantly higher than the baseline score (13.00±1.89) (P<0.05). Furthermore, in the group of virtual simulation priority, of which the students were trained by using Simodont virtual simulation system first and then conventional phantom-simulator system, the score of access and coronal cavity preparation was significantly higher than the score of training by using conventional phantom-simulator system only (14.30±1.77) (P<0.05). In the group of phantom-simulator priority, of which the students were trained by using conventional phantom-simulator system first and then Simodont virtual simulation system, the score of access and coronal cavity preparation was significantly higher than the score of training by using Simodont virtual simulation system only (14.10±1.45) (P<0.05). Moreover, in the group of virtual simulation priority, the score of training by using conventional phantom-simulator system after using Simodont virtual simulation system was significantly higher than that of training by using Simodont virtual simulation system only (P<0.05). The results of the questionnaire showed that the students fully agreed that "the Simodont virtual simulation system has the characteristics of repeatability, multi-dimension and multiple practice, and provides me with more attention to details" [80% (16/20)], however "it needs to be improved and upgraded to be close to the conventiaonl phantom-simulator system" [90% (18/20)]. Conclusions: Compared with using the conventional phantom-simulator system only, the preclinical teaching effectiveness of access and coronal cavity preparation could be effectively improved by using Simodont virtual simulation system combined with the phantom-simulator training system and might influenced by the training sequence.

Discuss on the Application of Computer Virtual Reality Technology in College Sports Training

With the computer technology slowly infiltrated into various industries, sports teaching also began to use the computer virtual reality technology for sports training. Cognition of sports teams to virtual reality training system with the computer software developers and investigate the cooperation between sports teams, and computer virtual reality technology is also being used in training such as diving or trampolining, the application is able to help athletes improve the motion precision, reduce the risk in daily training.

Construction and Application of Virtual Experiment Teaching System for Integrated Mapping of Unmanned Ship

At present, China’s water pollution situation is still very serious, and water pollution prevention is an important part of ecological environment protection, and water quality monitoring and pollution traceability are the basis of prevention and control work. Unmanned ship system uses GNSS intelligent navigation and positioning technology, automatic obstacle avoidance technology, real-time communication technology, sonar sounding technology and manual control technology, etc., by collecting underwater terrain, water quality information and underground pipe detection, it can complete water quality detection unmanned, intelligent and efficient. In this paper, the system architecture and measurement principle of unmanned equipment are comprehensively expounded, and based on WebGL and virtual simulation technology, an open unmanned equipment virtual simulation training system is constructed, which can directly carry out assembly training of unmanned aircraft and unmanned ship and flight control training of unmanned aircraft on the Web side. Finally, the feasibility and development prospect of unmanned equipment in sewage monitoring are discussed with an application example.

Simulation of Indoor Fire Dynamics of Residential Buildings with Full-Scale Fire Test

Along with simulated firefighting training, the development of virtual training systems and associated content has recently drawn attention as an alternative method for advanced firefighting training. In particular, to develop virtual training content, it is important to understand the combustion characteristics of indoor combustible materials and appropriately simulate their behavior. In this study, seven types of building fires were selected. From these types, indoor combustible materials of residential buildings were analyzed, and combustion tests were performed for each of three types of combustible materials. Furthermore, three types of indoor spaces, determined according to the combination of indoor combustible materials they contained, were divided into full-scale compartments, and a full-scale fire test was performed using this setup. Finally, the heat release rate and smoke production release of individual combustible materials and the room test were measured over time, and a fire dynamics simulation (FDS) was performed. As a result, it was confirmed that an effective evaluation of the occurrence and spread of fire indoors in buildings is possible through full-scale fire tests and FDS simulations. It is expected that simulation can be used as firefighting training content in the future by applying indoor combustible data and implementing complex and various fire development conditions.

Investigating the P300 Response as a Marker of Working Memory in Virtual Training Environments

Conventional performance metrics fail to offer high-resolution evaluation of learning and memory during training tasks; the P300 component of the event-related potential (ERP) is a promising tool for enhancing the assessment of training quality in virtual environments, but this technique is yet to be investigated. A driver training simulator and scenario were developed to explore the capability of the P300 for this purpose. A user study was conducted with 32 participants divided into two groups objectively determined by driving performance scores, thus enabling observations of the P300 response to be equated to varying levels of learning and memory. Participant electroencephalogram data were recorded during the procedure, which was postprocessed to filter and extract ERPs to capture neural responses to specific events in the virtual training scenario. These were combined to produce a result for each participant, which was then grand averaged to create an overall ERP for each group. Across the eight electrode sites, statistically significant differences were found between the grand average waveforms of the two groups, with high memory retention producing significantly greater peak-to-peak amplitude (U = 9.00, p = 0.045), peak latency (U = 0.00, p $<; $ 0.001), and positive area (U = 13.00, p = 0.05) of the waveform than low memory retention. The evidenced relationship between the P300 response and working memory in this context suggests that it has the potential for monitoring learning and memory in stimulus-driven virtual training systems.