Virtual Surgery Training Research Articles

Improved shape-from-template method with perspective space constraints for disappearing features

Inferring the 3D surface shape of a known template from 2D images captured by a monocular camera is a challenging problem. Due to the severely underconstrained nature of the problem, inferring shape accurately becomes particularly challenging when the template exhibits high curvature, resulting in the disappearance of feature points and significant differences between the inferred and actual deformations. To address this problem, this paper proposes a concise and innovative approach that utilizes a physical simulator incorporating the object’s material properties and deformation law. We utilize a view frustum space constructed from the contours of a monocular camera image to effectively restrict the physically-based free motion of the template. Additionally, we employ mesh denoising techniques to ensure the smoothness of the surface following deformation. To evaluate our shape inference results, we utilize a ground truth 3D point cloud generated from multiple viewpoint images. The results demonstrate the superior performance of our approach compared to other methods in accurately inferring deformations, particularly in scenarios where feature points are unobservable. This method carries significant practical implications across diverse domains, including virtual reality, digital modeling, and medical surgery training.

A New Composite Control Strategy for an Astronaut Virtual Operation Training System Based on Cable-Driven Technology

In recent years, virtual microgravity training technology for astronauts based on cable-driven designs has emerged, and it solves the following problems: high costs, short training times, and low safety of existing equipment. However, this technology does not solve the reduced motion accuracy problem of the operated object due to the elastic deformation of cables, and this problem will reduce the operational experience of astronauts during training. In view of this problem, a cable-driven virtual operation training system for astronauts is designed, and a new composite control strategy based on parallel cables is proposed, which effectively improves motion control accuracy by allocating cable tension and using a tension compliance control method to suppress the influence of cable deformation. In addition, the desired tension of cables is optimized based on the system’s workspace so that the system can achieve more complex virtual microgravity training tasks. Finally, verification via experiments demonstrated that the training system and the new composite control strategy are effective.

Using Beta Rhythm From EEG to Assess Physicians' Operative Skills in Virtual Surgical Training

The advancement of virtual reality technology has ushered in new developments in the medical field. The use of virtual surgery training simulators alleviates the paucity of training resources and high training expenses associated with traditional surgical capabilities. Regardless of the type of schooling, doctors must continue to educate themselves. The postoperative evaluation mechanism is incomplete. Traditional objective evaluation indicators are unable to meet surgeons' stringent expectations. The electroencephalograph (EEG) rhythm index is proposed in this article as a new tool for evaluating and distinguishing between novice and expert doctors. The experiment uses a cutting training module from neurosurgery training and compares it with established assessment metrics to determine the correct rate of classification of new evaluation metrics, classifying testers by both metrics and finding a 20% increase in correctness. Additionally, this article compares the energy topographic maps of different EEG rhythms of novices and experts. For classification, two-machine learning algorithms, SVM and random forest, are utilized at the same time. The findings reveal that the accuracy of distinguishing indicators based on EEG cycles is 10% higher than that of typical objective evaluation indicators, regardless of the categorization method. ROC curve analysis was also used to compare the two classification models. The AUC value for the EEG rhythm evaluation index model was 0.971, whereas the AUC value for the classic objective evaluation index model was 0.761, which explains the EEG rhythm evaluation index. The model demonstrates a categorization standard that is reliable.

Assessment of students' satisfaction with virtual robotic surgery training

ObjectiveNowadays, in Bulgaria there is a trend of increasing entry into the surgical field of robot-assisted surgery operations, which suggests a need for the establishment of a large number of specialists in this field in a short period. Based on these arguments, the Medical University of Varna was the first university in the country to introduce a robotic surgery training program for medical students. The study aims to investigate the medical students’ satisfaction on robotic surgery training provided at Medical University of Varna with da Vinci Skills Simulator. DesignDuring the summer semester of the academic 2020/2021 and 2021/2022 years, a pilot training of robotic surgery was conducted with 5th year students in Medicine. Within one month, the students had the opportunity to get acquainted with the simulator of da Vinci Xi robotic system. The training was divided into two modules: a two-week theoretical module and a two-week practical module. After completing the training, students filled out a questionnaire dedicated to assess their satisfaction with the proposed training. Correlation between their responses and the objective parameters assessed on the simulator was calculated. ResultsThirty participants (16 men and 14 women) shared their opinion on easiness of use and usefulness of the robotic simulator in training of surgery activities. Students’ responses highly evaluated both aspects with average five-point Likert scale scores of 4.3 and 4.5, respectively. 93% of the participants would continue their further education and training in robotic surgery field. In addition, there was no correlation between objective evaluation by the simulator and students responses. ConclusionsTraining in robotic surgery proves to be a useful approach for training students to develop skills and profession in the field of surgery. The results suggest that training in this field may be accomplished even at the student level, by exploiting the robotic surgery in realistic scenario and thus, in a timely manner to find out the surgical direction they want to be further evolved.

A novel virtual cutting method for deformable objects using high-order elements combined with mesh optimisation.

Virtual cutting of deformable objects plays an important role in many applications, especially in digital medicine, such as soft tissue cutting in virtual surgery training system. We developed a novel virtual cutting algorithm, combined with mesh optimisation. A new local mesh processing method is used to control the number and quality of the elements created during the cutting process. At the same time, high-order tetrahedral elements are used to fit the cutting surface and reduce the mesh size. In this paper, single cut, multiple cut and intersecting cut are performed on the mesh model, combined with a force feedback device, and the result obtained is that the visual feedback is higher than 30Hz, and the tactile feedback is 800∼1000Hz. Experimental results show that the method proposed in this paper can effectively eliminate low-quality elements and control the mesh size, thereby ensuring real-time simulation.

Cloud computing-enabled IIOT system for neurosurgical simulation using augmented reality data access

In recent years, statistics have indicated that the number of patients with malignant brain tumors has increased sharply. However, most surgeons still perform surgical training using the traditional autopsy and prosthesis model, which encounters many problems, such as insufficient corpse resources, low efficiency, and high cost. With the advent of the 5G era, a wide range of Industrial Internet of Things (IIOT) applications have been developed. Virtual Reality (VR) and Augmented Reality (AR) technologies that emerged with 5G are developing rapidly for intelligent medical training. To address the challenges encountered during neurosurgery training, and combining with cloud computing, in this paper, a highly immersive AR-based brain tumor neurosurgery remote collaborative virtual surgery training system is developed, in which a VR simulator is embedded. The system enables real-time remote surgery training interaction through 5G transmission. Six experts and 18 novices were invited to participate in the experiment to verify the system. Subsequently, the two simulators were evaluated using face and construction validation methods. The results obtained by training the novices 50 times were further analyzed using the Learning Curve-Cumulative Sum (LC-CUSUM) evaluation method to validate the effectiveness of the two simulators. The results of the face and content validation demonstrated that the AR simulator in the system was superior to the VR simulator in terms of vision and scene authenticity, and had a better effect on the improvement of surgical skills. Moreover, the surgical training scheme proposed in this paper is effective, and the remote collaborative training effect of the system is ideal.

Application of virtual simulation experiment system in instrumental analysis teaching in colleges

With the rapid development of computer information technology, the virtual simulation technology is coming as the times require. In recent years, due to its good interactivity, versatility and openness, virtual laboratories have received increasing widely attention in the field of instrumental analysis experiment teaching. The current teaching demonstration is limited by the laboratory equipment, resulting in a lot of students fell behind the class. Combined with the advantages and experimental teaching characteristics of virtual simulation technology, a set of virtual experiment teaching system was designed. The system includes experimental purposes, experimental content, experimental principles, teaching animations, virtual operation training, precautions, and experimental virtual tests. The virtual experiment teaching system can be used as an important supplement to the experimental teaching, and improve teaching effects.

1420 ViRtual prOficiency Based prOgression for Robotic Training (VROBOT): A Prospective Cohort Study Protocol

Abstract Introduction Robotic surgery is an evolving field that requires specialist training. Historically, robotic surgery training has lacked standardisation. Recently, training centres have introduced proficiency-based modules and curriculums to certify and progress the skills of novice robotic surgeons. However, training tends to be self-directed and non-interactive. Limited interactive teaching does exist but can be inaccessible and expensive. We aim to validate the effectiveness of the current Fundamentals of Robotic Surgery (FRS) training curriculum with the addition of interactive virtual classroom teaching. Method 16 novice surgical trainees will be assigned to two training groups. The interventions will be implemented following a one-week robotic skills induction. Both groups will receive access to the FRS curriculum for one week. The intervention group will additionally receive virtual classroom robotic skills training. The primary outcome will be the objective performance scores after training using a synthetic model based on task errors, time taken and contact pressure. In week 3, each group will receive the alternate intervention and objective performance scores will be measured to determine the trajectory of scores. Results Significant objective performance improvement following the intervention will be indicative of intervention quality. Conclusions This will be the first feasibility study evaluating the efficacy of interactive virtual robotic surgery training. It will determine the effect size of virtual classroom training on the development of basic robotic surgical skills in addition to the proficiency-based FRS curriculum. The findings will assist the development and implementation of further resource-efficient virtual robotic surgical skills training programs.

Investigating the Effect of VR + Haptics Approach on Students’ Flow Experience and Outcomes: An Empirical Study on VR Laparoscopy

Background. Virtual reality-based surgery training has become a promising trend in the sector of clinical education. Prior studies have confirmed the effectiveness of VR-based surgical simulators on training surgeons. Most existing papers employing subjective methods explored the students’ overall perceptions of surgical skills. However, few studies, from the multidimension perspective of learning performance, investigate how VR improves surgery skills. Participants. 37 college students were recruited in comparative experimental research. The experimental group was equipped with a VR + haptic surgical simulator, while the VR simulator without haptic feedback was used for the control group. Method. The study resorted to physiological approaches to investigate the influence of the VR laparoscopic surgical training system on students’ performance. Results. The experimental group scored higher than the control group in flow experience and has better performance in the four dimensions of operation evaluation skills. Conclusion. The study deposited that learners are more likely to exert to flow experience in a learning situation with haptic feedback, which will further improve medical students’ performance.

Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems

BackgroundCompared with traditional thoracotomy, video-assisted thoracoscopic surgery (VATS) has less minor trauma, faster recovery, higher patient compliance, but higher requirements for surgeons. Virtual surgery training simulation systems are important and have been widely used in Europe and America. Augmented reality (AR) in surgical training simulation systems significantly improve the training effect of virtual surgical training, although AR technology is still in its initial stage. Mixed reality has gained increased attention in technology-driven modern medicine but has yet to be used in everyday practice. MethodsThis study proposed an immersive AR lobectomy within a thoracoscope surgery training system, using visual and haptic modeling to study the potential benefits of this critical technology. The content included immersive AR visual rendering, based on the cluster-based extended position-based dynamics algorithm of soft tissue physical modeling. Furthermore, we designed an AR haptic rendering systems, whose model architecture consisted of multi-touch interaction points, including kinesthetic and pressure-sensitive points. Finally, based on the above theoretical research, we developed an AR interactive VATS surgical training platform. ResultsTwenty-four volunteers were recruited from the First People's Hospital of Yunnan Province to evaluate the VATS training system. Face, content, and construct validation methods were used to assess the tactile sense, visual sense, scene authenticity, and simulator performance. ConclusionsThe results of our construction validation demonstrate that the simulator is useful in improving novice and surgical skills that can be retained after a certain period of time. The video-assisted thoracoscopic system based on AR developed in this study is effective and can be used as a training device to assist in the development of thoracoscopic skills for novices.

Animation of virtual medical system under the background of virtual reality technology

AbstractVirtual reality technology is a technology that uses a variety of scientific technologies such as computer systems, image displays, and sensors to perform human–computer interaction. Since the advent of virtual reality technology, the virtual reality + medical model has been highly expected. At present, three‐dimensional (3D) animation technology can be used in combination with virtual reality technology in medical systems. 3D animation technology can transform the transformed human medical image into a 3D character model, and animate the corresponding bones, and observe the model on the screen and operate the model like real surgery, completing virtual medical content learning, virtual medical skill training, virtual surgery preview. Applying 3D animation technology to virtual reality surgery training can effectively bypass the limitations of traditional surgery training modes, form multiple interactive real‐time effects, and bring a rich perceptual experience to the audience. Based on the above background, the research content of this article is the study of virtual medical system animation under the background of virtual reality technology. This paper analyzes the feasibility of virtual reality technology and 3D animation modeling, and designs a virtual medical system based on this. Finally, through experimental simulations, the results show that the gap h is getting larger and larger after the first approximation, and the gap between the Euler explicit method and the true value is larger, while the implicit method is more stable, and the gap does not increases too quickly with the increase of h. Hence choosing the implicit method to calculate Δt to obtain a larger value will not affect the stability of the solution.

GPU-friendly data structures for real time simulation

Simulators for virtual surgery training need to perform complex calculations very quickly to provide realistic haptic and visual interactions with a user. The complexity is further increased by the addition of cuts to virtual organs, such as would be needed for performing tumor resection. A common method for achieving large performance improvements is to make use of the graphics hardware (GPU) available on most general-use computers. Programming GPUs requires data structures that are more rigid than on conventional processors (CPU), making that data more difficult to update. We propose a new method for structuring graph data, which is commonly used for physically based simulation of soft tissue during surgery, and deformable objects in general. Our method aligns all nodes of the graph in memory, independently from the number of edges they contain, allowing for local modifications that do not affect the rest of the structure. Our method also groups memory transfers so as to avoid updating the entire graph every time a small cut is introduced in a simulated organ. We implemented our data structure as part of a simulator based on a meshless method. Our tests show that the new GPU implementation, making use of the new graph structure, achieves a 10 times improvement in computation times compared to the previous CPU implementation. The grouping of data transfers into batches allows for a 80–90% reduction in the amount of data transferred for each graph update, but accounts only for a small improvement in performance. The data structure itself is simple to implement and allows simulating increasingly complex models that can be cut at interactive rates.

Study on Collision Detection and Force Feedback Algorithm in Virtual Surgery.

The development of virtual reality technology is expected to solve traditional surgical training. The lack of methods has brought revolutionary advances in technology. The virtual surgery system based on collision detection and force feedback can enable the operator to have stronger interaction, which is an exploration of the feature of touch in virtual reality technology. Reality is an important indicator of the virtual surgical system. This article improves the realism of the system from the visual and tactile senses and uses the surrounding ball collision detection and force feedback algorithms to build a realistic surgical platform. In the virtual surgery training system, the introduction of force feedback greatly improves the sense of presence during virtual surgery interaction. The operator can feel the softness and hardness of different tissues and organs through the force feedback device. Virtual reality is an interdisciplinary comprehensive technology that has been widely used in military, film, medical, and gaming fields. Virtual reality can simulate the objective world and display it visually, making people feel immersive. Virtual surgery provides surgeons with a recyclable surgical practice platform and can help doctors perform preoperative rehearsals and predict the results of surgery. The design of collision detection and force feedback algorithms is a prerequisite to ensure the immersion and transparency of the virtual surgical training system. This article mainly introduces the collision detection and force feedback algorithm research in virtual surgery, with the intention of providing some ideas and directions for the development of virtual surgery. This paper proposes two collision detection algorithms, space decomposition method and hierarchical bounding box method, and three force feedback algorithms including spring mass point algorithm, Runge–Kutta method, and Euler method to construct virtual surgery collision detection and force feedback. Experiment with the Overall System Architecture. This paper proves through experimental results that the average collision detection time after the application of the improved collision detection and force feedback algorithm in the virtual surgery system is more than 80.7% less than the traditional method, which greatly improves the detection speed.

Mathematical Modeling and Virtual Reality Simulation of Surgical Tool Interactions With Soft Tissue: A Review and Prospective

This is an informed assessment of the state of the art and an extensive inventory of modeling approaches and methods for soft tissue/medical cutting tool interaction and of the associated medical processes and phenomena. Modeling and simulation through numerical, theoretical, computational, experimental, and other methods was discussed in comprehensive review sections each of which is concluded with a plausible prospective discussion biased toward the development of so-called virtual reality (VR) simulator environments. The finalized prospective section reflects on the future demands in the area of soft tissue cutting modeling and simulation mostly from a conceptual angle with emphasis on VR development requirements including real-time VR simulator response, cost-effective “close-to-reality” VR implementations, and other demands. The review sections that serve as the basis for the suggested prospective needs are categorized based on: (1) Major VR simulator applications including virtual surgery education, training, operation planning, intraoperative simulation, image-guided surgery, etc. and VR simulator types, e.g., generic, patient-specific and surgery-specific and (2) Available numerical, theoretical, and computational methods in terms of robustness, time effectiveness, computational cost, error control, and accuracy of modeling of certain types of virtual surgical interventions and their experimental validation, geared toward ethically driven artificial “phantom” tissue-based approaches. Digital data processing methods used in modeling of various feedback modalities in VR environments are also discussed.

Stable and efficient collision detection scheme for hip-surgery training system

To satisfy real-time requirements, collision detection algorithms should be stable and efficient, and should be capable of accurate detection. This paper presents a collision detection algorithm that is based on the concept of dimension reduction of imitating latitude and longitude space. We achieve a rough and accurate detection function of the algorithm after four steps including latitude detection, bounding box detection, crossing-algorithm detection, and an affine coordinate system for accurate detection. The efficiency of the proposed collision detection algorithm is higher than that of the aligned axis bounding box (AABB) collision detection algorithm in terms of both rough detection and accurate detection. The efficiencies of coarse detection and accurate detection are basically the same, and the efficiencies of both types of detection are stable. The proposed collision detection algorithm has good stability, high efficiency, and accurate detection, and it can be used as the core algorithm of a collision detection module in virtual surgery training systems.

Study on Stereo Vision-based Parameterized Virtual Hand Modeling

Virtual hand modeling is one fundamental part of the model-based hand tracking system. The virtual hand model-based tracking system can be valuable in the field of the immersive human machine interaction simulation such as virtual surgery and virtual maintenance training. In this paper, a method of automatic stereo vision-based virtual hand modeling system is presented. We firstly built a template of the improved virtual hand model, then extracted the hand features from images captured by the binocular stereo vision system, and completed modeling the virtual hand in accordance with the estimated parameters. We propose a new method of combining local distance maximum and curvature vector to extract the hand features and introduce a method of recognizing the different finger tips. The advantages of feature extraction method loosen the high quality of contour extraction from images and enforce the robustness of feature extraction.

Interaction techniques in desktop virtual environment: the study of visual feedback and precise manipulation method

Gesture-based systems allow users to interact with a virtual reality application in a natural way. Visual feedback for the gesture-based interaction technique has an impact on the performance and the hand instability making the manipulation of the object less precise. This paper investigated two new interaction techniques in a virtual environment. It describes the influence of natural and non-natural virtual feedback in the selection process using the GITDVR-G interaction technique, which consists of a grasping visual feedback. The GITDVR-G was evaluated in a virtual knee surgery training system. The results showed that it was effective in terms of the task completion time, and that the participants preferred the natural grasping visual feedback. Besides that, the precise manipulation in a newly-designed interaction technique (Precise GITDVR-G) was evaluated. The Precise GITDVR-G includes a normal manipulation mode and a precise manipulation mode that can be triggered by hand gestures. During the precise manipulation mode, an inset view will appear and move with the selected object to provide a better view to users, while the movements of the virtual hand are scaled down to improve the precision. Four different configurations of the precise manipulation technique were evaluated, and the results showed that the unimanual control method with an inset view performed better in terms of the task performance time and the subjective feedback. The finding suggested that the realistic virtual grasping visual feedback can be applied in a virtual hand interaction technique, and that the inset view feature is helpful in the precise manipulation.

Real-time cutting simulation in virtual reality systems based on the measurement of porcine organs

A virtual soft tissues cutting model consistent with the organ specificity of real soft tissues was established in this paper, which was applied to the virtual operation training system. A measurement platform of soft tissue organ was designed and built, and the stress–strain and stress–relaxation data of pig liver and kidney were experimentally measured. Then, using the viscoelasticity mathematical formula, an improved virtual cutting model of the meshless classified balls-filling was constructed through VC++ and OpenGL. The cutting performance of the virtual soft tissues was further increased by leveraging the improved cutting classification algorithm. Finally, the extrusion and cutting simulation was enabled through the force feedback device, and the accuracy and effectiveness of this cutting model were validated by a comparative study of the virtual soft tissues cutting model and the stress–strain and stress–relaxation data of pig liver and kidney.

Application of 3D Visualization Technology Based on Lightweight Model in Virtual Operation Training

This paper discusses a kind of Visualization 3D technology, which is based on the light quantization model, which is a kind of advanced technology of computer data processing and image display, which was born in the middle of 1980s ". It is a kind of image display tool that uses 3D seismic data to display, describe and explain the phenomena and characteristics of subsurface geology. It can make the earth physicists and geologists "drill in" to the data body, a deeper understanding of the occurrence, development and mutual connection of various geological phenomena.

Virtual Training System with Physical Viscoelastic Model and Blood Flow Simulation Based on a Range-Based SPH Method

A physical mesh-less soft tissue cutting model with the viscoelastic creep characteristics has been proposed in this paper. The model is composed of filled spheres which are connected by Kelvin structure, so as to realize the cutting with viscoelastic creep characteristics. Then, it is further compared with the mass spring model in order to verify the effectiveness of the model. Secondly, a range-based Smoothed Particle Hydrodynamics (SPH) method with variable smoothing length is proposed, in order to simulate the blood flow simulation effect in the virtual surgery training system. Finally, the two are combined to be applied to the kidney soft tissue cutting experiment in surgery trainings. Experiments show there is a significant improvement on the cutting and simulation effect in terms of the viscoelasticity of the soft tissue cutting and the pressure and viscous force of blood flow.