Computer Aided Surgery System Research Articles

Surgical step recognition in laparoscopic distal gastrectomy using artificial intelligence: a proof-of-concept study.

Laparoscopic distal gastrectomy (LDG) is a difficult procedure for early career surgeons. Artificial intelligence (AI)-based surgical step recognition is crucial for establishing context-aware computer-aided surgery systems. In this study, we aimed to develop an automatic recognition model for LDG using AI and evaluate its performance. Patients who underwent LDG at our institution in 2019 were included in this study. Surgical video data were classified into the following nine steps: (1) Port insertion; (2) Lymphadenectomy on the left side of the greater curvature; (3) Lymphadenectomy on the right side of the greater curvature; (4) Division of the duodenum; (5) Lymphadenectomy of the suprapancreatic area; (6) Lymphadenectomy on the lesser curvature; (7) Division of the stomach; (8) Reconstruction; and (9) From reconstruction to completion of surgery. Two gastric surgeons manually assigned all annotation labels. Convolutional neural network (CNN)-based image classification was further employed to identify surgical steps. The dataset comprised 40 LDG videos. Over 1,000,000 frames with annotated labels of the LDG steps were used to train the deep-learning model, with 30 and 10 surgical videos for training and validation, respectively. The classification accuracies of the developed models were precision, 0.88; recall, 0.87; F1 score, 0.88; and overall accuracy, 0.89. The inference speed of the proposed model was 32 ps. The developed CNN model automatically recognized the LDG surgical process with relatively high accuracy. Adding more data to this model could provide a fundamental technology that could be used in the development of future surgical instruments.

A user-friendly system for identifying the optimal insertion direction and to choose the best pedicle screws for patient-specific spine surgery

Background and objectiveMany diseases of the spine require surgical treatments that are currently performed based on the experience of the surgeon. The basis of this study is to deliver an automatic and patient-specific algorithm able to come to the aid of the surgeons in pedicle arthrodesis operations, by finding the optimal direction of the screw insertion, the maximum screw diameter and the maximum screw length. ResultsThe paper introduce an algorithm based on the reconstructed geometry of a vertebra by 3D-scan that is able to identify the best introduction direction for screw and to select, from commercial and/or personalised databases, the best screws in order to maximize the occupation of the bone while not intersecting each other and not going through the walls of the pedicle and the bounds of the vertebral body. In fact, for pedicle arthrodesis surgery, the incorrect positioning of the screws may cause operating failures, an increase in the overall duration of surgery and, therefore, more harmful, real-time X-ray checks. In case of not availability on market, the algorithm also suggests parameters for designing and manufacturing an ‘ad hoc’ solution. The algorithm has been tested on 6 vertebras extracted by a medical database. Furthermore, the algorithm is based on a procedure through which the surgeon can freely choose the entering point of the screw (based on his/her own experience and will). A real patient vertebra has been processed with almost 400 different entering point, always giving a feedback on the possibility to use the entering point (in case of unavailability of a good trajectory) and on the individuation of the right trajectory and the choose of the better screws. ConclusionsIn very recent bibliography, several papers deal with procedure to screw’ trajectory planning in arthrodesis surgery by using Computer Aided surgery systems, and some of them used also modern methodologies (KBE, AI, Deep learning, etc.) methods for planning the surgery as better as possible. Nevertheless, no methodologies or algorithm have been still realized to plan the trajectory and choose the perfect fitting screws on the basis of the patient-specific vertebra. This paper represents a wind of novelty in this field and allow surgeons to use the proposed algorithm for planning their surgeries. Finally, it allows also the easy creation of a customized surgical template, characterized by two cylindrical guides that follow a correct trajectory previously calculated by means of that automatic algorithm generated on the basis of a vertebra CAD model for a specific patient. The surgeon will be able to set the template (drilling guides) on the patient's vertebra and safely apply the screws.

Towards better laparoscopic video segmentation: A class-wise contrastive learning approach with multi-scale feature extraction.

The task of segmentation is integral to computer-aided surgery systems. Given the privacy concerns associated with medical data, collecting a large amount of annotated data for training is challenging. Unsupervised learning techniques, such as contrastive learning, have shown powerful capabilities in learning image-level representations from unlabelled data. This study leverages classification labels to enhance the accuracy of the segmentation model trained on limited annotated data. The method uses a multi-scale projection head to extract image features at various scales. The partitioning method for positive sample pairs is then improved to perform contrastive learning on the extracted features at each scale to effectively represent the differences between positive and negative samples in contrastive learning. Furthermore, the model is trained simultaneously with both segmentation labels and classification labels. This enables the model to extract features more effectively from each segmentation target class and further accelerates the convergence speed. The method was validated using the publicly available CholecSeg8k dataset for comprehensive abdominal cavity surgical segmentation. Compared to select existing methods, the proposed approach significantly enhances segmentation performance, even with a small labelled subset (1-10%) of the dataset, showcasing a superior intersection over union (IoU)score.

Clinical application of bronchial high-frequency ventilation in 2-port thoracoscopic segmentectomy.

To evaluate the safety and clinical application of a computer-aided surgery system (CAS) combined with high-frequency bronchial ventilation in 2-port thoracoscopic anatomical segmentectomy. A total of 301 patients who underwent 2-port thoracoscopic segmentectomy between January 1, 2019 and March 1, 2022 in the 960th Hospital of the People's Liberation Army and the Department of Thoracic Surgery of Zibo Municipal Hospital were retrospectively analyzed. The experimental and control groups were created according to the different methods of appearing the intersegmental plane of the lung. The experimental group comprised 152 patients who underwent CAS reconstruction combined with high-frequency ventilation, and the control group comprised 149 patients who underwent CAS reconstruction combined with expansion collapse. The characteristics of the patients, including age, sex, smoking history, forced expiratory volume in 1 second/forced vital capacity, Maximal ventilation, diameter of pulmonary nodules, intraoperative blood loss, postoperative drainage volume, drainage tube removal time, length of hospital stay after extubation, postoperative complication rate, operation time and appearance time of the intersegmental plane, were compared between the 2 groups. All patients completed the operation between high-frequency bronchial ventilation and expansion collapse group. There was no significant difference in Forced expiratory volume in 1 second/Forced vital capacity [(101.05 ± 11.86) vs (101.86 ± 11.61)], maximum expiratory volume [(86.36 ± 17.59 L) vs (85.28 ± 17.68 L)], the diameter of lung nodules [(13.61 ± 3.51 cm) vs (13.21 ± 3.41 cm)], intraoperative blood loss [(47.50 ± 45.90 mL) vs (48.49 ± 34.65 mL)], postoperative drainage volume [(425.16 ± 221.61 mL) vs (444.70 ± 243.72 mL)], drainage tube removal time [(3.88 ± 1.85 days) vs (3.43 ± 1.81 days)], or postoperative hospital stay [(6.07 ± 2.14 days) vs (5.82 ± 1.88 days) between the experimental group and the control group (P > .05)]. There were significant differences in operation time [(95.05 ± 26.85 min) vs (117.85 ± 31.70 min), P = .017] and intersegmental plane appearance time [(2.37 ± 1.03 min) vs (14.20 ± 3.23 min), P < .001]. High-frequency bronchial ventilation is safe and feasible when used in quickly and accurately identifying the intersegmental plane and is worthy of clinical application in 2-port thoracoscopic segmentectomy.

Application of three-dimensional visualization technology in the anatomical variations of hilar bile ducts in Chinese population.

This study aimed to establish three-dimensional models of the biliary tract of Chinese people using the Hisense computer-aided surgery (CAS) system and to explore the branching patterns and variation types of the biliary system under the study of 3D reconstruction of the biliary tract. Three-dimensional models of the biliary tract were reconstructed in 50 patients using the Hisense CAS system. The branching patterns of intrahepatic bile ducts were observed. The biliary tract was classified according to the confluence of the right posterior sectoral duct (RPSD), right anterior sectoral duct (RASD) and left hepatic duct (LHD), and the presence or absence of accessory hepatic ducts. The 3D models of the bile ducts were successfully reconstructed in 50 Chinese patients. The branching patterns of the bile ducts were classified into seven types. The anatomy of the bile ducts was typical in 54% of cases (n = 27), showed triple confluence in 10% (n = 5), and crossover anomaly in 14% (n = 7), which means anomalous drainage of the RPSD into the LHD, anomalous drainage of the RPSD into the common hepatic duct (CHD) in 10% (n = 5), anomalous drainage of the RPSD into the cystic duct (CD) in 2% (n = 1), absence of left main hepatic duct in 1% (n = 1), presence of accessory duct in 8% (n = 4). Among them, there were three cases of accessory hepatic ducts coexisting with other variation types. By using the Hisense CAS system to establish 3D models of the biliary tract of the Chinese people, we established the branching model of the second-order bile ducts, which has important value for the classification of the biliary system and its variation types.

MVD-Net: Semantic Segmentation of Cataract Surgery Using Multi-View Learning.

Semantic segmentation of surgery scenarios is a fundamental task for computer-aided surgery systems. Precise segmentation of surgical instruments and anatomies contributes to capturing accurate spatial information for tracking. However, uneven reflection and class imbalance lead the segmentation in cataract surgery to a challenging task. To desirably conduct segmentation, a network with multi-view decoders (MVD-Net) is proposed to present a generalizable segmentation for cataract surgery. Two discrepant decoders are implemented to achieve multi-view learning with the backbone of U-Net. The experiment is carried out on the Cataract Dataset for Image Segmentation (CaDIS). The ablation study verifies the effectiveness of the proposed modules in MVD-Net, and superior performance is provided by MVD-Net in the comparison with the state-of-the-art methods. The source code will be publicly released.

Surgical instrument segmentation based on multi-scale and multi-level feature network.

Surgical instrument segmentation is critical for the field of computer-aided surgery system. Most of deep-learning based algorithms only use either multi-scale information or multi-level information, which may lead to ambiguity of semantic information. In this paper, we propose a new neural network, which extracts both multi-scale and multilevel features based on the backbone of U-net. Specifically, the cascaded and double convolutional feature pyramid is input into the U-net. Then we propose a DFP (short for Dilation Feature-Pyramid) module for decoder which extracts multi-scale and multi-level information. The proposed algorithm is evaluated on two publicly available datasets, and extensive experiments prove that the five evaluation metrics by our algorithm are superior than other comparing methods.

A framework for modelling the biomechanical behaviour of the human liver during breathing in real time using machine learning

Progress in biomechanical modelling of human soft tissue is the basis for the development of new clinical applications capable of improving the diagnosis and treatment of some diseases (e.g. cancer), as well as the surgical planning and guidance of some interventions. The finite element method (FEM) is one of the most popular techniques used to predict the deformation of the human soft tissue due to its high accuracy. However, FEM has an associated high computational cost, which makes it difficult its integration in real-time computer-aided surgery systems. An alternative for simulating the mechanical behaviour of human organs in real time comes from the use of machine learning (ML) techniques, which are much faster than FEM. This paper assesses the feasibility of ML methods for modelling the biomechanical behaviour of the human liver during the breathing process, which is crucial for guiding surgeons during interventions where it is critical to track this deformation (e.g. some specific kind of biopsies) or for the accurate application of radiotherapy dose to liver tumours. For this purpose, different ML regression models were investigated, including three tree-based methods (decision trees, random forests and extremely randomised trees) and other two simpler regression techniques (dummy model and linear regression). In order to build and validate the ML models, a labelled data set was constructed from modelling the deformation of eight ex-vivo human livers using FEM. The best prediction performance was obtained using extremely randomised trees, with a mean error of 0.07 mm and all the samples with an error under 1 mm. The achieved results lay the foundation for the future development of some real-time software capable of simulating the human liver deformation during the breathing process during clinical interventions.

多元計算解剖に基づいたコンピュータ診断・治療支援システム —研究計画と研究進捗—

多元計算解剖に基づいたコンピュータ診断・治療支援システム —研究計画と研究進捗—

Computer-Aided Diagnosis and Computer-Aided Surgery Systems Based on Multidisciplinary Computational Anatomy

Computer-Aided Diagnosis and Computer-Aided Surgery Systems Based on Multidisciplinary Computational Anatomy

Unscented Kalman Filter Based Sensor Fusion for Robust Optical and Electromagnetic Tracking in Surgical Navigation

Computer-aided surgery systems provide visual guidance to surgeons by showing the real-time pose of surgical instruments overlaid on preoperative medical images of the patient. Surgical instrument poses are localized in space using mainly optical tracking systems (OTS) and electromagnetic tracking systems (EMTS). OTS systems require clear line-of-sight, which is difficult to ensure in current overcrowded operating rooms. On the other hand, EMTSs provide less accuracy and suffer from magnetic field distortion in the presence of metal objects. In this paper, we propose a sensor fusion algorithm to compensate for the drawbacks of OTS and EMTS, and achieve robust tracking of surgical instruments. Spatial alignment of OTS and EMTS data will be achieved through a calibration procedure. The proposed sensor fusion algorithm uses an unscented Kalman filter (UKF), an extension of the standard Kalman filter based on a deterministic sampling of nonlinear functions. Quaternion representation for rotations is used to avoid singularities of other parameterizations (e.g., Euler angles). In cases of optical marker occlusion, our algorithm will take advantage of the EMTS to estimate the position of the hidden marker(s) and feed it in the UKF. The fusion algorithm will also keep track of an error matrix between OTS and EMTS measured poses, providing an up-to-date estimate of the electromagnetic distortion. This will be used to correct EMTS measurement before using it to compensate for possible optical line-of-sight occlusions. The proposed sensor fusion (SF) method is compared to a predictive UKF based on optical data (NSF). Our results show that the SF effectively compensates for short marker occlusion (nine samples) providing a continuous estimate of the instrument pose with an error significantly lower than the NSF method, and reaching a clinically acceptable accuracy. Furthermore, the proposed algorithm increases the accuracy of EMTS in the presence of magnetic field distortion.

Three-dimensional Planning in Orthognathic Surgery using Cone-beam Computed Tomography and Computer Software

The orthognathic surgery is the standard treatment for the correction of dentofacial deformities, in order to get a stable dental occlusion and facial harmony. The advancement of technology and the evolution of the concepts involved in the diagnosis and treatment plan in this area have been immeasurable, leading to the development of new methods, such as computer-aided jaw surgery system by a three-dimensional (3D) virtual surgical planning. The advent of the cone-beam computed tomography (CBCT) allows the acquisition of 3D images of the patient’s craniofacial complex and eliminates the ambiguity that can occur with two-dimensional (2D) images. Surgical simulation in 3D may benefit patients by providing a more accurate treatment plan and streamlined surgery, especially for patients with complex dentofacial deformities. The breakthrough of software tools for the diagnosis and treatment planning allows the construction of 3D surface models, dynamic cephalometry, semi-automatic mirroring in cases of asymmetry, interactive cutting of bone, bony segment repositioning, 3D splint manufacturing, bone reconstruction and visualization and prediction of the changes in hard and soft tissues of the face. The aim of this study was to report a case where the computed-assisted surgical planning predicts the possibility in achieving balance between aesthetic and function. Alterations in the virtual planning allow overcoming obstacles in actual surgery. Considering all possible details, the process provides greater predictability, practicality and precision in surgical planning.

Accuracy study of new computer-assisted orthopedic surgery software

PurposeThe new computerized system is based on image analysis and designed to aid in orthopedic surgeries by virtual trajectory of the guide wire, intra-operative planning and various measurements. Validation of the accuracy and safety of any computer-aided surgery system is essential before implementing it clinically. We examined the accuracy of guide-wire length and angle measurements and fusion of multiple adjacent images (panoramic view image, PVI®) of the new software. MethodsThis is a 2-part study. Part I: twenty guide wires were drilled to various depths in a synthetic femur model and the results obtained by the software measurements were compared with manual measurements by a caliper and a depth gauge. Part II: a sawbone femur shaft was osteotomized and various inclinations of >10° to the varus or valgus angles were tested. The manually obtained measurements of angles and lengths were compared to the new computerized system software PVI. ResultsThere was a significant positive linear correlation between all groups of the computerized length and the control measurements (r>0.983, p<0.01). There was no significant difference among different distances, angles or positions from the image intensifier. There was a significant positive linear correlation between the angle and length measurement on the PVI and the control measurement (r>0.993, p<0.01). ConclusionsThe new computerized software has high reliability in performing measurements of length using an aiming, positioning and referring device intra-operatively.

Regenerative repair of bone defect with osteoinductive hydroxyapatite fabricated to match the defect and implanted with combined use of computer-aided design, manufacturing, and surgery systems: A feasibility study in a canine model

Regenerative repair of bone defect with osteoinductive hydroxyapatite fabricated to match the defect and implanted with combined use of computer-aided design, manufacturing, and surgery systems: A feasibility study in a canine model

Three-dimensional surgical simulation

In this article, we discuss the development of methods for computer-aided jaw surgery, which allows us to incorporate the high level of precision necessary for transferring virtual plans into the operating room. We also present a complete computer-aided surgery system developed in close collaboration with surgeons. Surgery planning and simulation include construction of 3-dimensional surface models from cone-beam computed tomography, dynamic cephalometry, semiautomatic mirroring, interactive cutting of bone, and bony segment repositioning. A virtual setup can be used to manufacture positioning splints for intraoperative guidance. The system provides further intraoperative assistance with a computer display showing jaw positions and 3-dimensional positioning guides updated in real time during the surgical procedure. The computer-aided surgery system aids in dealing with complex cases with benefits for the patient, with surgical practice, and for orthodontic finishing. Advanced software tools for diagnosis and treatment planning allow preparation of detailed operative plans, osteotomy repositioning, bone reconstructions, surgical resident training, and assessing the difficulties of the surgical procedures before the surgery. Computer-aided surgery can make the elaboration of the surgical plan a more flexible process, increase the level of detail and accuracy of the plan, yield higher operative precision and control, and enhance documentation of cases.

Assessment of optical localizer accuracy for computer aided surgery systems

The technology for localization of surgical tools with respect to the patient's reference coordinate system in three to six degrees of freedom is one of the key components in computer aided surgery. Several tracking methods are available, of which optical tracking is the most widespread in clinical use. Optical tracking technology has proven to be a reliable method for intra-operative position and orientation acquisition in many clinical applications; however, the accuracy of such localizers is still a topic of discussion.In this paper, the accuracy of three optical localizer systems, the NDI Polaris P4, the NDI Polaris Spectra (in active and passive mode) and the Stryker Navigation System II camera, is assessed and compared critically.Static tests revealed that only the Polaris P4 shows significant warm-up behavior, with a significant shift of accuracy being observed within 42 minutes of being switched on. Furthermore, the intrinsic localizer accuracy was determined for single markers as well as for tools using a volumetric measurement protocol on a coordinate measurement machine. To determine the relative distance error within the measurement volume, the Length Measurement Error (LME) was determined at 35 test lengths. As accuracy depends strongly on the marker configuration employed, the error to be expected in typical clinical setups was estimated in a simulation for different tool configurations.The two active localizer systems, the Stryker Navigation System II camera and the Polaris Spectra (active mode), showed the best results, with trueness values (mean ± standard deviation) of 0.058 ± 0.033 mm and 0.089 ± 0.061 mm, respectively. The Polaris Spectra (passive mode) showed a trueness of 0.170 ± 0.090 mm, and the Polaris P4 showed the lowest trueness at 0.272 ± 0.394 mm with a higher number of outliers than for the other cameras.The simulation of the different tool configurations in a typical clinical setup revealed that the tracking error can be estimated to be 1.02 mm for the Polaris P4, 0.64 mm for the Polaris Spectra in passive mode, 0.33 mm for the Polaris Spectra in active mode, and 0.22 mm for the Stryker Navigation System II camera.

Accuracy of a newly developed integrated system for dental implant planning

To evaluate the accuracy of the first integrated system for cone-beam CT (CBCT) imaging, dental implant planning and surgical template-aided implant placement. On the basis of CBCT scans, a total of 54 implant positions were planned for 10 partially edentulous anatomical patient-equivalent models. Surgical guides were ordered from the manufacturer (SICAT). Two different types of guidance were assessed: for assessment of the SICAT system inherent accuracy vendor's titanium sleeves of 2 mm internal diameter and 5 mm length were utilized for pilot drills. The guide sleeves of the NobelGuide system were implemented for fully guided surgery and implant insertion. Deviations perpendicular to the implant axes at the crestal and apical end, as well as the angle deviations between the virtual planning data and the surgical results, were measured utilizing a follow-up CBCT investigation and referential marker-based registration. The SICAT system inherent mean deviation rates for the drilled pilot osteotomies were determined to be smaller than 500 mum even at the apical end. Mean angle deviations of 1.18 degrees were determined. Utilizing the NobelGuide sleeve-in-sleeve system for fully guided implant insertion in combination with the investigated template technology enabled to insert dental implants with the same accuracy. Crestal deviations, in general, were significantly lower than the apical deviations. Although hardly comparable due to different study designs and measurement strategies, the investigated SICAT system's inherent accuracy corresponds to the most favourable results for computer-aided surgery systems published so far. In combination with the NobelGuide surgical set for fully guided insertion, the same accuracy level could be maintained for implant positioning.

Comparison of 3 Optical Navigation Systems for Computer-Aided Maxillofacial Surgery

To compare the accuracy of 3 computer-aided surgery systems for maxillofacial reconstruction. Evaluation of 3 computer-aided surgery systems: StealthStation, VectorVision, and Voxim. The University of California, Davis, Department of Otolaryngology computer-aided surgery laboratory. Four fresh cadaveric heads. Mean target registration error. The StealthStation was the most accurate (mean [SD] target registration error, 1.00 [0.04] mm), followed by VectorVision (1.13 [0.05] mm) and then Voxim (1.34 [0.04] mm). All values met statistical significance (P < .05). Measurable accuracy differences were found among the navigation systems evaluated. The StealthStation was the most accurate. However, the differences are small, and the clinical significance for maxillofacial reconstruction is negligible.

A data model applied to the knowledge-based ICAS

Image-guided computer aided surgery system (ICAS) contributes to safeness and success of surgery operations by means of displaying anatomical structures and showing correlative information to surgeons in the process of operation. Based on analysis of requirements for ICAS, a new concept of clinical knowledge-based ICAS was proposed. Designing a reasonable data structure model is essential for realizing this new concept. The traditional data structure is limited in expressing and reusing the clinical knowledge such as locating an anatomical object, topological relations of anatomical objects and correlative clinical attributes. A data structure model called mixed adjacency lists by octree-path-chain (MALOC) was outlined, which can combine patient’s images with clinical knowledge, as well as efficiently locate the instrument and search the objects’ information. The efficiency of data structures was analyzed and experimental results were given in comparison to other traditional data structures. The result of the nasal surgery experiment proves that MALOC is a proper model for clinical knowledge-based ICAS that has advantages in not only locating the operative instrument precisely but also proving surgeons with real-time operation-correlative information. It is shown that the clinical knowledge-based ICAS with MALOC model has advantages in terms of safety and success of surgical operations, and help in accurately locating the operative instrument and providing operation-correlative knowledge and information to surgeons in the process of operations.

Evaluation of the accuracy of three different computer-aided surgery systems in dental implantology: optical tracking vs. stereolithographic splint systems

In dental implant surgery, computer-aided surgery (CAS) techniques can provide a high medical benefit. Two different techniques are established for transferring a CAS treatment planning to the patient: the use of surgical templates (splints) or intraoperative navigation using optical tracking. The aim of this study was to evaluate the total application accuracy of three different CAS systems (Artma virtual patient, RoboDent LapAccedo, Materialise SurgiGuide): two featuring optical tracking, one featuring stereolithographically manufactured splints. A total of 120 implants were placed into 20 human cadaver mandibles. Preoperative computed tomography (CT) scans imported to the corresponding software were used to plan the implant positions on the computer. Implant placement was performed using either optical tracking or stereolithographic splints. Postoperative CT scans were used to obtain the achieved implant positions. A semi-automatic approach was developed to compare planned and achieved implant positions. Deviations between planned and achieved positions were measured for each implant in position (Delta xy), depth (Delta z) and axis (Delta phi). Despite the different techniques of transfer, no statistically significant differences were found between all groups. The accuracy achieved corresponded well with the spatial resolution of the CT Scans used.