Computer-assisted Surgical Systems Research Articles

In silico analysis of the patient-specific acetabular cup anteversion safe zone

IntroductionVarious computer-assisted surgical systems claim to improve the accuracy of cup placement in total hip arthroplasties after assessing spinopelvic mobility to prevent prosthetic impingement. However, no study has yet analyzed the extent of the patient-specific cup anteversion safe zones. HypothesisWe hypothesized that most patients have a safe zone >10 °, except those with abnormal spinopelvic mobility, who have a much narrower safe zone. Materials and methodsWe simulated the risks of prosthetic impingement using the planned cup anteversion. The consecutive cohort included 341 patients who underwent total hip arthroplasty. Our primary endpoint was the patient-specific impingement-free zone for cup anteversion, which was then divided into four subgroups: 0 °, 1 ° to 5 °, 6 ° to 10 °, and >10 °. This data was then secondarily analyzed for abnormal spinopelvic mobility (the difference in the spinopelvic tilt [ΔSPT] from a standing to a flexed seated position >20 °). ResultsThe mean anteversion safe zone was 22.8 ° with 82.4% (281/341) of patients with a zone strictly >10 °. The mean safe zone was 8.9 ° (+/− 9 °) in patients with an ΔSPT ≥20 ° (18.2%), with 37.1% of these patients having a zone of 0 °, 16.13% a zone between 1 ° and 5 °, 8.06% a zone between 6 ° and 10 ° and 38.71% a zone >10 °. The mean safe zone was 25.9 ° (+/− 9 °) in patients with an ΔSPT <20 ° (81.8%), and the proportion of cases in each zone was 2.51%, 1.08%, 4.3%, and 92.11%, respectively (p < 0.001). ConclusionThe safe zone for anteversion appears to be fairly wide in most patients. However, identifying patients at risk of abnormal spinopelvic mobility seems necessary to identify the two-thirds of patients with a narrow safe zone. Level of evidenceIV; retrospective study

Mixed Reality Surgical Navigation System; Positional Accuracy Based on Food and Drug Administration Standard.

Computer assisted surgical navigation systems are designed to improve outcomes by providing clinicians with procedural guidance information. The use of new technologies, such as mixed reality, offers the potential for more intuitive, efficient, and accurate procedural guidance. The goal of this study is to assess the positional accuracy and consistency of a clinical mixed reality system that utilizes commercially available wireless head-mounted displays (HMDs), custom software, and localization instruments. Independent teams using the second-generation Microsoft HoloLens© hardware, Medivis SurgicalAR© software, and localization instruments, tested the accuracy of the combined system at different institutions, times, and locations. The ASTM F2554-18 consensus standard for computer-assisted surgical systems, as recognized by the U.S. FDA, was utilized to measure the performance. 288 tests were performed. The system demonstrated consistent results, with an average accuracy performance that was better than one millimeter (.75 ± SD .37mm). Independently acquired positional tracking accuracies exceed conventional in-market surgical navigation tracking systems and FDA standards. Importantly, the performance was achieved at two different institutions, using an international testing standard, and with a system that included a commercially available off-the-shelf wireless head mounted display and software.

Minimal data requirement for realistic endoscopic image generation with Stable Diffusion.

Computer-assisted surgical systems provide support information to the surgeon, which can improve the execution and overall outcome of the procedure. These systems are based on deep learning models that are trained on complex and challenging-to-annotate data. Generating synthetic data can overcome these limitations, but it is necessary to reduce the domain gap between real and synthetic data. We propose a method for image-to-image translation based on a Stable Diffusion model, which generates realistic images starting from synthetic data. Compared to previous works, the proposed method is better suited for clinical application as it requires a much smaller amount of input data and allows finer control over the generation of details by introducing different variants of supporting control networks. The proposed method is applied in the context of laparoscopic cholecystectomy, using synthetic and real data from public datasets. It achieves a mean Intersection over Union of 69.76%, significantly improving the baseline results (69.76 vs. 42.21%). The proposed method for translating synthetic images into images with realistic characteristics will enable the training of deep learning methods that can generalize optimally to real-world contexts, thereby improving computer-assisted intervention guidance systems.

Intraoperative estimation of liver boundary conditions from multiple partial surfaces

PurposeA computer-assisted surgical system must provide up-to-date and accurate information of the patient’s anatomy during the procedure to improve clinical outcome. It is therefore essential to consider the tissue deformations, and a patient-specific biomechanical model (PBM) is usually adopted. The predictive capability of the PBM is highly influenced by proper definition of attachments to the surrounding anatomy, which are difficult to estimate preoperatively.MethodsWe propose to predict the location of attachments using a deep neural network fed with multiple partial views of the intraoperative deformed organ surface directly encoded as point clouds. Compared to previous works, providing a sequence of deformed views as input allows the network to consider the temporal evolution of deformations and to handle the intrinsic ambiguity of estimating attachments from a single view.ResultsThe method is applied to computer-assisted hepatic surgery and tested on both a synthetic and in vivo human open-surgery scenario. The network is trained on a patient-specific synthetic dataset in less than 5 h and produces a more accurate intraoperative estimation of attachments than applying the ones generally used in liver surgery (i.e., fixing vena cava or falciform ligament). The obtained results show 26% more accurate predictions than other solution previously proposed.ConclusionsTrained with patient-specific simulated data, the proposed network estimates the attachments in a fast and accurate manner also considering the temporal evolution of the deformations, improving patient-specific intraoperative guidance in computer-assisted surgical systems.

PEg TRAnsfer Workflow recognition challenge report: Do multimodal data improve recognition?

Background and objective: In order to be context-aware, computer-assisted surgical systems require accurate, real-time automatic surgical workflow recognition. In the past several years, surgical video has been the most commonly-used modality for surgical workflow recognition. But with the democratization of robot-assisted surgery, new modalities, such as kinematics, are now accessible. Some previous methods use these new modalities as input for their models, but their added value has rarely been studied. This paper presents the design and results of the “PEg TRAnsfer Workflow recognition” (PETRAW) challenge with the objective of developing surgical workflow recognition methods based on one or more modalities and studying their added value.Methods: The PETRAW challenge included a data set of 150 peg transfer sequences performed on a virtual simulator. This data set included videos, kinematic data, semantic segmentation data, and annotations, which described the workflow at three levels of granularity: phase, step, and activity. Five tasks were proposed to the participants: three were related to the recognition at all granularities simultaneously using a single modality, and two addressed the recognition using multiple modalities. The mean application-dependent balanced accuracy (AD-Accuracy) was used as an evaluation metric to take into account class balance and is more clinically relevant than a frame-by-frame score.Results: Seven teams participated in at least one task with four participating in every task. The best results were obtained by combining video and kinematic data (AD-Accuracy of between 93% and 90% for the four teams that participated in all tasks).Conclusion: The improvement of surgical workflow recognition methods using multiple modalities compared with unimodal methods was significant for all teams. However, the longer execution time required for video/kinematic-based methods(compared to only kinematic-based methods) must be considered. Indeed, one must ask if it is wise to increase computing time by 2000 to 20,000% only to increase accuracy by 3%. The PETRAW data set is publicly available at www.synapse.org/PETRAW to encourage further research in surgical workflow recognition.

Global–local multi-stage temporal convolutional network for cataract surgery phase recognition

BackgroundSurgical video phase recognition is an essential technique in computer-assisted surgical systems for monitoring surgical procedures, which can assist surgeons in standardizing procedures and enhancing postsurgical assessment and indexing. However, the high similarity between the phases and temporal variations of cataract videos still poses the greatest challenge for video phase recognition.MethodsIn this paper, we introduce a global–local multi-stage temporal convolutional network (GL-MSTCN) to explore the subtle differences between high similarity surgical phases and mitigate the temporal variations of surgical videos. The presented work consists of a triple-stream network (i.e., pupil stream, instrument stream, and video frame stream) and a multi-stage temporal convolutional network. The triple-stream network first detects the pupil and surgical instruments regions in the frame separately and then obtains the fine-grained semantic features of the video frames. The proposed multi-stage temporal convolutional network improves the surgical phase recognition performance by capturing longer time series features through dilated convolutional layers with varying receptive fields.ResultsOur method is thoroughly validated on the CSVideo dataset with 32 cataract surgery videos and the public Cataract101 dataset with 101 cataract surgery videos, outperforming state-of-the-art approaches with 95.8% and 96.5% accuracy, respectively.ConclusionsThe experimental results show that the use of global and local feature information can effectively enhance the model to explore fine-grained features and mitigate temporal and spatial variations, thus improving the surgical phase recognition performance of the proposed GL-MSTCN.

Perpendicular probing and screwing technique: A simple method for accurate pedicle screw placement based on the human internal reference frame for angle estimation.

The pedicle screw (PS) is widely used for spinal fixation surgery. However, PS malpositioning can cause critical complications; thus, the accuracy of ascertaining PS trajectory is paramount. This study aimed to demonstrate the accuracy and safety of a simple and cost-effective PS placement technique using a human internal reference frame for angle estimation. Ex vivo lumbar porcine spine samples were fixed to a wooden board with rostrocaudal and mediolateral rotational angles adjusted by two angle vises. PS entry points (EPs) were identified using clear anatomical vertebral landmarks. PS placement was performed on one side using the perpendicular probing and screwing technique (PPST), wherein the attitude angle of the sample was adjusted such that the longitudinal axis of the target pedicle was perpendicular to the ground. The pedicle probe and PS driver were manually maintained perpendicular to the ground during probing and PS placement. PS placement on the contralateral side was performed freehand as a control. Offsets between the preoperatively planned and implanted PS rotational angles measured using computed tomography for PPST and freehand method were analyzed. Pedicle wall penetration was also evaluated. The mean ± standard error of the medial rotational offsets was 5.83° ± 0.57° in the freehand group versus 2.89° ± 0.31° in the PPST group (p <0.001), and the rostrocaudal rotational offsets were 4.81° ± 0.65° in the freehand group versus 2.92° ± 0.45° in the PPST group (p = 0.01). The mean pedicle wall penetration distance was significantly reduced by PPST (0.28 ± 0.12 mm vs 0.80 ± 0.17 mm in the freehand group, p = 0.0071). Thus, PPST improved PS positioning accuracy, resulting in reduced pedicle wall penetration and increased PS placement safety. This simple technique is also potentially cost-effective for institutions without computer-assisted surgical systems.

Comparison of the accuracy of two different dynamic navigation system registration methods for dental implant placement: A retrospective study.

Dynamic navigation approaches are widely employed in the context of implant placement surgery, with registration being integral to the accuracy of such navigation. Relatively few studies to date, however, have compared different registration approaches, and such a comparison has the potential to guide the development of more accurate and reliable clinical registration methodology. This study was developed to compare the accuracy of dynamic navigation-based dental implant placement conducted using either U-tube or cusp registration methods. Medical records from all patients that had undergone implant surgery between August 2019 and October 2020 in the First Clinical Division of the Peking University Hospital of Stomatology were retrospectively reviewed, with 64 patients and 99 implants ultimately meeting with study inclusion criteria. Implant placement accuracy was gauged via the superimposition of the planned implant position in preoperative cone-beam computed tomography (CBCT) images with the true postoperative implant position in postoperative CBCT images. Accuracy was measured based upon the angular deviation, entry deviation (3-dimensional [3D] deviation in the coronal aspect of the alveolar ridge), and apex deviation (3D deviation in the apical area of the implant) when comparing these two positions. The angular deviation, entry deviation, and apex deviation of all analyzed implants were 3.29 ± 0.17°, 1.29 ± 0.07 mm, and 1.43 ± 0.08 mm, respectively, while in the cusp registration group these respective values were 3.25 ± 1.58°, 1.28 ± 0.60 mm, and 1.34 ± 0.63 mm as compared to 3.35 ± 1.78°, 1.30 ± 0.78 mm, 1.55 ± 0.9 mm in the U-tube group, respectively. No significant differences in accuracy were observed when comparing these two registration techniques. Dynamic computer-assisted surgical systems can facilitate accurate implantation, and both the U-tube and cusp registration methods exhibit similar levels of accuracy. As the cusp registration technique can overcome some of the limitations of the U-tube strategy without the need for an additional registration device, it may be more convenient for clinical use and warrants further research.

The cassandra paradox: looking into the crystal Ball of radiomics in thoracic surgery

Artificial Intelligence Surgery is a surgical journal that focuses on the interface of computers and the art of surgery. The aim of this journal is to create a space for discourse in terms of topics related to artificial intelligence (AI) in surgery such as smart surgical technology/digital surgery, computer-assisted surgical systems and surgical data science. The intention research direction on robotics includes clinical and technological work on tele-manipulation, robotic-assisted technologies, robotic Natural Orifice Transluminal Endoscopic Surgery (NOTES) and fully autonomous robots.

Pragmatism in transforming surgical workplace with artificial intelligence

Artificial Intelligence Surgery is a surgical journal that focuses on the interface of computers and the art of surgery. The aim of this journal is to create a space for discourse in terms of topics related to artificial intelligence (AI) in surgery such as smart surgical technology/digital surgery, computer-assisted surgical systems and surgical data science. The intention research direction on robotics includes clinical and technological work on tele-manipulation, robotic-assisted technologies, robotic Natural Orifice Transluminal Endoscopic Surgery (NOTES) and fully autonomous robots.

Why Artificial Intelligence Surgery (AIS) is better than current Robotic-Assisted Surgery (RAS)

Artificial Intelligence Surgery is a surgical journal that focuses on the interface of computers and the art of surgery. The aim of this journal is to create a space for discourse in terms of topics related to artificial intelligence (AI) in surgery such as smart surgical technology/digital surgery, computer-assisted surgical systems and surgical data science. The intention research direction on robotics includes clinical and technological work on tele-manipulation, robotic-assisted technologies, robotic Natural Orifice Transluminal Endoscopic Surgery (NOTES) and fully autonomous robots.

Applications of machine learning in surgery: ethical considerations

Artificial Intelligence Surgery is a surgical journal that focuses on the interface of computers and the art of surgery. The aim of this journal is to create a space for discourse in terms of topics related to artificial intelligence (AI) in surgery such as smart surgical technology/digital surgery, computer-assisted surgical systems and surgical data science. The intention research direction on robotics includes clinical and technological work on tele-manipulation, robotic-assisted technologies, robotic Natural Orifice Transluminal Endoscopic Surgery (NOTES) and fully autonomous robots.

MIcro-surgical anastomose workflow recognition challenge report

Background and Objective: Automatic surgical workflow recognition is an essential step in developing context-aware computer-assisted surgical systems. Video recordings of surgeries are becoming widely accessible, as the operational field view is captured during laparoscopic surgeries. Head and ceiling mounted cameras are also increasingly being used to record videos in open surgeries. This makes videos a common choice in surgical workflow recognition. Additional modalities, such as kinematic data captured during robot-assisted surgeries, could also improve workflow recognition. This paper presents the design and results of the MIcro-Surgical Anastomose Workflow recognition on training sessions (MISAW) challenge whose objective was to develop workflow recognition models based on kinematic data and/or videos.Methods: The MISAW challenge provided a data set of 27 sequences of micro-surgical anastomosis on artificial blood vessels. This data set was composed of videos, kinematics, and workflow annotations. The latter described the sequences at three different granularity levels: phase, step, and activity. Four tasks were proposed to the participants: three of them were related to the recognition of surgical workflow at three different granularity levels, while the last one addressed the recognition of all granularity levels in the same model. We used the average application-dependent balanced accuracy (AD-Accuracy) as the evaluation metric. This takes unbalanced classes into account and it is more clinically relevant than a frame-by-frame score.Results: Six teams participated in at least one task. All models employed deep learning models, such as convolutional neural networks (CNN), recurrent neural networks (RNN), or a combination of both. The best models achieved accuracy above 95%, 80%, 60%, and 75% respectively for recognition of phases, steps, activities, and multi-granularity. The RNN-based models outperformed the CNN-based ones as well as the dedicated modality models compared to the multi-granularity except for activity recognition.Conclusion: For high levels of granularity, the best models had a recognition rate that may be sufficient for applications such as prediction of remaining surgical time. However, for activities, the recognition rate was still low for applications that can be employed clinically. The MISAW data set is publicly available at http://www.synapse.org/MISAW to encourage further research in surgical workflow recognition.

What is Artificial Intelligence Surgery?

Artificial Intelligence Surgery is a surgical journal that focuses on the interface of computers and the art of surgery. The aim of this journal is to create a space for discourse in terms of topics related to artificial intelligence (AI) in surgery such as smart surgical technology/digital surgery, computer-assisted surgical systems and surgical data science. The intention research direction on robotics includes clinical and technological work on tele-manipulation, robotic-assisted technologies, robotic Natural Orifice Transluminal Endoscopic Surgery (NOTES) and fully autonomous robots.

COVID-19 in Iceland: the rising role of artificial intelligence in public health

Artificial Intelligence Surgery is a surgical journal that focuses on the interface of computers and the art of surgery. The aim of this journal is to create a space for discourse in terms of topics related to artificial intelligence (AI) in surgery such as smart surgical technology/digital surgery, computer-assisted surgical systems and surgical data science. The intention research direction on robotics includes clinical and technological work on tele-manipulation, robotic-assisted technologies, robotic Natural Orifice Transluminal Endoscopic Surgery (NOTES) and fully autonomous robots.

Statement on artificial intelligence surgery by women-in-surgery - Italia: can artificial intelligence be the great equalizer in surgery?

Artificial Intelligence Surgery is a surgical journal that focuses on the interface of computers and the art of surgery. The aim of this journal is to create a space for discourse in terms of topics related to artificial intelligence (AI) in surgery such as smart surgical technology/digital surgery, computer-assisted surgical systems and surgical data science. The intention research direction on robotics includes clinical and technological work on tele-manipulation, robotic-assisted technologies, robotic Natural Orifice Transluminal Endoscopic Surgery (NOTES) and fully autonomous robots.

Markerless Navigation System for Orthopaedic Knee Surgery: A Proof of Concept Study

Current computer-assisted surgical navigation systems mainly rely on optical markers screwed into the bone for anatomy tracking. The insertion of these percutaneous markers increases operating complexity and causes additional harm to the patient. A markerless tracking and registration algorithm has recently been proposed to avoid anatomical markers for knee surgery. The femur points were directly segmented from the recorded RGBD scene by a neural network and then registered to a pre-scanned femur model for the real-time pose. However, in a practical setup such a method can produce unreliable registration results, especially in rotation. Furthermore, its potential application in surgical navigation has not been demonstrated. In this paper, we first improved markerless registration accuracy by adopting a bounded-ICP (BICP) technique, where an estimate of the remote hip centre, acquired also in a markerless way, was employed to constrain distal femur alignment. Then, a proof-of-concept markerless navigation system was proposed to assist in typical knee drilling tasks. Two example setups for global anchoring were proposed and tested on a phantom leg. Our BICP-based markerless tracking and registration method has better angular accuracy and stability than the original method, bringing our straightforward, less invasive markerless navigation approach one step closer to clinical application. According to user tests, our proposed optically anchored navigation system achieves comparable accuracy with the state-of-the-art (3.64± 1.49 mm in position and 2.13±0.81° in orientation). Conversely, our visually anchored, optical tracker-free setup has a lower accuracy (5.86± 1.63 mm in position and 4.18±1.44° in orientation), but is more cost-effective and flexible in the operating room.

Offline identification of surgical deviations in laparoscopic rectopexy.

According to a meta-analysis of 7 studies, the median number of patients with at least one adverse event during the surgery is 14.4%, and a third of those adverse events were preventable. The occurrence of adverse events forces surgeons to implement corrective strategies and, thus, deviate from the standard surgical process. Therefore, it is clear that the automatic identification of adverse events is a major challenge for patient safety. In this paper, we have proposed a method enabling us to identify such deviations. We have focused on identifying surgeons' deviations from standard surgical processes due to surgical events rather than anatomic specificities. This is particularly challenging, given the high variability in typical surgical procedure workflows. We have introduced a new approach designed to automatically detect and distinguish surgical process deviations based on multi-dimensional non-linear temporal scaling with a hidden semi-Markov model using manual annotation of surgical processes. The approach was then evaluated using cross-validation. The best results have over 90% accuracy. Recall and precision for event deviations, i.e. related to adverse events, are respectively below 80% and 40%. To understand these results, we have provided a detailed analysis of the incorrectly-detected observations. Multi-dimensional non-linear temporal scaling with a hidden semi-Markov model provides promising results for detecting deviations. Our error analysis of the incorrectly-detected observations offers different leads in order to further improve our method. Our method demonstrated the feasibility of automatically detecting surgical deviations that could be implemented for both skill analysis and developing situation awareness-based computer-assisted surgical systems.

Implant placement accuracy in total knee arthroplasty: validation of a CT-based measurement technique.

The primary goal of many computer-assisted surgical systems like robotics for total knee arthroplasty (TKA) is to accurately execute a preoperative plan. To assess whether the preoperative plan was executed accurately in 3D, one option is to compare the planned and postoperative implant placement using a preoperative and postoperative CT scan of the patient's limb. This comparison requires a 3D-to-3D surface registration between the preoperative and postoperative 3D bone models and between the planned and postoperative 3D implants. Hence, the present study aimed at validating this measurement technique by determining (I) the anatomical regions that result in the lowest 6-degree of freedom (DoF) errors for 3D-to-3D surface registration of bone models, (II) the 6-DoF errors for 3D-to-3D surface registration of the implant models, and (III) the 6-DoF of the complete measurement technique. Four different regions of the femur were tested to determine which one would result in the most accurate 3D-to-3D registration of the bone models using 12 cadaveric lower limb specimens. Next, total knee arthroplasties were performed on six specimens, and the accuracy of the 3D-to-3D implant registration was evaluated against a gold standard registration performed using fiducial markers. The most accurate 3D-to-3D bone registration was obtained when using the largest anatomical regions available after TKA, being the full 3D femur model or the femur model without the distal femur which resulted in root mean square errors within 0.2 mm for translations and 0.2° for rotation. The accuracy of the 3D-to-3D femoral and tibial implant registration was within 0.7 mm for translations and 0.4°-0.6° for rotations, respectively. The accuracy for the overall procedure was within 0.9 mm and 0.6° for both femur and tibia when using femoral regions resulting in accurate 3D-to-3D bone registration. In conclusion, this measurement technique can be used in applications where measurement errors up to 0.9 mm in translations and up to 0.6° in rotations in component placement are acceptable.

Application of computer-assisted system in surgery for pediatric solid pseudopapillary tumor

Objective To explore the value of Hisense computer-assisted surgical systems (CAS) for precise surgery of pediatric solid pseudopapillary tumor. Methods A total of 5 cases with pancreatic solid pseudopapi-llary tumor who were admitted at the Affiliated Hospital of Qingdao University from June 2015 to September 2018 were adopting.Upper abdominal 64-slice dynamic enhanced computed tomography (CT) scan was performed.3D models were created by computer-assisted surgery systems.Based on 3D model, surgical planning, preoperative simulated tumor resection, intraoperative assisted guidance were performed.Operation time, intraoperative blood loss volume, blood transfusion rate were analyzed. Results Hisense CAS three-dimensional reconstruction could clearly show the adjacent relationship between pancreas, tumor and peripheral vascular organs.According to the preoperative virtual resection, pancreatic tumor resection was more accurate.Postoperative pathological results were solid pseudopapillary tumor of the pancreas.Among them, 2 tumors were located in the head of the pancreas, 1 case was located in the pancreatic neck, and 2 cases in the tail of the pancreas.The operation time was 150-360 min, with an average of 279 min.The average intraoperative blood loss was 40 mL, of which the minimum amount of bleeding was 5 mL, and the blood transfusion rate was 40%(2/5 cases). Surgical tumor removal was achieved successfully in 5 cases.All children were followed up for 6 months to 3 years, and no recurrence or metastasis was observed. Conclusions Three-dimensional reconstruction of computer-assisted surgery system can clearly show the adjacent relationship between tumor and surroun-ding vascular organs, and help to make the best surgical plan before surgery to improve the accuracy and safety of the operation. Key words: Pancreatic solid pseudopapillary tumor; Surgery; Computer-assisted; Imaging; Three-dimensional