Field Of Computer-assisted Surgery Research Articles

Active learning using deep Bayesian networks for surgical workflow analysis.

For many applications in the field of computer-assisted surgery, such as providing the position of a tumor, specifying the most probable tool required next by the surgeon or determining the remaining duration of surgery, methods for surgical workflow analysis are a prerequisite. Often machine learning-based approaches serve as basis for analyzing the surgical workflow. In general, machine learning algorithms, such as convolutional neural networks (CNN), require large amounts of labeled data. While data is often available in abundance, many tasks in surgical workflow analysis need annotations by domain experts, making it difficult to obtain a sufficient amount of annotations. The aim of using active learning to train a machine learning model is to reduce the annotation effort. Active learning methods determine which unlabeled data points would provide the most information according to some metric, such as prediction uncertainty. Experts will then be asked to only annotate these data points. The model is then retrained with the new data and used to select further data for annotation. Recently, active learning has been applied to CNN by means of deep Bayesian networks (DBN). These networks make it possible to assign uncertainties to predictions. In this paper, we present a DBN-based active learning approach adapted for image-based surgical workflow analysis task. Furthermore, by using a recurrent architecture, we extend this network to video-based surgical workflow analysis. To decide which data points should be labeled next, we explore and compare different metrics for expressing uncertainty. We evaluate these approaches and compare different metrics on the Cholec80 dataset by performing instrument presence detection and surgical phase segmentation. Here we are able to show that using a DBN-based active learning approach for selecting what data points to annotate next can significantly outperform a baseline based on randomly selecting data points. In particular, metrics such as entropy and variation ratio perform consistently on the different tasks. We show that using DBN-based active learning strategies make it possible to selectively annotate data, thereby reducing the required amount of labeled training in surgical workflow-related tasks.

Virtual reconstruction of bilateral midfacial defects by using statistical shape modeling

PurposeMirroring and manual adaptation as the main virtual reconstruction method of midfacial defects is time demanding and ignores asymmetrical skull shapes. By using a statistical shape model (SSM), the reconstruction can be automatized and specified. The current study aims to show the ability of the SSM in the virtual reconstruction of artificial bilateral defects. MethodsBased on 131 pathologically unaffected CT scans of the adult midface region, an SSM was created. DICOM data were generated, segmented and registered on one mesh, which serves as template for the SSM. The SSM consists of the registered surface meshes and includes the shape variability of the cranial vault. Fractured or missing parts were calculated by the known shape variability of healthy midface data. Using 25 CT scans not included in the SSM, the precision of the reconstruction of virtually placed bilateral defects of the orbital floor (Group 1) and bilateral naso-orbital-ethmoid (NOE) fractures (Group 2). Distances to the corresponding parts of the intact skull were calculated to show the accuracy of the virtual reconstruction method. ResultsAll defects could be reconstructed by using the SSM and GM technique. The analysis shows a high accuracy of the SSM-driven reconstruction, with a mean error of 0.75 ± 0.18 mm in group 1 and with a mean error of 0.81 ± 0.23 mm in group 2. ConclusionThe precision of the SSM-driven reconstruction is high and its application is easy for the clinician because of the automatization of the virtual reconstruction process in the field of computer-assisted surgery (CAS). Respecting of the natural asymmetry of the skull and the methods of GM are reasons for the high precision and the automatization of the new shown reconstruction workflow.

A hand-eye calibration method for augmented reality applied to computer-assisted orthopedic surgery

Purpose Recent technology advances in Augmented Reality (AR) is creating an opportunity for a paradigm shift in the field of Computer Assisted Surgery (CAS). Despite having the potential of being a promising technology in clinical applications, its accuracy, robustness, and performances under specific environmental conditions have not been fully investigated and evaluated. In an attempt to address the limitations and uncertainties that arise from using computer vision, we propose a Hand-Eye (HE) based approach to determine an invariant mapping between the HoloLens headset (i.e., rigid body defined by a unique set of passive markers) and the OHMDs virtual camera. Methods In order to achieve the required precision for using the Microsoft HoloLens in a surgical application, an invariant mapping was estimated. First, a set of non-collinear retroreflective markers were rigidly attached to the Microsoft HoloLens headset. Consequently, the OHMDs's intrinsic parameters, as well as their radial and tangential lenses distortion coefficients, were determined by solving simultaneously a set of homogeneous system of equations by using nonlinear optimization techniques. The accuracy of the proposed HE based method has been evaluated by using a retro-projection error estimation method. Results Initially, a surgical frame has been anchored to the organ of interest. After loading the patient-specific dataset, a hologram of the organ of interest appears in front of the user's initial position. Correspondences between the hologram and the fiducial markers can then be established by using the tracked surgical pointer and then be mapped with respect to the surgical frame. After finishing this process, an orthonormal basis is created in the physical space, which has its counterpart in the image space and, therefore, a transformation can be determined to map the hologram with respect to the surgical frame. Conclusions OHMDs may be able to create an opportunity for a paradigm shift in the field of CAS, since all this detailed information could easily be displayed on virtual and interactive screens in operating rooms on the same time and in real time. A very important aspect of using such OHMDs is that it can be very intuitive, especially among inexperienced surgeon, since surgical instruments and maneuvers motions are represented with respect to the surgeon's visual system and, therefore, attention disruptions are more likely to be reduced in terms of the surgeon's gaze, considering that the surgeon would not need to be searching for a conventional display located at static location in the operating room.

Review: Current concepts in computer-assisted hip arthroscopy.

In the last 15years, hip arthroscopy has become increasingly popular in addressing femoroacetabular impingement (FAI) because of its minimally invasive approach. However, assessing the adequacy of bone resection when correcting FAI can be difficult because visualisation and spatial awareness of the joint are poor. Recent advances in technology in the field of computer-assisted surgery and navigation and robotic surgery in orthopaedics as a resource for preoperative planning and intraoperative assistance have been widely reported. This technology is expected to upgrade surgical planning and operative techniques, decrease human error and improve patient outcomes by precisely defining the divergent anatomy and kinematics of the hip joint. This review attempts to bring the reader up-to-date with the current developments in the field of computer assisted hip arthroscopy, and discusses our experience with pre-operative planning, navigation and robotics and also provides a platform for future research in this arena.

Complex Pelvic Reconstruction using Patient-Specific Instrumentation and a 3D-Printed Custom Implant following Tumor Resection

AbstractResection of a periacetabular sarcoma is a difficult surgical problem due to the complex bony and soft tissue anatomy. The challenge of pelvic tumor resection lies in achieving clear margins while also reconstructing the defect with a durable construct. Limb salvage surgery involving these types of tumors, as well as other tumors of the pelvis, has made great progress recently due to remarkable technological innovation in the field of computer-assisted surgery and custom implant design. In this study, the authors review the case of a 9-year-old child with an osteosarcoma arising from the proximal femur with local spread into the hip joint and a skip metastasis in the ipsilateral acetabulum. They undertook limb-sparing extra-articular resection of the sarcoma with a reconstruction consisting of a 3D-custom printed pelvic implant and a proximal femur extendible prosthesis. The resection of the pelvis was performed with patient-specific cutting guides, rather than computer-assisted navigation. The resection resulted in a clear margin and the patient has had no evidence of local recurrence. These patient-specific guides closely matched the planned resection in terms of orientation in the pelvis, which allowed for straightforward implantation of the custom prosthesis at the time of surgery. The result has been a durable reconstruction that has produced an excellent functional outcome for this patient.

Planning of skull reconstruction based on a statistical shape model combined with geometric morphometrics.

Virtual reconstruction of large cranial defects is still a challenging task. The current reconstruction procedures depend on the surgeon's experience and skills in planning the reconstruction based on mirroring and manual adaptation. The aim of this study is to propose and evaluate a computer-based approach employing a statistical shape model (SSM) of the cranial vault. An SSM was created based on 131 CT scans of pathologically unaffected adult crania. After segmentation, the resulting surface mesh of one patient was established as template and subsequently registered to the entire sample. Using the registered surface meshes, an SSM was generated capturing the shape variability of the cranial vault. The knowledge about this shape variation in healthy patients was used to estimate the missing parts. The accuracy of the reconstruction was evaluated by using 31 CT scans not included in the SSM. Both unilateral and bilateral bony defects were created on each skull. The reconstruction was performed using the current gold standard of mirroring the intact to the affected side, and the result was compared to the outcome of our proposed SSM-driven method. The accuracy of the reconstruction was determined by calculating the distances to the corresponding parts on the intact skull. While unilateral defects could be reconstructed with both methods, the reconstruction of bilateral defects was, for obvious reasons, only possible employing the SSM-based method. Comparing all groups, the analysis shows a significantly higher precision of the SSM group, with a mean error of 0.47mm compared to the mirroring group which exhibited a mean error of 1.13mm. Reconstructions of bilateral defects yielded only slightly higher estimation errors than those of unilateral defects. The presented computer-based approach using SSM is a precise and simple tool in the field of computer-assisted surgery. It helps to reconstruct large-size defects of the skull considering the natural asymmetry of the cranium and is not limited to unilateral defects.

Computer-assisted Orthopaedic Surgery.

Nowadays, operating rooms can be inefficient and overcrowded. Patient data and images are at times not well integrated and displayed in a timely fashion. This lack of coordination may cause further reductions in efficiency, jeopardize patient safety, and increase costs. Fortunately, technology has much to offer the surgical disciplines and the ongoing and recent operating room innovations have advanced preoperative planning and surgical procedures by providing visual, navigational, and mechanical computerized assistance. The field of computer-assisted surgery (CAS) broadly refers to surgical interface between surgeons and machines. It is also part of the ongoing initiatives to move away from invasive to less invasive or even noninvasive procedures. CAS can be applied preoperatively, intraoperatively, and/or postoperatively to improve the outcome of orthopaedic surgical procedures as it has the potential for greater precision, control, and flexibility in carrying out surgical tasks, and enables much better visualization of the operating field than conventional methods have afforded. CAS is an active research discipline, which brings together orthopaedic practitioners with traditional technical disciplines such as engineering, computer science, and robotics. However, to achieve the best outcomes, teamwork, open communication, and willingness to adapt and adopt new skills and processes are critical. Because of the relatively short time period over which CAS has developed, long-term follow-up studies have not yet been possible. Consequently, this review aims to outline current CAS applications, limitations, and promising future developments that will continue to impact the operating room (OR) environment and the OR in the future, particularly within orthopedic and spine surgery.

Solving challenges in inter- and trans-disciplinary working teams: Lessons from the surgical technology field.

Engineering a medical technology is a complex process, therefore it is important to include experts from different scientific fields. This is particularly true for the development of surgical technology, where the relevant scientific fields are surgery (medicine) and engineering (electrical engineering, mechanical engineering, computer science, etc.). Furthermore, the scientific field of human factors is important to ensure that a surgical technology is indeed functional, process-oriented, effective, efficient as well as user- and patient-oriented. Working in such trans- and inter-disciplinary teams can be challenging due to different working cultures. The intention of this paper is to propose an innovative cooperative working culture for the interdisciplinary field of computer-assisted surgery (CAS) based on more than ten years of research on the one hand and the interdisciplinary literature on working cultures and various organizational theories on the other hand. In this paper, a retrospective analysis of more than ten years of research work in inter- and trans-disciplinary teams in the field of CAS will be performed. This analysis is based on the documented observations of the authors, the study reports, protocols, lab reports and published publications. To additionally evaluate the scientific experience in an interdisciplinary research team, a literature analysis regarding scientific literature on trans- and inter-disciplinarity was performed. Own research and literature analyses were compared. Both the literature and the scientific experience in an interdisciplinary research team show that consensus finding is not always easy. It is, however, important to start trans- and interdisciplinary projects with a shared mental model and common goals, which include communication and leadership issues within the project teams, i.e. clear and unambiguous information about the individual responsibilities and objectives to attain. This is made necessary due to differing leadership cultures within the cooperating disciplines. Another research outcome is the relevance of a cooperative learning culture throughout the complete duration of the project. Based on this cooperation, new ideas and projects were developed, i.e. a training concept for surgical trainers including technological competence for surgeons. An adapted innovative paradigm for a cooperating working culture in CAS is based on a shared mental model and common goals from the very beginning of a project. All actors in trans- and inter-disciplinary teams need to be interested in cooperation. This will lead to a common view on patients and technology models.

Three-dimensional histological specimen preparation for accurate imaging and spatial reconstruction of the middle and inner ear

Purpose This paper presents a highly accurate cross-sectional preparation technique. The research aim was to develop an adequate imaging modality for both soft and bony tissue structures featuring high contrast and high resolution. Therefore, the advancement of an already existing microgrinding procedure was pursued. The central objectives were to preserve spatial relations and to ensure the accurate three-dimensional reconstruction of histological sections.Methods Twelve human temporal bone specimens including middle and inner ear structures were utilized. They were embedded in epoxy resin, then dissected by serial grinding and finally digitalized. The actual abrasion of each grinding slice was measured using a tactile length gauge with an accuracy of one micrometre. The cross-sectional images were aligned with the aid of artificial markers and by applying a feature-based, custom-made auto-registration algorithm. To determine the accuracy of the overall reconstruction procedure, a well-known reference object was used for comparison. To ensure the compatibility of the histological data with conventional clinical image data, the image stacks were finally converted into the DICOM standard.Results The image fusion of data from temporal bone specimens’ and from non-destructive flat-panel-based volume computed tomography confirmed the spatial accuracy achieved by the procedure, as did the evaluation using the reference object.Conclusion This systematic and easy-to-follow preparation technique enables the three-dimensional (3D) histological reconstruction of complex soft and bony tissue structures. It facilitates the creation of detailed and spatially correct 3D anatomical models. Such models are of great benefit for image-based segmentation and planning in the field of computer-assisted surgery as well as in finite element analysis. In the context of human inner ear surgery, three-dimensional histology will improve the experimental evaluation and determination of intra-cochlear trauma after the insertion of an electrode array of a cochlear implant system.

Computer assisted maxillofacial surgery.

Technical developments strongly influence modern medicine. This is especially obvious in imaging technology. Today, one of the most difficult tasks for surgeons is transferring all the available imaging information for their patients into one "complete picture". In the operation theatre, this picture then has to be applied to the patient. Computer-assisted surgery (CAS) promises to help in fulfilling this task and, thereby, to fully utilize the possibilities offered by modern imaging techniques. Today's standard procedures for CAS in the maxillofacial region are described in technical principles and clinical applications. They are evaluated and discussed based on the available literature and in light of practical experience of more than ten years in the field of CAS. In addition, an outlook is given into developments of the near future that have appeared in the current literature. While technical development is leading toward a complete integration of all processes surrounding the patient and his or her surgery, basic CAS has moved from research to clinical care. Before it is routinely used, more investigation about its effectiveness and benefits has to be done, especially since increasing medical care costs are an issue in all countries.

Placement of C2 laminar screws using three-dimensional fluoroscopy-based image guidance by Eric W. Nottmeier and Andrew B. Foy

In their work colleagues Nottmeier and Foy present a retrospective series of eight patients treated with posterior C1–C2 fusion by laminar screws using three-dimensional fluoroscopy-based image guidance. They conclude from the results of their work that: “In this study, it has been shown that C2 laminar screws can be safely and accurately placed using 3D fluoroscopy-based image guidance. Advantages of this technology over traditional techniques include real-time 3D computerized feedback to the surgeon during placement, as well as the ability to size the screw to the patient’s anatomy using intraoperative computerized planning.” In the last decade, technology has advanced to the point where regular computer workstations are able to calculate in real-time the transformation matrices needed to determine the position of a camera-tracked object (in a surgical setting, typically a frame with fiducials) in relation to a camera-tracked reference base. By attaching the reference base to a patient and (surgical) instruments to camera-tracked objects, the workstation can calculate the respective positions/orientations and display their virtual interactions. When three-dimensional data sets representing the patient’s anatomy from modalities including CT/MRI/3D rotational X-ray imaging, are manipulated so that virtual and real reference bases are exactly aligned, surgical instruments and/or implants can be displayed on anatomical structures not visible for the surgeon intraoperatively. This is, in a nutshell, current “computer-assisted surgery” or “image-guided surgery”. Assuming that transformation matrix calculations are mathematical processes carried out with high fidelity and great reproducibility, imperfections in navigational accuracy must have other sources. Calibration errors in 3D fluoroscopy-based systems have, for example, been shown to account for approximately 1 mm accuracy in a phantom model [1]. In a more realistic cadaveric model, the accuracy of navigation deteriorated to approximately 2.5 mm, mainly due to bending of instruments and/or references during manipulation [2]. In clinical practice more inaccuracies can be introduced by occasional blocking of the camera field of view; inadvertently touching/hitting references and, especially, a nonrigid connection between the patient reference base and the actual surgical site. In the present study, the patient reference base was attached to a head clamp, which was, in turn, connected via pins to the patient’s skull while surgery was performed at the levels C1 and C2. This experimental setup represents quite a long chain of connections from the reference base to the surgical site with considerable opportunity for movement between the two extremes. This is without taking into account additional movements due to, for example, patient ventilation and direct manipulation of C1 and C2 by surgical instruments. Considering that the C0–C1–C2 complex is highly mobile, it came as a small surprise for this reviewer to find such accurate screw placement in the current study. As can be found in the article, the investigators did verify the image-guided technique (“After accuracy of navigation was confirmed by the surgeon ... ”). The interesting question that follows automatically is: “How?” Checking accuracy by probing the patient reference base with a navigated instrument and finding a perfectly matched position/orientation on the computer display does nothing for the flexibility/movement errors that may be present at the more distant surgical site. Most often, surgeons verify accuracy by a direct intraoperative view of the surgical site to see whether, for example, the proposed drill trajectory or planned screw length is in accordance to their own experience from conventional surgeries before moving on with navigation. This leads to the strange paradox that is really bugging contemporary image-guided procedures: the employment of a technique to enhance surgical accuracy requiring the surgeon’s intraoperative verification of its accuracy. The reluctance of surgeons to show complete faith in image guidance seems justified considering the errors demonstrated in accuracy studies. Using computer navigation without proper anatomical and surgical knowledge would be irresponsible. Armed with proper knowledge, however, navigation becomes largely superfluous since surgeons trust their surgical experience/skills and the direct intraoperative view over computer-generated images anytime, and adjust their actions accordingly. Which surgeon would happily volunteer when asked to perform a percutaneous procedure having to trust solely on information provided by any image-guided technique commercially available today? Until now, no large-scale long-term and properly controlled study has been published in the field of computer-assisted surgery that has convincingly showed any actual patient benefit when using image guidance. Lacking a control group and using a navigation technique susceptible to the aforementioned accuracy problems, the present study may, unfortunately, not be able to change this status.

The use of navigation for total knee arthroplasty

Purpose of reviewNavigation of total knee arthroplasty is an exciting new development in the field of computer-assisted surgery. It allows the surgeon to obtain accurate mechanical alignment, to verify the alignment and adjust surgery accordingly while in the operating theatre. All navigation system

A Review of Magnetic Resonance Imaging Compatible Manipulators in Surgery

Developments in magnetic resonance imaging (MRI), coupled with parallel progress in the field of computer-assisted surgery, mean that an ideal environment has been created for the development of MRI-compatible robotic systems and manipulators, capable of enhancing many types of surgical procedure. However, MRI does impose severe restrictions on mechatronic devices to be used in or around the scanners. In this article a review of the developments in the field of MRI-compatible surgical manipulators over the last decade is presented. The manipulators developed make use of different methods of actuation, but they can be reduced to four main groups: actuation transmitted through hydraulics, pneumatic actuators, ultrasonic motors based on the piezoceramic principle and remote manual actuation. Progress has been made concerning material selection, position sensing, and different actuation techniques, and design strategies have been implemented to overcome the multiple restrictions imposed by the MRI environment. Most systems lack the clinical validation needed to continue on to commercial products.