Surgical Workflow Research Articles

Introduction of In Vivo Confocal Laser Endomicroscopy and Real-Time Telepathology for Remote Intraoperative Neurosurgery-Pathology Consultation.

Precise communication between neurosurgeons and pathologists is crucial for optimizing patient care, especially for intraoperative diagnoses. Confocal laser endomicroscopy (CLE) combined with a telepathology software platform (TSP) provides a novel venue for neurosurgeons and pathologists to review CLE images and converse intraoperatively in real-time. To describe the feasibility of integrating CLE and a TSP in the surgical workflow for real-time review of in vivo digital fluorescence tissue imaging in 3 patients with intracranial tumors. Although the neurosurgeon used the CLE probe to generate fluorescence images of histoarchitecture within the operative field that were displayed on monitors in the operating room, the pathologist simultaneously remotely viewed the CLE images. The neurosurgeon and pathologist discussed in real-time the histological structures of intraoperative imaging locations. The neurosurgeon placed the CLE probe at various locations on and around the tumor, in the surgical resection bed, and on surrounding brain tissue with communication through the TSP. The neurosurgeon oriented the pathologist to the location of the CLE, and the pathologist and neurosurgeon discussed the CLE images in real-time. The TSP and CLE were integrated successfully and rapidly in the operating room in all 3 cases. No patient had perioperative complications. Two novel digital neurosurgical cellular imaging technologies were combined with intraoperative neurosurgeon-pathologist communication to guide the identification of abnormal histoarchitectural tissue features in real-time. CLE with the TSP may allow rapid decision-making during tumor resection that may hold significant advantages over the frozen section process and surgical workflow in general.

Intraoperative Integration of Multimodal Imaging to Improve Neuronavigation: A Technical Note

Brain shift may cause significant error in neuronavigation, leading the surgeon to possible mistakes. Intraoperative magnetic resonance imaging (MRI) is the most reliable technique in brain tumor surgery. Unfortunately, it is highly expensive and time consuming and, at the moment, it is available only in few neurosurgical centers. In this case series the surgical workflow for brain tumor surgery is described where neuronavigation of preoperative MRI, intraoperative computed tomography (CT) scan, and ultrasound (US) as well as rigid and elastic image fusion between preoperative MRI and intraoperative US and CT, respectively, was applied to 4 brain tumor patients in order to compensate for surgically induced brain shift by using a commercially available software (Elements Image Fusion 4.0 with Virtual iMRI Cranial; Brainlab AG, München, Germany). Four illustrative cases demonstrated successful integration of different components of the described intraoperative surgical workflow. The data indicate that intraoperative navigation update is feasible by applying intraoperative 3-dimensional US and CT scanning as well as rigid and elastic image fusionapplied depending on the degree of observed brain shift. Integration of multiple intraoperative imaging techniques combined with rigid and elastic image fusion of preoperative MRI may reduce the risk of incorrect neuronavigation during brain tumor resection. Further studies are needed to confirm the present findings in a larger population.

Real-time intraoperative surgical telepathology using confocal laser endomicroscopy.

Communication between neurosurgeons and pathologists is mandatory for intraoperative decision-making and optimization of resection, especially for invasive masses. Handheld confocal laser endomicroscopy (CLE) technology provides in vivo intraoperative visualization of tissue histoarchitecture at cellular resolution. The authors evaluated the feasibility of using an innovative surgical telepathology software platform (TSP) to establish real-time, on-the-fly remote communication between the neurosurgeon using CLE and the pathologist. CLE and a TSP were integrated into the surgical workflow for 11 patients with brain masses (6 patients with gliomas, 3 with other primary tumors, 1 with metastasis, and 1 with reactive brain tissue). Neurosurgeons used CLE to generate video-flow images of the operative field that were displayed on monitors in the operating room. The pathologist simultaneously viewed video-flow CLE imaging using a digital tablet and communicated with the surgeon while physically located outside the operating room (1 pathologist was in another state, 4 were at home, and 6 were elsewhere in the hospital). Interpretations of the still CLE images and video-flow CLE imaging were compared with the findings on the corresponding frozen and permanent H&E histology sections. Overall, 24 optical biopsies were acquired with mean ± SD 2 ± 1 optical biopsies per case. The mean duration of CLE system use was 1 ± 0.3 minutes/case and 0.25 ± 0.23 seconds/optical biopsy. The first image with identifiable histopathological features was acquired within 6 ± 0.1 seconds. Frozen sections were processed within 23 ± 2.8 minutes, which was significantly longer than CLE usage (p < 0.001). Video-flow CLE was used to correctly interpret tissue histoarchitecture in 96% of optical biopsies, which was substantially higher than the accuracy of using still CLE images (63%) (p = 0.005). When CLE is employed in tandem with a TSP, neurosurgeons and pathologists can view and interpret CLE images remotely and in real time without the need to biopsy tissue. A TSP allowed neurosurgeons to receive real-time feedback on the optically interrogated tissue microstructure, thereby improving cross-functional communication and intraoperative decision-making and resulting in significant workflow advantages over the use of frozen section analysis.

Dynamic Navigation System for Immediate Implant Placement in the Maxillary Aesthetic Region

(1) Background: The achievement of an optimal implant position is still a critical consideration in implantology, especially in the aesthetic area. Dynamic navigation is a computer-aided procedure that allows the surgeon to follow on a screen the three-dimensional position of instruments in real time during implant site preparation. The aim of this proof-of-concept study was to assess the clinical and radiographical performance and accuracy of the surgical workflow during maxillary immediate implant placement assisted by DTX studio Implant software planning and X-Guide Navigation. (2) Methods: Twelve consecutive patients requiring at least one implant in the maxillary aesthetic region were treated. Clinical outcome measures were biological complications and implant or prosthetic success rate. The accuracy was measured by calculating the deviation between the real implant position obtained from the postoperative cone beam computed tomography (CBCT) scan and the planned implant position. (3) Results: The average deviation at the implant shoulder was 0.77 ± 0.25 mm and at the apical point was 1.2 ± 0.61 mm. The depth error was 0.5 ± 0.21 mm. The axis deviation was 2.5 ± 0.41 degrees. No biological complications or implant and prosthetic failures occurred after mean 6-month follow-up. (4) Conclusions: Within the limitations of this study, it seems that the dynamic navigation system for implant placement in the maxillary aesthetic region is accurate for prosthetically driven implant placement.

Full digital implant workflow utilizing the one-abutment one-time concept

This case presentation aims to demonstrate a full digital implant planning, surgical and prosthetic workflow utilizing the one-abutment one-time (OAOT) concept. Type III implant placement after socket preservation with platelet-rich fibrin (PRF) was scheduled for a patient at location 46. Initial intraoral scans with a corresponding digital wax-up and CBCT data were imported into implant planning software (SMOP). The surgical template was 3D printed, while a digital scan-body representing implant position was extracted and uploaded into CAD software (EXOCAD) in order to plan and mill a definitive custom hybrid zirconia abutment and a PMMA temporary crown prior to surgery. Static navigation for implant placement utilizing a pouch roll flap technique together with the placement of the definitive abutment and the temporary crown were used in order to achieve an immediate non-functional loading supplemented with the OAOT protocol. Four months postoperatively, a prefabricated zirconia framework (designed in the CAD software on top of the custom abutment) was porcelain veneered after rescanning the abutment intra-orally for 3D printed models and was delivered. The implant healed uneventfully resulting in a favorable soft tissue marginal contour without any buccal recession. Periapical X-rays showed no signs of marginal bone loss at the 1 and 2 years postoperative controls. The OAOT concept together with the utilized flap technique for soft tissue thickening resulted in zero bone loss and marginal gum stability. Digital technologies can help to preplan and fabricate individualized abutments prior to surgery to fulfill OAOT protocols.

Augmented Reality-Assisted Percutaneous Pedicle Screw Instrumentation: A Cadaveric Feasibility and Accuracy Study

Percutaneous pedicle screw instrumentation is the keystone of minimally invasive spine surgery. Percutaneous screw placement demands experience and relies greatly on intra-operative image guidance. This study aims to validate the feasibility and accuracy of augmented-reality (AR)-assisted percutaneous pedicle screw instrumentation. One cadaveric torso was prepared for this study. After a pre-operative computed tomography (CT) scan, the images were transferred to an AR station to generate a 3D hologram. The 3D hologram and navigation images were projected to a pair of goggles with a display screen. With registration, the 3D spine hologram was overlayed onto the cadaver. Bilateral instrumentation from T6 to L5 was performed by two surgeons using AR assistance. A post-operative CT scan was obtained. The Gertzbein–Robbins scale (grade 0–3) was used for accuracy assessment. A total of 24 screws were placed. The overall screw accuracy was 87.5%. There were three major medial breaches that occurred on Rt T6/7/8, which were the most distant screws from the iliac reference. The cause of the three major medial breaches appeared to be related to their distance from the iliac reference. AR-assisted percutaneous pedicle screw instrumentation could improve anatomical visualization, facilitate surgical workflow, and provide an intuitive way of performing surgery.

Fluoroscopic imaging: New advances

Today's orthopedic surgery could not be imagined without intraoperative x-ray-based imaging. This enables surgeons to assess operative interim steps as well as the result before wound closure and finishing the procedure. Although there have been mobile C-arms used for decades, there are recent advances that do not only affect the quality of the imaging itself but also the way, the information is processed and presented. These very exciting developments will change the integration of imaging into the surgical workflows, giving options of augmented reality, reduction of radiation dose, automatized acquisition and analysis of images and low-level guidance in procedures.This paper gives a review of current innovations and possible future trends in fluoroscopic 2D and 3D imaging.

A multi-sensor architecture combining human pose estimation and real-time location systems for workflow monitoring on hybrid operating suites

Despite the advancements to improve patient safety, a significant number of errors still occur in Operating Suites (OS). To improve medical decision-making and the resulting quality of care, it is essential to monitor and understand medical activities’ workflow and interactions. Although some strategies employ different sensor devices, their focus is not on generating complete workflow information. They only combine different data sources to generate their final output, lacking at least one piece of information from a workflow. To tackle this challenge, this paper presents ▪ , a distributed architecture model for sensor data acquisition and processing. ▪ ’s main contribution lies in its multi-sensor data fusion algorithms to extract a computational representation of activities in surgical procedures. In addition, ▪ is flexible to accommodate different deployment configurations,combining depth cameras, Ultra-Wideband positioning systems,and deep learning-based human pose estimation (HPE) mechanisms. The workflow monitoring mechanism was deployed in an actual hybrid OS for an extensive evaluation of the proposal. Our experiments demonstrate that the architecture can capture the information required to monitor the surgical workflow. In particular, the proposed HPE methodology accurately detects the poses of medical staff members, with a maximum error of 5cm.

Intraoperative technology increases operating room times in primary total knee arthroplasty.

Optimization of patient outcomes and identification of factors to improve the surgical workflow are increasingly important. Operating room time is one modifiable factor that leads to greater hospital efficiency as well as improved outcomes such as shorter length of stay and fewer infections and readmissions. The aim of this study was to identify factors associated with operative time disparities in total knee arthroplasty (TKA). A retrospective review of 7659 consecutive primary TKA cases was conducted. Patient demographic data, discrete operating room (OR) times, use of technology (i.e. robotic-assisted surgery, computer navigation), surgeon experience and the level of training of the first assistant were collected. Multivariate regression analysis was used to determine the effect of hospital characteristics on operative times. Operative times of five minutes or greater were considered to be clinically significant. While the use of technology (182.64 ± 39.85 vs 158.70 ± 37.45min; B = 26.09; p < 0.0001) and greater surgeon experience (162.14 ± 39.87 vs 158.69 ± 33.18min, B = 3.15, p = 0.002) were found to increase OR times, level of training of the first assist (161.65 vs 156.4min; Β = -0.264; p = 0.487) did not. Of the discrete OR times examined, incision time and total time under anesthesia were negatively impacted by the use of technology. Use of technology was the only study variable found to significantly increase OR times. With increased operative times and limited evidence that technology improves long-term patient outcomes, surgeons should carefully consider the benefits and cost of technology in TKA.

Intercalary and geographic lower limb tumor resections with the use of 3D printed Patient Specific Instruments- when less is more

BackgroundPrimary bone sarcomas are associated with critically sized bone defects and require complete resection with negative margins. Recent advancements in health care have pioneered novel approaches such as the implementation of 3D surgical technologies. This study presents oncological and functional outcomes following tumor resections of long bones with the use of customized 3D-printed Patient Specific Instruments (PSIs). MethodsThis single-center retrospective study is comprised of seventeen patients who underwent either intercalary (N = 12) or geographic (N = 5) resections with various reconstruction methods including allograft (N = 8), vascularized fibula (Capanna) (N = 7), and 3D printed customized titanium implants (N = 2), between the years 2016–2020. All patients were operated on with a 3D surgical workflow, including intraoperative PSIs, and were followed up postoperatively for at least 12 months (average 31.40 ± 12.13 months) to assess oncological and functional outcomes. ResultsAll patients demonstrated negative surgical margins, apart from one patient who had planned positive margins. Three patients suffered from short-term complications, and three patients underwent revision surgery due to graft non-union or pathological fracture. One patient suffered from local recurrence and underwent above-knee amputation. Three patients suffered from lung metastasis. MSTS at 12-month follow-up was 26.9.±5.87. ConclusionCustomized 3D-printed osteotomy PSIs provide surgeons with a novel tool for optimizing bone resection and reconstruction in long bones surgeries, thus minimizing overall tissue trauma and reducing the risk of damage to nervous and vascular structures. This study demonstrates that the use of PSIs has the potential to improve functional and oncological outcomes. We believe that this technique will become increasingly popular in the future as a widely applicable, highly accurate, cost-effective optimization tool.

Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos.

Out of all existing frameworks for surgical workflow analysis in endoscopic videos, action triplet recognition stands out as the only one aiming to provide truly fine-grained and comprehensive information on surgical activities. This information, presented as 〈instrument, verb, target〉 combinations, is highly challenging to be accurately identified. Triplet components can be difficult to recognize individually; in this task, it requires not only performing recognition simultaneously for all three triplet components, but also correctly establishing the data association between them. To achieve this task, we introduce our new model, the Rendezvous (RDV), which recognizes triplets directly from surgical videos by leveraging attention at two different levels. We first introduce a new form of spatial attention to capture individual action triplet components in a scene; called Class Activation Guided Attention Mechanism (CAGAM). This technique focuses on the recognition of verbs and targets using activations resulting from instruments. To solve the association problem, our RDV model adds a new form of semantic attention inspired by Transformer networks; called Multi-Head of Mixed Attention (MHMA). This technique uses several cross and self attentions to effectively capture relationships between instruments, verbs, and targets. We also introduce CholecT50 - a dataset of 50 endoscopic videos in which every frame has been annotated with labels from 100 triplet classes. Our proposed RDV model significantly improves the triplet prediction mAP by over 9% compared to the state-of-the-art methods on this dataset.

The impact of ensemble learning on surgical tools classification during laparoscopic cholecystectomy

Laparoscopic surgery also know as minimally invasive surgery (MIS), is a type of surgical procedure that allows a surgeon to examine the organs inside of the abdomen without having to make large incisions in the skin. It unifies the competence and skills of highly trained surgeons with the power and precision of machines. Furthermore, surgical instruments are inserted through the abdomen with the help of a laparoscope, which is a tube with a high-intensity light and a high-resolution camera at the end. In addition, recorded videos from this type of surgery have become a steadily more important information source. However, MIS videos are often very long, thereby, navigating through these videos is time and effort consuming. The automatic identification of tool presence in laparoscopic videos leads to detecting what tools are used at each time in surgery and helps in the automatic recognition of surgical workflow. The aim of this paper is to predict surgical tools from laparoscopic videos using three states of the arts CNNs, namely: VGG19, Inception v-4, and NASNet-A. In addition, an ensemble learning method is proposed, combining the three CNNs, to solve the tool presence detection problem as a multi-label classification problem. The proposed methods are evaluated on a dataset of 80 cholecystectomy videos (Cholec80 dataset). The results present an improvement of approximately 6.19% and a mean average precision of 97.84% when the ensemble learning method is applied.

Clinical study of skill assessment based on time sequential measurement changes

Endoscopic sinus surgery is a common procedure for chronic sinusitis; however, complications have been reported in some cases. Improving surgical outcomes requires an improvement in a surgeon’s skills. In this study, we used surgical workflow analysis to automatically extract “errors,” indicating whether there was a large difference in the comparative evaluation of procedures performed by experts and residents. First, we quantified surgical features using surgical log data, which contained surgical instrument information (e.g., tip position) and time stamp. Second, we created a surgical process model (SPM), which represents the temporal transition of the surgical features. Finally, we identified technical issues by creating an expert standard SPM and comparing it to the novice SPM. We verified the performance of our methods by using the clinical data of 39 patients. In total, 303 portions were detected as an error, and they were classified into six categories. Three risky operations were overlooked, and there were 11 overdetected errors. We noted that most errors detected by our method involved dangers. The implementation of our methods of automatic improvement points detection may be advantageous. Our methods may help reduce the time for reviewing and improving the surgical technique efficiently.

Robotic Endoscope Control Via Autonomous Instrument Tracking

Many keyhole interventions rely on bi-manual handling of surgical instruments, forcing the main surgeon to rely on a second surgeon to act as a camera assistant. In addition to the burden of excessively involving surgical staff, this may lead to reduced image stability, increased task completion time and sometimes errors due to the monotony of the task. Robotic endoscope holders, controlled by a set of basic instructions, have been proposed as an alternative, but their unnatural handling may increase the cognitive load of the (solo) surgeon, which hinders their clinical acceptance. More seamless integration in the surgical workflow would be achieved if robotic endoscope holders collaborated with the operating surgeon via semantically rich instructions that closely resemble instructions that would otherwise be issued to a human camera assistant, such as “focus on my right-hand instrument.” As a proof of concept, this paper presents a novel system that paves the way towards a synergistic interaction between surgeons and robotic endoscope holders. The proposed platform allows the surgeon to perform a bimanual coordination and navigation task, while a robotic arm autonomously performs the endoscope positioning tasks. Within our system, we propose a novel tooltip localization method based on surgical tool segmentation and a novel visual servoing approach that ensures smooth and appropriate motion of the endoscope camera. We validate our vision pipeline and run a user study of this system. The clinical relevance of the study is ensured through the use of a laparoscopic exercise validated by the European Academy of Gynaecological Surgery which involves bi-manual coordination and navigation. Successful application of our proposed system provides a promising starting point towards broader clinical adoption of robotic endoscope holders.

From Start-to-Finish: A Partnership With Advanced Practice Providers in Tele-Consultation Within Pediatric Urology

BackgroundTelemedicine is now commonplace and an accepted modality for delivering health care due to the coronavirus pandemic. It has proven useful in increasing access to traditionally limited subspecialties and bridged the gap of local regional geographic restraints. We report a follow up of our experience building tele-consultation capacity for pediatric urology in a resource constrained area. Specifically, we hypothesize that hybrid tele-consultation is a safe modality for postoperative evaluation. A secondary goal of this study is to evaluate perceived barriers to implementing this practice from the perspective of advanced practice providers (APPs).MethodsAfter obtaining IRB approval (IRB# 1910741113), patients were prospectively recruited for tele-consultation encounters with a tertiary academic center between August 2018 and March 2020 (pre-dating the coronavirus pandemic). Advanced practice providers received additional training in physical diagnosis and examination prior to returning to satellite clinics in their local communities and facilitating tele-consultation with an academic pediatric urologist. Postoperative outcomes were analyzed out to eight weeks and any peri- and postoperative complications were assessed using the Clavien-Dindo classification system. Finally, anonymous surveys were administered to the APPs at the conclusion of the recruitment period to assess their satisfaction with telemedicine clinics and perceived barriers to implementation.Results92 telemedicine encounters took place between August 2018 and March 2020. The most common reasons for presentation were undescended testis (UDT) (n=15), urinary tract infection (n=13) and enuresis (n=13). The most common surgical case booked by the APPs in the satellite clinic was orchiopexy (n=10). Mean distance travel saved was 299.8 miles. In 23 patients who were schedule surgery, only two patients (8.7%) had an initial diagnosis that differed with their APP pre-operative diagnosis. Of the 22 patients who underwent surgery, all elected telemedicine visits for postoperative evaluation. No patient experienced complications. Initial consultations averaged 14 minutes (range 9 to 20 minutes), while mean duration of postoperative evaluations was 9 minutes (range 6 to 13 minutes). With regards to surveys, APPs reported an overall positive experience with tele-consultation (4.5/5, n=7) and all planned on incorporating it into their future practice (4.9/5, n =7).ConclusionA hybrid tele-consultation video conferencing clinic with heavy reliance on APPs is a safe, reliable and economical way to address pediatric subspecialty surgical needs in the rural setting. With proper training and exposure, APPs can safely perform preoperative assessments and correctly refer patients for surgical intervention, adding efficiency to the surgical subspecialty workflow.

Single Position Lateral Lumbar Interbody Fusion With Posterior Instrumentation Utilizing Computer Navigation and Robotic Assistance: Retrospective case review and surgical technique considerations

Study DesignRetrospective cohort study.ObjectiveTo determine safety and short-term outcomes of single-position lateral lumbar interbody fusion (LLIF) with bilateral posterior instrumentation and robotic assistance. The article also describes surgical technique considerations for the procedure.Methods20 patients underwent single-position LLIF with posterior instrumentation and robotic assistance. The patients were followed for a minimum of 3 months post-operatively.ResultsAverage operative time was 211 ± 34 minutes, average blood loss was 51.25 ± 17 cc’s, and average length of stay was 1.4 ± .75 days. There were no intraoperative complications, readmissions, revision surgeries, and no incidence of hardware malposition. Significant improvement in pain and ODI scores was noted at 3 month follow up.ConclusionsThe study demonstrated safety and short-term clinical efficacy of minimally invasive single-position lateral lumbar interbody fusion with bilateral posterior instrumentation utilizing robotic assistance and navigation. There are certain surgical technique considerations that must be followed to ensure optimal surgical workflow and predictable outcomes.

SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgery

Comprehension of surgical workflow is the foundation upon which artificial intelligence (AI) and machine learning (ML) holds the potential to assist intraoperative decision making and risk mitigation. In this work, we move beyond mere identification of past surgical phases, into prediction of future surgical steps and specification of the transitions between them. We use a novel Generative Adversarial Network (GAN) formulation to sample future surgical phases trajectories conditioned on past video frames from laparoscopic cholecystectomy (LC) videos and compare it to state-of-the-art approaches for surgical video analysis and alternative prediction methods. We demonstrate the GAN formulation&#x2019;s effectiveness through inferring and predicting the progress of LC videos. We quantify the horizon-accuracy trade-off and explored average performance, as well as the performance on the more challenging, and clinically relevant transitions between phases. Furthermore, we conduct a survey, asking 16 surgeons of different specialties and educational levels to qualitative evaluate predicted surgery phases.

Reducing prediction volatility in the surgical workflow recognition of endoscopic pituitary surgery

Purpose:Workflow recognition can aid surgeons before an operation when used as a training tool, during an operation by increasing operating room efficiency, and after an operation in the completion of operation notes. Although several methods have been applied to this task, they have been tested on few surgical datasets. Therefore, their generalisability is not well tested, particularly for surgical approaches utilising smaller working spaces which are susceptible to occlusion and necessitate frequent withdrawal of the endoscope. This leads to rapidly changing predictions, which reduces the clinical confidence of the methods, and hence limits their suitability for clinical translation.Methods:Firstly, the optimal neural network is found using established methods, using endoscopic pituitary surgery as an exemplar. Then, prediction volatility is formally defined as a new evaluation metric as a proxy for uncertainty, and two temporal smoothing functions are created. The first (modal, M_n) mode-averages over the previous n predictions, and the second (threshold, T_n) ensures a class is only changed after being continuously predicted for n predictions. Both functions are independently applied to the predictions of the optimal network.Results:The methods are evaluated on a 50-video dataset using fivefold cross-validation, and the optimised evaluation metric is weighted-F_1 score. The optimal model is ResNet-50+LSTM achieving 0.84 in 3-phase classification and 0.74 in 7-step classification. Applying threshold smoothing further improves these results, achieving 0.86 in 3-phase classification, and 0.75 in 7-step classification, while also drastically reducing the prediction volatility.Conclusion:The results confirm the established methods generalise to endoscopic pituitary surgery, and show simple temporal smoothing not only reduces prediction volatility, but actively improves performance.

Targeted Dual-Modal PET/SPECT-NIR Imaging: From Building Blocks and Construction Strategies to Applications.

Simple SummaryTargeted dual-modal imaging agents for whole body PET/SPECT imaging and fluorescence-guided surgery (FGS) have significant potential to improve surgical workflow. Here we present an overview of key design considerations and fluorescent building blocks, along with potential future directions in this exciting field.Molecular imaging is an emerging non-invasive method to qualitatively and quantitively visualize and characterize biological processes. Among the imaging modalities, PET/SPECT and near-infrared (NIR) imaging provide synergistic properties that result in deep tissue penetration and up to cell-level resolution. Dual-modal PET/SPECT-NIR agents are commonly combined with a targeting ligand (e.g., antibody or small molecule) to engage biomolecules overexpressed in cancer, thereby enabling selective multimodal visualization of primary and metastatic tumors. The use of such agents for (i) preoperative patient selection and surgical planning and (ii) intraoperative FGS could improve surgical workflow and patient outcomes. However, the development of targeted dual-modal agents is a chemical challenge and a topic of ongoing research. In this review, we define key design considerations of targeted dual-modal imaging from a topological perspective, list targeted dual-modal probes disclosed in the last decade, review recent progress in the field of NIR fluorescent probe development, and highlight future directions in this rapidly developing field.

Guided Activity Prediction for Minimally Invasive Surgery Safety Improvement in the Internet of Medical Things

With the application of the Internet of Medical Things (IoMT) in minimally invasive surgery (MIS), surgeons now have a better chance at hard-to-treat cases by carrying out more complicated MIS workflows. However, a scheduled surgical workflow is often required to be updated based on the patient’s internal tissue states. Perioperative complications could occur if in-time adjustments are lacking in the operating rooms when needed. To help manage the uncertainty of live surgical workflows in the IoMT environment, we propose an MIS safety improvement framework. It helps surgeons in predicting surgical workflows with limited MIS video frames by embedding our proposed model GuidedNet. To predict future surgical activities, we first build three isomorphic neural networks to capture the spatio-temporal information. Then we establish a guidance fusion module to handle the contextual information. It guides the GuidedNet to recognize the surgical stage. Moreover, we build a novel joint loss function to train the GuidedNet to predict the future surgical stage. We evaluate the approach on a large dataset that contains 80 cholecystectomy videos (Cholec-80) and compare it with the state-of-art. Experiments show that the GuidedNet can assist surgeons in carrying out MIS as well as guide the next stage of surgery for improving surgical safety. Comparing to the state-of-the-art, our approach can obtain better predict accuracy (up to 79%) with less computing resource consumption. The result also shows that our approach has a high application prospect in video classification in other Internet-of-Things scenarios.