Laparoscopic Video Research Articles

Automated Surgical Instrument Detection from Laparoscopic Gastrectomy Video Images Using an Open Source Convolutional Neural Network Platform

The common use of laparoscopic intervention produces impressive amounts of video data that are difficult to review for surgeons wishing to evaluate and improve their skills. Therefore, a need exists for the development of computer-based analysis of laparoscopic video to accelerate surgical training and assessment. We developed a surgical instrument detection system for video recordings of laparoscopic gastrectomy procedures. This system, the use of which might increase the efficiency of the video reviewing process, is based on the open source neural network platform, YOLOv3. A total of 10,716 images extracted from 52 laparoscopic gastrectomy videos were included in the training and validation data sets. We performed 200,000 iterations of training. Video recordings of 10 laparoscopic gastrectomies, independent of the training and validation data set, were analyzed by our system, and heat maps visualizing trends of surgical instrument usage were drawn. Three skilled surgeons evaluated whether each heat map represented the features of the corresponding operation. After training, the testing data set precision and sensitivity (recall) was 0.87 and 0.83, respectively. The heat maps perfectly represented the devices used during each operation. Without reviewing the video recordings, the surgeons accurately recognized the type of anastomosis, time taken to initiate duodenal and gastric dissection, and whether any irregular procedure was performed, from the heat maps (correct answer rates ≥ 90%). A new automated system to detect manipulation of surgical instruments in video recordings of laparoscopic gastrectomies based on the open source neural network platform, YOLOv3, was developed and validated successfully.

Effects of a Virtual Pointer on Trainees' Cognitive Load and Communication Efficiency in Surgical Training.

We investigated the cognitive load effect of a trainer providing surgical instruction by pointing/drawing over laparoscopic video to a trainee. Results showed that while cognitive load is higher overall with the use of the instructional system, there is a decrease by the second experience of being instructed by the Virtual Pointer. Further analysis showed that trainees were more likely to perform the surgical task and watch/listen to the trainer's instruction at the same time when the instructional system was used. This is thought to be an indication of more efficient communication when using the instructional system. Thus, although there is a small cognitive overload with the instructional system initially, the more efficient communication allows trainees to better integrate the knowledge and instructions being conveyed into the actions they must perform - indicating a better learning environment.

TACTICS OF SURGICAL TREATMENT FOR ISOLATED DUODENAL INJURY

Currently the diagnosis and surgical treatment of isolated duodenal injury is one of the most rare and, consequently, a complicated and completely unsolved problem in emergency surgery. In order to analyze the results of surgical treatment and to justify the optimal amount of intervention in the patients with isolated duodenal injury we have performed a study, wherein the cases of an open injury of the cut−stab character, the effects of a blow on the anterior abdominal wall have been considered. The admission time of patients generally made 6 hours. Examination radiography of abdominal organs with DeBakey − Henelt test, phagogastroduodenoscopy was performed. In cases of unexplained clinical picture the diagnostic video laparoscopy was urgently used. The factors, affecting an outcome of treating the isolated duodenal injury were as follows: the time of hospitalization of patients, size and localization of the bowel defect, timely diagnosis and choice of the surgery volume as well as the post−surgery complications. A thorough revision of a retroperitoneal space, being the one of the symptoms of the Winiwarter−Laffite triad, was accomplished. In injuries of the first and second degree, duration of an injury of no longer than 6 hours, it is necessary to close the defect of the duodenum by two−row sutures with a nasogastroduodenal intubation. At admission of the patients at later terms or with the injuries of the third degree it is expedient to exclude a passage of food on a gut by imposing a gastroenteroanastomosis with Brownian complicity and a decompression of bilious ways. On the basis of our findings the diagnostic and surgical algorithms, depending on admission time of patients to a hospital, size and localization of duodenum wall defect were designed. Key words: duodenal injury, tactics of surgical treatment, results of surgical treatment.

Impacto da obesidade Grau I na mecânica respiratória durante cirurgia videolaparoscópica: estudo longitudinal prospectivo

Introduction and objectivesThe association pneumoperitoneum and obesity in video laparoscopy can contribute to pulmonary complications, but has not been well defined in specific groups of obese individuals. We assessed the effects of pneumoperitoneum in respiratory mechanics in Grade I obese compared to non‐obese. MethodsProspective study including 20 patients submitted to video laparoscopic cholecystectomy, normal spirometry, divided into non‐obese (BMI ≤ 25kg.m–2) and obese (BMI > 30kg.mg–2), excluding Grade II and III obese. We measured pulmonary ventilation mechanics data before pneumoperitoneum (baseline), and five, fifteen and thirty minutes after peritoneal insufflation, and fifteen minutes after disinflation (final). ResultsMean BMI of non‐obese was 22.72 ± 1.43kg.m–2 and of the obese 31.78 ± 1.09kg.m–2, p < 0.01. Duration of anesthesia and of peritoneal insufflation was similar between groups. Baseline pulmonary compliance (Crs) of the obese (38.3 ± 8.3mL.cm H2O–1) was lower than of the non‐obese (47.4 ± 5.7mL.cm H2O–1), p = 0.01. After insufflation, Crs decreased in both groups and remained even lower in the obese at all moments assessed (GLM p < 0.01). Respiratory system peak pressure and plateau pressure were higher in the obese, albeit variations were similar at moments analyzed (GLM p > 0.05). The same occurred with elastic pressure, higher in the obese at all times (GLM p = 0.04), and resistive pressure showed differences in variations between groups during pneumoperitoneum (GLM p = 0,05). ConclusionsGrade I obese presented more changes in pulmonary mechanics than the non‐obese during video laparoscopies and the fact requires mechanical ventilation‐related care.

Impact of Grade I obesity on respiratory mechanics during video laparoscopic surgery

Introduction and objectivesThe association pneumoperitoneum and obesity in video laparoscopy can contribute to pulmonary complications, but has not been well defined in specific groups of obese individuals. We assessed the effects of pneumoperitoneum in respiratory mechanics in Grade I obese compared to non-obese. MethodsProspective study including 20 patients submitted to video laparoscopic cholecystectomy, normal spirometry, divided into non-obese (BMI ≤ 25 kg.m-2) and obese (BMI > 30 kg.mg-2), excluding Grade II and III obese. We measured pulmonary ventilation mechanics data before pneumoperitoneum (baseline), and five, fifteen and thirty minutes after peritoneal insufflation, and fifteen minutes after disinflation (final). Results: Mean BMI of non-obese was 22.72 ± 1.43 kg.m-2 and of the obese 31.78 ± 1.09 kg.m-2, p < 0.01. Duration of anesthesia and of peritoneal insufflation was similar between groups. Baseline pulmonary compliance (Crs) of the obese (38.3 ± 8.3 mL.cm H2O-1) was lower than of the non-obese (47.4 ± 5.7 mL.cm H2O-1), p = 0.01. After insufflation, Crs decreased in both groups and remained even lower in the obese at all moments assessed (GLM p < 0.01). Respiratory system peak pressure and plateau pressure were higher in the obese, albeit variations were similar at moments analyzed (GLM p > 0.05). The same occurred with elastic pressure, higher in the obese at all times (GLM p = 0.04), and resistive pressure showed differences in variations between groups during pneumoperitoneum (GLM p = 0,05). ConclusionsGrade I obese presented more changes in pulmonary mechanics than the non-obese during video laparoscopies and the fact requires mechanical ventilation-related care.

Content-based retrieval system for surgery videos

There has been tremendous growth of laparoscopic surgical video archives in the past few years. Finding the surgical videos manually from the database is very troublesome and time consuming. As a consequence, efficient methods for retrieval of such videos from a database are needed. In this paper, a content-based video retrieval (CBVR) system is presented using average intensity, colour and shape feature. In the proposed retrieval system, average intensity, colour and shape feature are used. The ordinal measure introduced by Bhat and Nayar, which is based on ranks rather than of their values is used in this work. This is important in the laparoscopic videos because adjacent frames can be distinguished on the basis of ranking of their average intensities. Experimental results reveal that mean average precision of the proposed technique is better than the existing motion intensity and direction descriptor (MIDD) approach. This reflects the effectiveness of proposed method.

Опыт использования видеолапароскопии при обследовании и лечении пациентов с острой тонкокишечной непроходимостью

Опыт использования видеолапароскопии при обследовании и лечении пациентов с острой тонкокишечной непроходимостью

Content-based retrieval system for surgery videos

There has been tremendous growth of laparoscopic surgical video archives in the past few years. Finding the surgical videos manually from the database is very troublesome and time consuming. As a consequence, efficient methods for retrieval of such videos from a database are needed. In this paper, a content-based video retrieval (CBVR) system is presented using average intensity, colour and shape feature. In the proposed retrieval system, average intensity, colour and shape feature are used. The ordinal measure introduced by Bhat and Nayar, which is based on ranks rather than of their values is used in this work. This is important in the laparoscopic videos because adjacent frames can be distinguished on the basis of ranking of their average intensities. Experimental results reveal that mean average precision of the proposed technique is better than the existing motion intensity and direction descriptor (MIDD) approach. This reflects the effectiveness of proposed method.

Surgical Tools Detection Based on Modulated Anchoring Network in Laparoscopic Videos

Minimally invasive surgery like laparoscopic surgery is an active research area of clinical practice for less pain and a faster recovery rate. Detection of surgical tools with more accurate spatial locations in surgical videos not only helps to ensure patient safety by reducing the incidence of complications but also makes a difference to assess the surgeon performance. In this paper, we propose a novel Modulated Anchoring Network for detection of laparoscopic surgery tools based on Faster R-CNN, which inherits the merits of two-stage approaches while also maintains high efficiency of comparable speed as state-of-the-art one-stage methods. Since objects like surgical instruments with a wide aspect ratio are difficult to recognize, we develop a novel training scheme named as modulated anchoring to explicitly predict arbitrary anchor shapes of objects of interest. For taking the relationship of different tools into consideration, it is useful to embed the relation module in our network. We evaluate our method using an existing dataset (m2cai16-tool-locations) and a new private dataset (AJU-Set), both collected from cholecystectomy surgical videos in hospital, covering information of seven surgical tools with spatial bounds. We show that our detector yields excellent detection accuracy of 69.6% and 76.5% over the introduced datasets superior to other recently used architectures. We further verify the efficiency of our method by analyzing the usage patterns of tools, the economy of the movement, and the dexterity of operations to assess surgical quality.

Surgical Tools Detection Based on Training Sample Adaptation in Laparoscopic Videos

The performance of object detection methods plays an important role in the recognition of surgical tools, and is a key link in the automated evaluation of surgical skills. In this paper, we propose a novel framework for one-stage object detection based on a sample adaptive process controlled by reinforcement learning, which can maintain the speed advantage while maintaining higher accuracy than two-stage object detection methods. We use m2cai16-tool-locations and AJU-Set, two datasets covering seven surgical tools with spatial information collected from hospital gallbladder surgery videos to evaluate and verify the effectiveness of our proposed framework. The experiments show that our proposed framework can make the one-stage object detection method achieve 70.1% and 77.3% accuracy on m2cai16-tool-locations and AJU-Set, respectively. We further validated the effectiveness of our proposed framework by analyzing the usage patterns, motion trajectories, and mobile values of surgical tools.

A practical new strategy to prevent bile duct injury during laparoscopic cholecystectomy. A single-center experience with 5539 cases.

Purpose Bile duct injury (BDI) is a catastrophic complication of cholecystectomy, and misidentification of the cystic anatomy is considered to be the main cause. Although several techniques have been developed to prevent BDI, such as the “critical view of safety”, the infundibular technique, the rates remain higher during laparoscopic cholecystectomy (LC) than during open surgery. We, here, propose a practical new strategy for ductal identification, that can help to prevent laparoscopic bile duct injury.Methods A retrospective study of 5539 patients who underwent LC from March 2007 to February 2019 at a single institution was conducted. The gallbladder infundibulum was classified by its position located on an imaginary clock with the gallbladder neck as the center point of the dial, 3-o’clock position as cranial, 6-o’clock as dorsal, 9-o’clock as caudal, and 12-o’clock as ventral, as well as the axial position. Patient demographics, pathologic variables and infundibulum classification were evaluated. Detailed analysis of ductal identification based on gallbladder infundibulum position was performed in this study. All infundibulum positions were recorded by intraoperative laparoscopic video or photographic images.Results All the patients successfully underwent LC during the study period. No conversion or serious complications such as biliary injury occurred. Gallbladders with infundibulum of 3-o’clock position, 6-o’clock position, 9-o’clock position, 12-o’clock position, axial position were 12.3%, 23.4%, 28.0%, 4.2%, and 32.1%, respectively. The 3-o’clock and 12-o’clock position were pitfalls that might cause biliary injury.Conclusion The gallbladder infundibulum as a navigator is useful for ductal identification to reduce BDI and improve the safety of LC.

Multi-instance multi-label learning for surgical image annotation.

Various techniques have been proposed in the literature for phase and tool recognition from laparoscopic videos. In comparison, research in multilabel annotation of still frames is limited. We describe a framework for multilabel annotation of images extracted from laparoscopic cholecystectomy (LC) videos based on multi-instance multiple-label learning. The image is considered as a bag of features extracted from local regions after coarse segmentation. A method based on variational Bayesian gaussian mixture models (VBGMM) is proposed for bag representation. Three techniques based on different feature extraction and bag representation models are employed for comparison. Four anatomical structures (abdominal wall, gallbladder, fat, and liver bed) and a tool-like object (specimen bag) were annotated in 482 images. Our method achieved the best performance on single label accuracy: 0.87 (highest) and 0.69 (lowest). Moreover, the performance was >20% higher in terms of four multilabel classification error metrics (one-error, ranking-loss, hamming-loss, and coverage). Our approach provides an accurate and efficient image representation for multilabel classification of still images captured in LC.

Predicting the quality of surgical exposure using spatial and procedural features from laparoscopic videos.

PURPOSE: Evaluating the quality of surgical procedures is a major concern in minimally invasive surgeries. We propose a bottom-up approach based on the study of Sleeve Gastrectomy procedures, for which we analyze what we assume to be an important indicator of the surgical expertise: the exposure of the surgical scene. We first aim at predicting this indicator with features extracted from the laparoscopic video feed, and second to analyze how the extracted features describing the surgical practice influence this indicator. METHOD: Twenty-nine patients underwent Sleeve Gastrectomy performed by two confirmed surgeons in a monocentric study. Features were extracted from spatial and procedural annotations of the videos, and an expert surgeon evaluated the quality of the surgical exposure at specific instants. The features were used as input of a classifier (linear discriminant analysis followed by a support vector machine) to predict the expertise indicator. Features selected in different configurations of the algorithm were compared to understand their relationships with the surgical exposure and the surgeon's practice. RESULTS: The optimized algorithm giving the best performance used spatial features as input ([Formula: see text]). It also predicted equally the two classes of the indicator, despite their strong imbalance. Analyzing the selection of input features in the algorithm allowed a comparison of different configurations of the algorithm and showed a link between the surgical exposure and the surgeon's practice. CONCLUSION: This preliminary study validates that a prediction of the surgical exposure from spatial features is possible. The analysis of the clusters of feature selected by the algorithm also shows encouraging results and potential clinical interpretations.

2940 Initial Outcome Eras Protocol

Study Objective Analyze outcome after introducing Enhanced Recovery After Surgery (ERAS) protocol Design Data was storage in all laparoscopy surgery using ERAS protocol first to seventh day pos operative Setting Patients underwent video laparoscopy at IAM ATENDIMENTO MEDICO HOSPITALAR, Assis, Brazil Patients or Participants 60 patients underwent video laparoscopy between 2018-november to 2019-april, after introducing ERAS protocol. Ten days before the surgery all patients received a written document orientation from all moment of the procedure (before, during and after surgery) based on ERAS protocol Interventions we used ERAS protocol as Olle Ljungqvist, et al published in JAMA Surg. 2017. To evaluate outcome, we call patients 10 days after surgery asking about pain, use of pain medication and satisfaction level. Measurements and Main Results We found 91% of satisfaction, no pain medication 3 days after surgery and VAS between 2-6 Conclusion ERAS protocol bring us outcome improvement in our laparoscopy surgery

Augmented reality navigation for liver resection with a stereoscopic laparoscope

ObjectiveUnderstanding the three-dimensional (3D) spatial position and orientation of vessels and tumor(s) is vital in laparoscopic liver resection procedures. Augmented reality (AR) techniques can help surgeons see the patient's internal anatomy in conjunction with laparoscopic video images. MethodIn this paper, we present an AR-assisted navigation system for liver resection based on a rigid stereoscopic laparoscope. The stereo image pairs from the laparoscope are used by an unsupervised convolutional network (CNN) framework to estimate depth and generate an intraoperative 3D liver surface. Meanwhile, 3D models of the patient's surgical field are segmented from preoperative CT images using V-Net architecture for volumetric image data in an end-to-end predictive style. A globally optimal iterative closest point (Go-ICP) algorithm is adopted to register the pre- and intraoperative models into a unified coordinate space; then, the preoperative 3D models are superimposed on the live laparoscopic images to provide the surgeon with detailed information about the subsurface of the patient's anatomy, including tumors, their resection margins and vessels. ResultsThe proposed navigation system is tested on four laboratory ex vivo porcine livers and five operating theatre in vivo porcine experiments to validate its accuracy. The ex vivo and in vivo reprojection errors (RPE) are 6.04 ± 1.85 mm and 8.73 ± 2.43 mm, respectively. Conclusion and SignificanceBoth the qualitative and quantitative results indicate that our AR-assisted navigation system shows promise and has the potential to be highly useful in clinical practice.

Virtual pointer for gaze guidance in laparoscopic surgery.

A challenge of laparoscopic surgery is learning how to interpret the indirect view of the operative field. Acquiring professional vision-understanding what to see and which information to attend to, is thereby an essential part of laparoscopic training and one in which trainers exert great effort to convey. We designed a virtual pointer (VP) that enables experts to point or draw free-hand sketches over an intraoperative laparoscopic video for a novice to see. This study aimed to investigate the efficacy of the virtual pointer in guiding novices' gaze patterns. We conducted a counter-balanced, within-subject trial to compare the novices' gaze behaviors in laparoscopic training with the virtual pointer compared to a standard training condition, i.e., verbal instruction with un-mediated gestures. In the study, seven trainees performed four simulated laparoscopic tasks guided by an experienced surgeon as the trainer. A Tobii Pro X3-120 eye-tracker was used to capture the trainees' eye movements. The measures include fixation rate, i.e., the frequency of trainees' fixations, saccade amplitude, and fixation concentration, i.e., the closeness of trainees' fixations. No significant difference in fixation rate or saccade amplitude was found between the virtual pointer condition and the standard condition. In the virtual pointer condition, trainees' fixations were more concentrated (p = 0.039) and longer fixations were more clustered, compared to the Standard condition (p = 0.008). The virtual pointer effectively improved surgical trainees' in-the-moment gaze focus during the laparoscopic training by reducing their gaze dispersion and concentrating their attention on the anatomical target. These results suggest that technologies which support gaze training should be expert-driven and intraoperative to efficiently modify novices' gaze behaviors.

Characterization and validation of multimodal annihilation-gamma/near-infrared/visible laparoscopic system.

.Minimally invasive robotic surgery using fluorescence-guided images with a video laparoscope has been widely used because of its advantages of small incision, fast recovery time, and efficiency. However, the penetration depth limitation of fluorescence is a disadvantage caused by the absorption and scattering in tissues and blood cells. If this limitation can be overcome by additional imaging modalities, the surgical procedure can be quite efficient and precise. High-energy annihilation-gamma photons have a stronger penetration capability than visible and fluorescence photons. To characterize and validate a multimodal annihilation-gamma/near-infrared (NIR)/visible laparoscopic imaging system, an internal detector composed of an annihilation-gamma detector and an optical system was assembled inside a surgical stainless pipe with an outer diameter of 15.8 mm and an external detector with a dimension of placed at the opposite side of the internal detector. Integrated images of 511-keV gamma rays, NIR fluorescence, and visible light were obtained simultaneously. The 511-keV gamma image could be clearly seen with the acquisition of 5 s, while NIR and visible images could be presented in real time. This multimodal system has the potential for improving the surgery time and the quality of patient care.

Targeted surgical coaching can improve operative self-assessment ability: A single-blinded nonrandomized trial

BackgroundThe aim of the study was to investigate the effect of targeted surgical coaching on self-assessment of laparoscopic operative skill. Accurate self-assessment is vital for autonomous professional development. Surgical coaching can be used for performance improvement, but its role in this domain has been insufficiently investigated. MethodsThis was a single site, nonrandomized, interrupted time series design trial. Participants were residents, fellows, and attending surgeons regularly performing laparoscopic general surgery operations. Each participant was enrolled in an individualized coaching program using review of personal and peer laparoscopic videos. The program involved 3 to 5 sessions over a period of 6 to 19 weeks. Coaching used case debriefing to target self-assessment proficiency, with a focus on objective interpretation of observations and facilitative capacity building. The primary outcome measure was self-assessment accuracy and correlation to expert ratings. The Objective Structured Assessment of Technical Skill global rating scale was utilized for evaluation. ResultsTwelve participants were recruited and completed the coaching program. At baseline, there was no correlation between self-assessment and expert ratings. After completion of the coaching program there was correlation between self-assessment and expert ratings (P = .003) and improved self-assessment accuracy compared to baseline (P = .041). ConclusionThis study has demonstrated that targeted coaching using video review of laparoscopic cases can improve operative self-assessment accuracy using the Objective Structured Assessment of Technical Skill.

Predicting Surgical Phases using CNN-NARX Neural Network

Abstract Online recognition of surgical phases is essential to develop systems able to effectively conceive the workflow and provide relevant information to surgical staff during surgical procedures. These systems, known as context-aware system (CAS), are designed to assist surgeons, improve scheduling efficiency of operating rooms (ORs) and surgical team and promote a comprehensive perception and awareness of the OR. State-of-the-art studies for recognizing surgical phases have made use of data from different sources such as videos or binary usage signals from surgical tools. In this work, we propose a deep learning pipeline, namely a convolutional neural network (CNN) and a nonlinear autoregressive network with exogenous inputs (NARX), designed to predict surgical phases from laparoscopic videos. A convolutional neural network (CNN) is used to perform the tool classification task by automatically learning visual features from laparoscopic videos. The output of the CNN, which represents binary usage signals of surgical tools, is provided to a NARX neural network that performs a multistep-ahead predictions of surgical phases. Surgical phase prediction performance of the proposed pipeline was evaluated on a dataset of 80 cholecystectomy videos (Cholec80 dataset). Results show that the NARX model provides a good modelling of the temporal dependencies between surgical phases. However, more input signals are needed to improve the recognition accuracy.

Future-State Predicting LSTM for Early Surgery Type Recognition.

This work presents a novel approach for the early recognition of the type of a laparoscopic surgery from its video. Early recognition algorithms can be beneficial to the development of "smart" OR systems that can provide automatic context-aware assistance, and also enable quick database indexing. The task is however ridden with challenges specific to videos belonging to the domain of laparoscopy, such as high visual similarity across surgeries and large variations in video durations. To capture the spatio-temporal dependencies in these videos, we choose as our model a combination of a convolutional neural network (CNN) and long short-term memory (LSTM) network. We then propose two complementary approaches for improving early recognition performance. The first approach is a CNN fine-tuning method that encourages surgeries to be distinguished based on the initial frames of laparoscopic videos. The second approach, referred to as " Future-State Predicting LSTM," trains an LSTM to predict information related to future frames, which helps in distinguishing between the different types of surgeries. We evaluate our approaches on a large dataset of 425 laparoscopic videos containing nine types of surgeries (Laparo425), and achieve on average an accuracy of 75% having observed only the first 10 min of a surgery. These results are quite promising from a practical standpoint and also encouraging for other types of image-guided surgeries.