MP10-10 COMPLEXITIES IN ANNOTATING SURGICAL VIDEOS TO BUILD SUPERVISED DEEP LEARNING MODELS FOR LAPAROSCOPIC LIVE DONOR NEPHRECTOMY

Abstract

You have accessJournal of UrologySurgical Technology & Simulation: Instrumentation & Technology I (MP10)1 Sep 2021MP10-10 COMPLEXITIES IN ANNOTATING SURGICAL VIDEOS TO BUILD SUPERVISED DEEP LEARNING MODELS FOR LAPAROSCOPIC LIVE DONOR NEPHRECTOMY Mahendra Bhandari, Hamid Ali, Mahesh Desai, C. Mallikarjuna, Aneesh Srivastava, Deepak Dubey, Vipin Tyagi, Rajesh Ahlawat, Carolyn Pratt, Gautum Ram Chaudhary, Trevor Zeffiro, and Madhu Reddiboina Mahendra BhandariMahendra Bhandari More articles by this author , Hamid AliHamid Ali More articles by this author , Mahesh DesaiMahesh Desai More articles by this author , C. MallikarjunaC. Mallikarjuna More articles by this author , Aneesh SrivastavaAneesh Srivastava More articles by this author , Deepak DubeyDeepak Dubey More articles by this author , Vipin TyagiVipin Tyagi More articles by this author , Rajesh AhlawatRajesh Ahlawat More articles by this author , Carolyn PrattCarolyn Pratt More articles by this author , Gautum Ram ChaudharyGautum Ram Chaudhary More articles by this author , Trevor ZeffiroTrevor Zeffiro More articles by this author , and Madhu ReddiboinaMadhu Reddiboina More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000001983.10AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Annotating surgical videos is the key step in building computer vision (CV) models to develop real-time surgical tools. Surgical annotation is time intensive and requires combined skill sets of domain knowledge and technical experts. It requires consensus among multiple annotators for labeling rules, methods for conflict resolution, and for addressing errors. We developed a CV model for use in an immersive AR/VR technology to provide surgeons guidance to navigate through opaque surgical tissues in real time during donor nephrectomy. We identified annotation challenges and solutions to improve annotation processes. METHODS: As a part of MICCAI 2018 challenge we built a model on porcine kidney surgery images. The model could successfully identify 8 of 11 objects with a high degree of accuracy. In order to move this towards a useful surgical tool, we trained the model on frames from 22 videos of live donor nephrectomy provided by partner centers which were annotated by three qualified urology and transplant specialists (A1, A2, A3). However, due to discrepancies between annotators as well as inconsistencies in how an individual annotator labeled different frames, the model performed poorly. A consensus meeting took place between the annotators to resolve conflicts. To improve the performance of the model, we focused first on images processed by A1 and cleaned the data. By removing 325 images with clear conflicts between consecutive frames, we identified 1499 images suitable for use. 20% of these images (299) were set aside for testing and the model was trained on the remainder. RESULTS: The model correctly predicted the presence of renal artery in 49 of 91 frames where the annotator labeled it, renal vein in 190 of 207 frames, and spleen in 72 of 75 frames. IOUs for background, renal artery, renal vein, and spleen were 0.959, 0.156, 0.553, and 0.596 respectively (see table). We hope to improve model performance with larger data sets incorporating annotators' consensus. Data from multiple annotators will allow us to automate data cleaning by removing frames with insufficient agreement between annotators. CONCLUSIONS: Accurate CV models are essential to the growth of automation in surgery and require the work of urological surgeons to manually label data in coordination with technical experts. Source of Funding: This material is based upon work supported by the National Science Foundation under SBIR Grant No. 1953822 © 2021 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 206Issue Supplement 3September 2021Page: e172-e172 Advertisement Copyright & Permissions© 2021 by American Urological Association Education and Research, Inc.MetricsAuthor Information Mahendra Bhandari More articles by this author Hamid Ali More articles by this author Mahesh Desai More articles by this author C. Mallikarjuna More articles by this author Aneesh Srivastava More articles by this author Deepak Dubey More articles by this author Vipin Tyagi More articles by this author Rajesh Ahlawat More articles by this author Carolyn Pratt More articles by this author Gautum Ram Chaudhary More articles by this author Trevor Zeffiro More articles by this author Madhu Reddiboina More articles by this author Expand All Advertisement Loading ...