You have accessJournal of UrologyCME1 May 2022MP32-03 AUTOMATED SEGMENTATION OF KIDNEY STONES DURING URETEROSCOPY USING COMPUTER VISION TECHNIQUES Shaan Setia, Zachary Stoebner, Daiwei Lu, Ipek Oguz, and Nicholas Kavoussi Shaan SetiaShaan Setia More articles by this author , Zachary StoebnerZachary Stoebner More articles by this author , Daiwei LuDaiwei Lu More articles by this author , Ipek OguzIpek Oguz More articles by this author , and Nicholas KavoussiNicholas Kavoussi More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000002581.03AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Visualization and image quality during ureteroscopy is essential for stone tracking during fragmentation. However, this can impacted by blood and debris intraoperatively. Novel applications of computer vision methods may augment endoscopic visualization and improve stone tracking. We sought to use computer vision techniques to automatically segment kidney stones in real-time during ureteroscopy. METHODS: Twenty separate videos of ureteroscopy were collected with digital ureteroscopes (Karl Storz Flex Xc). Frames from each video were extracted at 20 frames per second (fps). The training data was manually annotated to identify the stone in each frame. Three established image segmentation computer vision models, U-Net, U-Net++, and DenseNet, were trained using the annotated frames. Different iterations of each model were compared via hyperparameter tuning for optimization. Eighty percent of the data was used to train the models with 10% being used for validation and 10% being used for testing. Outcomes included Dice similarity coefficient (DSC), accuracy (per pixel), and area under the receiver operating characteristic curve (ROC-AUC). RESULTS: Mean duration of video was 22 seconds (SD±13s). The U-Net++ and U-Net models demonstrated similarly strong performances for stone segmentation with DSCs of 0.92 and 0.91, accuracies of 0.95 and 0.96, and ROC-AUC scores of 0.97 and 0.97, respectively. DenseNet had lower performance comparatively with a DSC of 0.73, accuracy of 0.87, and ROC-AUC of 0.91 (Fig. 1). Additionally, we implemented a working system for the processing of real-time video feeds with overlayed model predictions (Fig. 2). The models were able to annotate new videos at 30 fps and maintain accuracy. CONCLUSIONS: Computer vision models demonstrate strong performance for automatic stone segmentation during ureteroscopy. Annotating new videos at 30 fps demonstrates the feasibility of their application real-time during surgery. With further optimization, these models could augment ureteroscopic vision and improve stone treatment. Source of Funding: None © 2022 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 207Issue Supplement 5May 2022Page: e532 Advertisement Copyright & Permissions© 2022 by American Urological Association Education and Research, Inc.MetricsAuthor Information Shaan Setia More articles by this author Zachary Stoebner More articles by this author Daiwei Lu More articles by this author Ipek Oguz More articles by this author Nicholas Kavoussi More articles by this author Expand All Advertisement PDF DownloadLoading ...