PD58-08 AUTOMATING SUTURING SKILLS ASSESSMENT WITH A LIMITED SURGEON DATASET: META LEARNING

Abstract

You have accessJournal of UrologySurgical Technology & Simulation: Training & Skills Assessment (PD58)1 Sep 2021PD58-08 AUTOMATING SUTURING SKILLS ASSESSMENT WITH A LIMITED SURGEON DATASET: META LEARNING Andrew J. Hung, Sirisha Rambhatla, Daniel I. Sanford, Nilay Pachauri, Jessica H. Nguyen, and Yan Liu Andrew J. HungAndrew J. Hung More articles by this author , Sirisha RambhatlaSirisha Rambhatla More articles by this author , Daniel I. SanfordDaniel I. Sanford More articles by this author , Nilay PachauriNilay Pachauri More articles by this author , Jessica H. NguyenJessica H. Nguyen More articles by this author , and Yan LiuYan Liu More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000002092.08AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Automating technical skills assessment with machine learning models can revolutionize how we credential surgeons, and it can identify critical errors before they become patient complications. Common to many machine learning challenges, having limited training data can prevent our ability to accurately project skills assessment for a wide population of surgeons. Herein, we utilized meta learning, a specialized form of machine learning, to learn from a limited set of surgeons, and adapt learned representations to new additional surgeons. METHODS: Training and faculty surgeons completed a suturing simulation task on the Mimic Flex VR robotic simulator. Video and kinematic data (instrument XYZ coordinates and pose) were collected. Three human raters utilized the video and a suturing assessment tool to provide ground truth labels for skill domains. Kinematic data was utilized by two baseline models and the proposed Meta learning model to project skills assessment. For each skill domain, surgeons with minimum two ideal and non-ideal performance examples were included. Data from five surgeons were randomly selected for the test set, remainder was utilized for training. Three two-layer LSTM-based models projected skills assessment: No-customization, Fine-tune (customized the base model for new surgeons), and Meta learning (not only customized predictions, but also updated the base model thereby maximizing learning). Best area under the curve (AUC) metric across epochs was used to assess performance. RESULTS: In total, 21 surgeons contributed performance data for analysis. 16 surgeons contributed examples to Needle positioning scoring, 19 to Needle entry angle, 20 to Needle driving, and 20 to Needle withdraw. For all skill domains, Meta learning achieved the highest performance in projecting the ground truth labels (AUC = 0.72-0.88) (Table). In all skill domains, Meta learning outperformed both the Fine-tune and No customization models. Fine-tune outperformed No customization in all domains, except for Needle withdrawal. CONCLUSIONS: Our initial experiments with Meta learning demonstrate that it can augment model performance in automated suturing skill assessment with a limited surgeon dataset by applying learned patterns to new additional surgeons. Overall, Meta learning projection results are very robust. Source of Funding: Research reported in this publication was supported in part by the National Institute Of Biomedical Imaging And Bioengineering of the National Institutes of Health under Award Number K23EB026493. © 2021 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 206Issue Supplement 3September 2021Page: e1019-e1020 Advertisement Copyright & Permissions© 2021 by American Urological Association Education and Research, Inc.MetricsAuthor Information Andrew J. Hung More articles by this author Sirisha Rambhatla More articles by this author Daniel I. Sanford More articles by this author Nilay Pachauri More articles by this author Jessica H. Nguyen More articles by this author Yan Liu More articles by this author Expand All Advertisement Loading ...