Predicting Surgical Experience After Robotic Nerve-sparing Radical Prostatectomy Simulation Using a Machine Learning-based Multimodal Analysis of Objective Performance Metrics.

Abstract

Machine learning methods have emerged as objective tools to evaluate operative performance in urological procedures. Our objectives were to establish machine learning-based methods for predicting surgeon caseload for nerve-sparing robot-assisted radical prostatectomy using our validated hydrogel-based simulation platform and identify potential metrics of surgical expertise. Video, robotic kinematics, and force sensor data were collected from 35 board-certified urologists at the 2022 American Urological Association conference. Video was annotated for surgical gestures. Objective performance indicators were derived from robotic system kinematic data. Force metrics were calculated from hydrogel model integrated sensors. Data were fitted to 3 supervised machine learning models-logistic regression, support vector machine, and k-nearest neighbors-which were used to predict procedure-specific learning curve proficiency. Recursive feature elimination was used to optimize the best performing model. Logistic regression predicted caseload with the highest AUC score for 5/7 possible data combinations (force, 64%; objective performance indicators + gestures, 94%; objective performance indicators + force, 90%; gestures + force, 93%; objective performance indicators + gestures + force, 94%). Support vector machine predicted the highest AUC score for objective performance indicators (82%) and gestures (94%). Logistic regression with recursive feature elimination was the most effective model reaching 96% AUC in predicting case-specific experience. Most contributory features were identified across all model types. We have created a machine learning-based algorithm utilizing a novel combination of objective performance indicators, gesture analysis, and integrated force metrics to predict surgical experience, capable of discriminating between surgeons with low or high robot-assisted radical prostatectomy caseload with 96% AUC in a standardized, simulation-based environment.