Deep Learning for Instrument Detection and Assessment of Operative Skill in Surgical Videos

Abstract

Surgical performance has been shown to be directly related to patient outcomes. There is significant variation in surgical performance and therefore a need to measure operative skill accurately and reliably. Despite this, current means of surgical performance assessment rely on expert observation which is labor-intensive, prone to rater bias and unreliable. We present an automatic approach to surgical performance assessment through the tracking of instruments in endoscopic video. We annotate the spatial bounds of surgical instruments in 2600 images and use this new dataset to train Mask R-CNN, a state-of-the-art instance segmentation framework. We show that we can successfully achieve spatial detection of surgical instruments by generating a pixel-by-pixel mask over the detected instrument and achieving an overall mAP of 0.839 for an IoU of 0.5. We leverage the results from our instrument detection framework to assess surgical performance through the generation of instrument trajectory maps and instrument metrics such as moving distance, smoothness of instrument movement and concentration of instrument movement.