Recognizing multiple needle insertion attempts for performance assessment in central venous catheterization training

Abstract

<strong>Purpose: </strong>Computer-assisted skill assessment has traditionally been focused on general metrics related to tool motion and usage time. While these metrics are important for an overall evaluation of skill, they do not address critical errors made during the procedure. This study examines the effectiveness of utilizing object detection to quantify the critical error of making multiple needle insertion attempts in central venous catheterization. <strong>Methods:</strong> 6860 images were annotated with ground truth bounding boxes around the syringe attached to the needle. The images were registered using the location of the phantom, and the bounding boxes from the training set were used to identify the regions where the needle was most likely inserting the phantom. A Faster region-based convolutional neural network was trained to identify the syringe and produce the bounding box location for images in the test set. A needle insertion attempt began when the location of the predicted bounding box fell within the identified insertion region. To evaluate this method, we compared the computed number of insertions to the number of insertions identified by human reviewers. <strong>Results:</strong> The object detection network had an overall mean average precision (mAP) of 0.71. This tracking method computed an average of 4.40 insertion attempts per recording compared to a reviewer count of 1.39 attempts per recording. <strong>Conclusions:</strong> The difference in the number of insertion attempts identified by the computer and reviewers decreases with an increasing mAP, making this method suitable for detecting multiple needle insertions using an object detection network with a high accuracy.