Validation of Ultrasound Guidance Targeting Tasks: Relationship of Performance and Clinical Experience.

Abstract

To assess whether health care provider performance on a novel task trainer and a set of newly created ultrasound guidance targeting tasks relates with their level of clinical experience. In a prospective cohort study, two assessment sessions were carried out in which inexperienced (n=21 and n=15) and experienced (n=14 and n=10) health care providers completed five previously described ultrasound guidance targeting tasks. Raw completion time, number of targeting errors, and error-adjusted completion time for each task were compared between groups with a t test for independent measures. Receiver operating characteristic analyses were performed to estimate whether error-adjusted completion time or number of errors could accurately differentiate between groups. Raw completion time was similar between groups. The number of errors and error-adjusted completion time were significantly lower in the experienced health care providers in all but the out-of-plane dowel task. The receiver operating characteristic analyses of number of errors demonstrated very high areas under the curve (0.93-0.98), sensitivities (100%), specificities (76-86%), positive likelihood ratios (4.2-7.0), and negative likelihood ratios (<0.001) indicating excellent differentiation between experienced and inexperienced health care providers. These values were notably lower in the receiver operating characteristic analyses of error-adjusted completion times. Task performance related well with a health care provider's level of clinical experience providing evidence of construct validity. For each task, we were able to determine a cutoff for number of errors that categorized experienced and inexperienced health care providers with very favorable sensitivity, specificity, positive likelihood ratios, and negative likelihood ratios. Our findings suggest it may be possible to use these cutoffs to objectively classify trainees as competent or not competent.