PURPOSE: To assess the effect of training and the inter- and intra-rater variability of a targeting curriculum for ultrasound guided invasive procedures. BACKGROUND: We previously demonstrated the content, face, and construct validity of our curriculum. Here we assess its inter- and intra-rater reliability. METHODS: Echogenic targets (wooden dowels, plastic straws, and wooden balls) were placed in water filled buckets covered with simulated skin (ballistic gelatin). We assessed 23 1st year medical students performing two targeting tasks (guiding the needle to multiple targets in a random sequence), held 4 weekly one hour training sessions, and reassessed their performance. Completion time in seconds, errors (as recorded by multiple assessors), and error adjusted time (5 sec penalty/error) were recorded. These measures were compared before and after training using a repeated measures t-test. Inter- and intra-rater inter-class correlation coefficients (ICC) were calculated. RESULTS: Completion time, errors, and error adjusted time on the dowel tasks were 176±70, 7.4±, 213±95, respectively before training, and 95±61, 2.7±3.4, and 109±74, respectively after training (all P<.0001). The completion time, errors, and error adjusted time on the straw task were 319±98, 9.4±4.6, and 366±114, respectively before training, and 179±57, 3.1±3.2, and 195±67, respectively after training (all P<.0001). The inter-rater ICC for the dowel and straw tasks were 0.83 and 0.91, respectively. The intra-rater ICCs were 0.93 and 0.90, respectively. DISCUSSION: Conclusions: Student performance on the targeting tasks improved markedly after 4 weekly training sessions. The inter- and intra-rater reliability was excellent. This provides additional validity evidence for our novel ultrasound guided invasive procedure task targeting curriculum.
Read full abstract