Cranial ultrasound performance assessment tool: development and validation.

Abstract

A standard assessment tool for direct evaluation of procedural skills to ensure proficiency of trainees is necessary forcranial ultrasound (US) in clinical practice. This study created and validated an assessment tool for cranial US performanceby radiologists. An initial evaluation tool for cranial US using criteria was developed based on existing literature. The assessment form was modified using a three-round Delphi process by an expert panel, conducted betweenJanuary 2021 and April 2021. Rubric scales for grading were added once consensus regarding generated items was reached.Experts confirmed the final assessment tool using a rubric scale. Two raters evaluated cranial US performance of 27 residentsin video clips using the tool. Reliability and percent agreement were assessed. Seventeen pediatric radiologists working in different settings participated in the expert panel. The content validation of the proposed evaluation tool was enabledby expert pediatric radiologists. Following three rounds of the Delphi process, the initial 14-item assessment form became a final 15-item form. A three-part rubric scale was used in the final form (preparation, US machine operation, and cranial USperformance). Interrater reliability was evaluated with Cohen's Kappa. The Kappa value and percent interrater agreement for most items was moderate to almost perfect (0.42-0.93 and 77.8-100%, respectively). The Cronbach's alpha values for bothraters were 0.856 and 0.891. This study produced the first validated cranial US assessment tool using a modifiedDelphi method. The final assessment form is a simple and reliable tool.