Development of a Bronchoscopy-Radiologic Skills and Task Assessment Tool (BRadSTAT): A Tool for Evaluating the Radiological Skills of Bronchoscopists with Different Experience

Abstract

Background: Competency using radiologic images for bronchoscopic navigation is presumed during subspecialty training, but no assessments objectively measure combined knowledge of radiologic interpretation and ability to maneuver a bronchoscope into peripheral airways. Objectives: The objectives of this study were (i) to determine whether the Bronchoscopy-Radiology Skills and Tasks Assessment Tool (BRadSTAT) discriminates between bronchoscopists of various levels of experience and (ii) to improve construct validity using study findings. Methods: BRadSTAT contains 10 questions that assess chest X-ray and CT scan interpretation using multiple images per question and 2 technical skill assessments. After administration to 33 bronchoscopists (5 Beginners, 9 Intermediates, 10 Experienced, and 9 Experts), discriminative power was strengthened using differential weighting on CT-related questions, producing the BRadSTAT-CT score. Cut points for both scores were determined via cross-validation. Results: Mean BRadSTAT scores for Beginner, Intermediate, Experienced, and Expert were 74 (±13 SD), 78 (±14), 86 (±9), and 88 (±8), respectively. Statistically significant differences were noted between Expert and Beginner, Expert and Intermediate, and Experienced and Beginner (all p ≤ 0.05). Mean BRadSTAT-CT scores for Beginner, Intermediate, Experienced, and Expert were 63 (±14), 74 (±15), 82 (±13), and 90 (±9), respectively, all statistically significant (p ≤ 0.03). Cut points for BRadSTAT-CT had lower sensitivity but greater specificity and accuracy than for BRadSTAT. Conclusion: BRadSTAT represents the first validated assessment tool measuring knowledge and skills for bronchoscopic access to peripheral airways, which discriminates between bronchoscopists of various experience levels. Refining BRadSTAT produced the BRadSTAT-CT, which had higher discriminative power. Future studies should focus on their usefulness in competency-based bronchoscopy programs.