Non-technical skill assessments across levels of US surgical training

Abstract

BackgroundTo ensure safe patient care, regulatory bodies worldwide have incorporated non-technical skills proficiency in core competencies for graduation from surgical residency. We describe normative data on non-technical skill ratings of surgical residents across training levels using the US-adapted Non-Technical Skills for Surgeons (NOTSS-US) assessment tool. MethodsWe undertook an exploratory, prospective cohort study of 32 residents—interns (postgraduate year 1), junior residents (postgraduate years 2–3), and senior residents (postgraduate years 4–5)—across 3 US academic surgery residency programs. Faculty went through online training to rate residents, directly observed residents while operating together, then submitted NOTSS-US ratings on specific resident’s intraoperative performance. Mean NOTSS-US ratings (total range 4–20, sum of category scores; situation awareness, decision-making, communication/teamwork, leadership each ranged 1–5, with 1=poor, 3=average, 5=excellent) were stratified by residents’ training level and adjusted for resident-, rater-, and case-level variables, using mixed-effects linear regression. ResultsFor 80 operations, the overall mean total NOTSS-US rating was 12.9 (standard deviation, 3.5). The adjusted mean total NOTSS-US rating was 16.0 for senior residents, 11.6 for junior residents, and 9.5 for interns. Adjusted differences for total NOTSS-US ratings were statistically significant across the following training levels: senior residents to interns (6.5; 95% confidence interval, 4.3–8.7; P < .001), senior to junior residents (4.4; 95% confidence interval, 2.5–6.2; P < .001), and junior residents to interns (2.1; 95% confidence interval, 0.3–3.9; P = .017). Differences in adjusted NOTSS-US ratings across residents’ training levels persisted for individual NOTSS-US behavior categories. ConclusionThese data and online training materials can support US residency programs in determining competency-based performance milestones to develop surgical trainees’ non-technical skills.