PASS (Penn Assessment of Surgical Skills) -validation of a new tool to assess technical and nontechnical skills of surgical residents

Abstract

Introduction To maintain high surgical performance, attention needs to be paid both to residents' technical and non-technical skills. An integrated tool may be useful to assess overall performance in OR. Aim: To validate a new residents' evaluation tool, Penn Assessment of Surgical Skills (PASS). Methods A prospective study was conducted with eight (n=8) PGY4 residents. Using case scenario and images, an attending assessed their pre-operative planning, OR dynamics, team work and technical skills for surgical management of a simulated abdominal aortic aneurysm in the operating room. Each resident performed at least one proximal or distal anastomosis using a bifurcated portex graft. The performance was scored using the PASS tool by the surgeon and an independent observer. To validate the tool, Kappa of 0.8 to1 was considered significant. Results The tool has 7 domains of assessment, 3 for non technical and 4 for technical skills [ table ]. Scores given were positive +2, negative −2 and neutral +1 for each sub area. Scores were lower for domains, 5, 6 and 7. They were graded into four levels based on score percentage: Level I) ≥80% - Excellent, II) 60-79% - Good; III) 40-59% - Average; IV) ≤40% - Poor. Kappa correlation was significant. Performances were Level I/II and III/IV for nontechnical and technical skills. Domains Rater I: Mean Score (%) Rater II: Mean Score (%) 1. Understanding the information and situation: Max score-6 5.3 (88%) 5 (83%) 2. Communication skills: Max score-4 3.33 (83%) 4 (100%) 3. Leadership qualities: Max score-8 6 (75%) 5.3 (66%) 4.Tissue handling: Max score-6 3.33 (55%) 3.33 (55%) 5. Dynamics of instruments, needle handling and application: Max score-6 −2 (—) 0.75 (—) 6. Suturing, knot tying and anastomotic techniques: Max score-4 2 (50%) 2.33 (58.3%) 7.Flow and application/ knowledge of techniques/steps: Max score-4 2.37 (59%) 2.37 (59%) Conclusions PASS is an integrated evaluation tool to assess residents' technical and non technical skills. It also gave an understanding of the weak areas of a resident. In order to validate it further, this tool needs to be applied in a live OR.