Objective assessment of surgical competence in gynaecological laparoscopy: development and validation of a procedure‐specific rating scale

Abstract

The purpose of this study was to develop a global- and a procedure-specific rating scale based on a well-validated generic model (objective structured assessment of technical skills) for assessment of technical skills in laparoscopic gynaecology. Furthermore, we aimed to investigate the construct validity and the interrater agreement (IRA) of the rating scale. We investigated both the gamma coefficient (Kendall's rank correlation), which is a measure of the strength of dependence between observations, and the kappa value for each of the ten individual items included in the rating scale. Prospective cohort, observer-blinded study. Departments of Obstetrics and Gynaecology in Zealand, Denmark. Twenty one gynaecologists or gynaecological trainees. Twenty-one video recordings of right side laparoscopic salpingectomies were collected prospectively, eight from novices (defined as <10 procedures), seven from intermediate experienced (20-50 procedures) and six from experts (> 200 procedures). All operations were performed by the same operative principles and using a standardised technique. The recordings were analysed by two independent, blinded observers. Construct validity of the rating scale based on operative performance (median of total score) and interrater reliability. There were significant differences between the three groups: median score of novices 24.00 versus intermediate 29.50 versus expert 39.50, P < 0.003) The IRA was 0.83 overall. The gamma correlation coefficient was 0.91. The kappa values varied from 0.510-0.933 for each of the individual items of the rating scale. The procedure-specific rating scale for laparoscopic salpingectomy is a valid and reliable tool for assessment of technical skills in gynaecological laparoscopy.