Development, Feasibility, Validity, and Reliability of a Scale for Objective Assessment of Operative Performance in Laparoscopic Gastric Bypass Surgery

Abstract

There is no objective scale for assessment of operative skill in laparoscopic gastric bypass (LGBP). The objective of this study was to develop and demonstrate feasibility of use, validity, and reliability of a Bariatric Objective Structured Assessment of Technical Skill (BOSATS) scale. The BOSATS scale was developed using a hierarchical task analysis (HTA), a Delphi questionnaire, and a panel of international experts in bariatric surgery. The feasibility of use, reliability, and validity of the developed scale were demonstrated by reviewing 52 prospectively collected video recordings of LGBP performed by novice and experienced surgeons. A total of 214 discrete steps were identified in HTA. A total of 12 and 17 panel members completed the first and second round of the Delphi questionnaire, respectively. Consensus among the panel was achieved after the second round (Cronbach's alpha = 0.85). The BOSATS scale demonstrated high inter-rater (intraclass correlation coefficient [ICC] = 0.954; p < 0.001) and test-retest reliability (ICC = 0.99; p < 0.001). Significant differences between BOSATS scores of experienced and novice surgeon groups were noted for the creation of jejunojejunostomy (JJ), gastric pouch, linear stapled gastrojejunostomy (GJ), circular stapled GJ, and hand-sewn GJ. Moderate to high correlations between BOSATS scale and Objective Structured Assessment of Technical Skills Global Rating Scale (OSATS GRS) were seen for JJ (rho = 0.59; p = 0.001), gastric pouch (rho = 0.48; p = 0.0004), linear stapled GJ (rho = 0.70; p = 0.0001), and hand-sewn GJ (rho = 0.96; p < 0.0001). The BOSATS scale is a feasible to use, reliable, and valid instrument for objective assessment of operative performance in LGBP. Implementation of this scale is expected to facilitate deliberate practice and provide a means for future certification in bariatric surgery.