Remote evaluation of laparoscopic performance using the global operative assessment of laparoscopic skills

Abstract

Although numerous assessment tools currently exist to evaluate laparoscopic surgical skills, no studies have demonstrated the reliability of such tools when used with telementoring technology. This study aimed to determine the reliability of the Global Operative Assessment of Laparoscopic Skills (GOALS) rating scale for assessing laparoscopic skills remotely and to identify how factors unique to remote assessment such as bandwidth and image quality influence its reliability. Four trained observers evaluated 19 participants for their technical performance during a laparoscopic cholecystectomy using the GOALS assessment tool. One observer assessed the study participants directly in the operating room, whereas the three remaining observers were randomly assigned and blinded to a high- (1.5 Mbps), medium- (256 kbps), or low- (64.4 kbps) bandwidth restriction and observed remotely via Skype. The Maryland Visual Comfort Scale was used to evaluate the video quality of the respective connections. The intraclass correlation coefficient (ICC) calculated for the total GOALS score demonstrated a statistically significant correlation of high, medium, and low bandwidths respectively with ICC 0.693 (95 % confidence interval [CI], 0.226-0.883), 0.518 (95 % CI 0.089-0.783), and 0.499 (95 % CI 0.025-0.781). There was a statistically significant difference in the overall perceived visual quality between the high/low (Z = -3.222; P = 0.001) and the medium/low (Z = -3.567; P < 0.001) bandwidth comparison but no difference between the high/medium bandwidths (Z = -0.610; P = 0.542). The data suggest that the GOALS assessment tool retains its reliability for intraoperative assessment of laparoscopic skills when used remotely. This is a key requirement in telesimulation programs allowing for structured feedback between the mentor and the mentee. This study quantifies the effect that bandwidth has on the reliability of remote assessment, demonstrating that higher bandwidths improve the utility of these tools.