Expert and Crowdsourced Evaluation of Image Quality From a Novel Endoscopy Phone Light Adapter

Abstract

To evaluate the image quality of cystourethroscopy using a novel 3D printed phone light adapter with subject expert and crowdsourced evaluators. A simple 3D printed light adaptor for a flexible cystoscopy was developed and made open source. Two videos were then recorded of a simulated cystourethroscopy, one using the novel adapter and the other using a traditional endoscopy light source. Expert evaluators (urology trainees and attendings) were then asked to evaluate the video quality using a double stimulus impairment scale. They were also asked to rate their level of confidence in using the novel adapter in clinical scenarios. Using Amazon's Mechanical Turk marketplace, 100 crowdsourced evaluators viewed the same videos and completed the same rating scale. The Mann-Whitney U test was then used to compare the expert and crowdsourced ratings. Expert and crowdsourced evaluators saw minimal degradation of video quality for the simulated urethroscopy (P= .66). However, while expert evaluators did identify degradation in the cystoscopy video, the crowdsourced evaluators did not (P=.012). 96% of the expert evaluators would either "often" or "always" use the novel adapter for difficult Foley placements and removal of ureteric stents. The novel light adapter caused minimal degradation in image quality for urethroscopy as compared to a traditional endoscopy light source, with vast majority of raters believing it would be adequate to perform common bedside cystoscopy procedures.