Inter-Observer Agreement On Assessment of Photo Documentation in Bleeding Peptic Ulcer

Abstract

Background: Photo documentation is readily available at many endoscopic units. To date, however, there are few published evaluations on the utility of this technique in confirming the main findings of the endoscopy. Methods: In order to study the level of inter-observer agreement in assessment of photo documentation, all endoscopists who contributed to a large multinational study in peptic ulcer bleeding (PUB study; NCT00251979) were invited to assess the photo documentation of 25 cases from the study. Each case was classified by assessing the stigmata of recent bleeding according to the Forrest classification (6 classes) or as ‘not assessable’. The assessments were made without knowledge about the classification previously performed by the original endoscopist and, independently, by the two members of the Endpoint Committee (EpC). Thirteen of the 25 cases selected were deemed ‘excellent’ (i.e. agreement between the two EpC members and the original investigator). The remaining 12 cases yielded some degree of disagreement. Kappa scores were calculated for each Forrest grade, for all assessors combined and according to the level of assessor experience with endoscopic procedures (>10y, ‘experienced’; and ≤10y, ‘less experienced’). Results: A total of 115 assessors (endoscopists/investigators) completed the survey. The majority specialized in gastroenterology (n=83); the remainder were primarily surgeons (n=23). 52 assessors were experienced with endoscopic procedures (>10y), while 59 assessors were considered less experienced (≤10y); experience status was unknown for 4 assessors. For the 13 cases considered excellent, 70% of the assessors agreed with the classification made by the original investigator. For the remaining 12 cases, the corresponding value was 21%. The mean (95% CI) kappa score between all assessors was 0.51 (0.51, 0.52). The mean (95% CI) kappa score for each Forrest grade were: Ia (spurting), 0.68 (0.67, 0.68); Ib (oozing), 0.51 (0.51, 0.52); IIa (non-bleeding visible vessel), 0.35 (0.34, 0.35); and IIb (adherent clot), 0.39 (0.39, 0.40). Similar findings were apparent when kappa scores were examined according to the assessors' level of endoscopic experience. Conclusions: Bleeding peptic ulcer can be adequately documented by endoscopic photos, the highest inter-observer agreement being apparent in Forrest class Ia. Supported by AstraZeneca R&D, Mölndal, Sweden.