Tu1106 Observer Agreement and Construct Validity in Central Endoscopic Assessment of Disease Activity in Ulcerative Colitis

Abstract

Background: Highly functional evaluative indices have low observer variability. Several endoscopic indices (EIs) are currently available to evaluate disease activity in ulcerative colitis (UC), however none of these instruments has been fully validated and their operating properties remain largely unknown. Central reading of EIs by is believed to improve consistency and quality in RCTs compared to site-reader scoring thereby increasing statistical efficiency. However, central reader observer variability across multiple EIs has not been studied. The construct validity of EIs and endoscopic predictors across multiple central readers is unclear. For these reasons, we assessed interand intra-observer agreement of central reading for the existing UC EIs.Methods: Seven expert readers independently reviewed 50 video recordings of sigmoidoscopic evaluation from patients with ulcerative colitis. Disease activity was assessed for each video on three separate occasions using the Modified Baron Score (MBS), Modified Mayo Clinic Score (MMCS), and the Ulcerative Colitis Endoscopic Index of Severity (UCEIS). A 100 mm Visual Analog Scale (VAS) global rating of severity was also completed. Interand intra-observer agreement were estimated for the EIs using a two-way random effects ANOVA model. A mixed model assessed the construct validity of endoscopic descriptors by measuring their correlation with VAS score. Results: Intraobserver intra-class correlation coefficients (ICCs) for the total MBS, MMCS, UCEIS, and VAS scores (95% CIs) were 0.88 (0.84-0.92), 0.89 (0.85-0.92), 0.89 (0.85-0.93), and 0.91 (0.88-0.94), respectively. Corresponding inter-rater agreement ICCs were 0.78 (0.71-0.85), 0.79 (0.72-0.85), 0.83 (0.77-0.88), and 0.78 (0.70-0.85). Intra-rater and inter-rater ICCs for UCEIS descriptors were: vascular pattern, 0.79 (0.73-0.84) and 0.68 (0.610.76); bleeding, 0.78 (0.73-0.83) and 0.64 (0.56-0.72); and erosions, 0.86 (0.82-0.90) and 0.77 (0.70-0.83). The correlation between each central reader's VAS and UCEIS descriptors are shown in Table 1. Conclusions: Central reading of EIs of UC severity by experts had excellent intraand interobserver agreement. These findings imply that central reading is highly reliable for the assessment of UC endoscopic severity. UCEIS descriptors substantially correlate with a global rating of endoscopic severity. Further study of the responsiveness and the predictive properties of EIs is merited to determine the optimal index for measuring endoscopic disease activity in UC. Table 1. Correlation between UCEIS descriptors and VAS