A Prospective Assessment of Inter-Observer Agreement Among Endosonographers in the Classification and Diagnosis of Pancreatic Cysts

Abstract

Introduction: Pancreatic cystic lesions (PCLs) are commonly investigated with endoscopic ultrasound (EUS). Interobserver agreement (IOA) between radiologists to classify PCLs or between Endosonographers (ESs) in characterizing PCLs has been reported as poor to moderate. In this prospective blinded study, we sought to assess the IOA between expert ESs in a step-wise process to diagnose PCLs Methods: As part of a recently concluded multicenter, single blinded randomized controlled trial comparing standard and flexible nitinol needles in fine needle aspiration (FNA) of PCLs, consecutive patients who underwent EUS-FNA of pancreatic cysts were included. Only patients who did not undergo surgical resection during the study were included, as a “clinical diagnosis” was essential in the absence of surgical pathology. Clinical, imaging, cytology and biochemical/tumor marker results from 175 patients were independently reviewed by a consensus board of two expert ESs (ES1 and ES2) blinded to randomization group. An independent diagnosis was provided after each step of a 3-tier process mimicking clinical practice: Step 1-Clinical presentation and radiological/EUS imaging; Step 2- Physical characteristics of the aspirate (color and viscosity); Step 3- CEA/Cytology results. A “final” diagnosis was made by the amalgamation of the observations of both the ESs (Consensus board diagnosis (CBDx)). When ES1 and ES2 differed in diagnosis after step 3, a third ES (ES3) was invited to weigh in and consolidate a CBDx Results: Cohen's Kappa was calculated to determine IOA among ESs (Table 1). Agreement between ES1 and ES2 after Step1, Step2 and Step3 was 95%, 94% and 95% respectively. Kappa between ES1 and ES2 at similar junctures was 0.91, 0.88 and 0.91 respectively.Since the proportion of cases involving ES3 was very small, this reviewer was excluded from the analysis. In order to assess the cumulative impact of data provided in each step in the final diagnosis (diagnostic progression throughout the process), McNemar Bowker CS test was calculated (Table 2). As ES2 progressed from Step1 to CBDx, there were significant changes from Step-1 to Step-3; Step-1 and CBDx as well as Step-2 and CBDx (all p<0.05), but a similar trend was not observed in ES1 Conclusion: IOA between expert ESs in a multi-step diagnostic process of PCLs was almost perfect. FNA-derived data resulted in a significant impact on the final diagnosis, highlighting the incremental role FNA plays in the diagnosis of PCLs798_A Figure 1. Summary of Inter-Observer Agreement Analysis798_B Figure 2. Summary of Pre-step and Post-step analysis of changes/progression in diagnosis