Inter-observer variability of visual analysis of “stress”-only adenosine first-pass myocardial perfusion imaging in relation to clinical experience and reading criteria

Abstract

To assess the inter-observer agreement of adenosine “stress”-only visual analysis of perfusion MR images in relation to experience and reading criteria. 106 adenosine perfusion MR examinations out of 350, 46 consecutive positive examinations and 60 randomly selected negative examinations were visually analysed by three individual readers (two residents and a technician) with different levels of experience. Readings (blinded for any information) were compared with the reading of an expert radiologist. After a month the examinations were presented again (randomly) without knowledge regarding the first readings. This time readings were performed with the systematical use of reading criteria. Agreement with the expert reading was good for the most experienced resident (k = 0.88). Kappa was 0.48 for the least experienced, and 0.57 for the technician. After the second systematical reading inter-observer agreement increased to 0.9, 0.68 and 0.77 respectively. Overall kappa increased from 0.59 to 0.71. The use of reading criteria significantly improved the performance of the least experienced reader (P = 0.01). Visual analysis of adenosine “stress”-only first-pass perfusion MR images has moderate to very good agreement. Performance is experience related, but the systematic use of reading criteria significantly increased performance for the least experienced observer.