Assessment of observer variation in measuring the radiographic vertebral index in patients with multiple myeloma

Abstract

The Radiographic Vertebral Index (RVI) was assessed as a possible outcome measure for bone disease in myeloma by evaluating within and between reader reproducibility. Four readers (2 radiologists and 2 clinical hematologists) independently scored, on two separate occasions, the RVI on 40 radiographs from patients with myeloma. Each vertebra from third thoracic (T3) to fifth lumbar (L5) received a score of “1” if normal, “2” if biconcave and “4” if crushed or fractured. RVI global scores, therefore, could vary from a minimum of 15, for no damage, to a potential maximum of 60 in which all vertebrae are collapsed. Agreement was determined for each vertebra using crude percentage agreement and the kappa statistic (which corrects for chance-expected agreement) for categorical data, and for global score using analysis of variance and calculating intra-class correlation. With increasing mean abnormality score on individual vertebrae there was a corresponding increase in kappa and reduction in crude percentage agreement. Within readers, the mean percentage agreement across all vertebrae varied from 85.6 to 90.3% and the observed differences just reach statistical significance ( p = 0.048). Mean kappa values ranged from 0.48 to 0.63 and were similar across readers. Differences in intra-reader agreement were not related to subspecialty. When between reader percentage agreement and kappa scores were assessed for radiologists and non-radiologist clinicians, no difference could be detected. Agreement with respect to intra-reader mean global RVI scores was excellent as illustrated by the intra-class correlation coefficient which varied from 0.89 to 0.94. The mean intra-class correlation for radiologists was 0.92, compared with 0.91 for non-radiologists. Inter-reader agreement for global RVI score was assessed by a repeated measures analysis of variance of mean global scores and a highly significant difference ( p < 0.001) was detected among the four readers. This indicates an important extra component of variance between readers over and above the variation within readers. The data show reasonably good reproducibility of the RVI within readers and suggests that sequential RVI assessments on individual patients should be performed by a single observer. The RVI may be a useful outcome measure for evaluating bone disease in myeloma, but needs to be validated prospectively in the setting of a clinical trial.