Reliability of Cobb angle measurement using PACS in less-experienced medical personnel

Abstract

Background: Scoliosis is a common spinal malformation, which is typically diagnosed and assessed for severity by measuring the Cobb angle on a plain radiograph. Although manual methods are still considered the gold standard, recent studies have shown that PACS (picture archiving and communication systems methods) have comparable intraobserver and interobserver reliability levels for experienced clinicians. However, there is limited data for these reliability measures in less experienced medical personnel, especially with digital methods like PACS. Methods: 10 radiographic images of scoliosis patients were obtained via a retrospective chart review. After a brief training exercise, Cobb angle measurements were performed twice on each radiograph, separated by one week, using the PACS method. Subject groups ranged in their level of prior training and experience with Cobb angle measurements from inexperienced medical students, to more experienced resident physicians, and highly experienced orthopedic surgeons and radiologists. Data was compared for interrater reliability levels within groups and between groups, and for individual intra-rater reliability levels. Intra-class correlation coefficient (ICC) between averaged measured were calculated and a Cronbach’s α obtained. An α of ≥ 0.9 is considered excellent agreement, 0.9 > α ≥ 0.8 is good, 0.8 > α ≥ 0.7 is acceptable, 0.7 > α ≥ 0.6 is questionable and α < 0.6 is poor or unacceptable reliability. Results: Resident physicians, orthopedic surgeons, and radiologist subject groups had significant levels of between-group and within-group inter-rater reliability as well as individual intra-rater reliability levels (α ≥ 0.9, p 0.9, p<0.001) while the other had poor intra-rater reliability (α < 0.6, p=0.035). Between-group inter-rater reliability for medical students and more experienced professionals was poor (α < 0.6). Conclusions: While PACS has been shown to be a reliable method for measuring Cobb angle and an acceptable substitute for conventional manual techniques, it is not a reliable method in less experienced trainees such as medical students.