Validation of the Carotid Intima–Media Thickness Variability: Can Manual Segmentations Be Trusted as Ground Truth?

Abstract

The common carotid artery intima–media thickness (IMT) is widely accepted and used as an indicator of atherosclerosis. Recent studies, however, have found that the irregularity of the IMT along the carotid artery wall has a stronger correlation with atherosclerosis than the IMT itself. We set out to validate IMT variability (IMTV), a parameter defined to assess IMT irregularities along the wall. In particular, we analyzed whether or not manual segmentations of the lumen–intima and media–adventitia can be considered reliable in calculation of the IMTV parameter. To do this, we used a total of 60 simulated ultrasound images with a priori IMT and IMTV values. The images, simulated using the Fast And Mechanistic Ultrasound Simulation software, presented five different morphologies, four nominal IMT values and three different levels of variability along the carotid artery wall (no variability, small variability and large variability). Three experts traced the lumen–intima (LI) and media–adventitia (MA) profiles, and two automated algorithms were employed to obtain the LI and MA profiles. One expert also re-traced the LI and MA profiles to test intra-reader variability. The average IMTV measurements of the profiles used to simulate the longitudinal B-mode images were 0.002 ± 0.002, 0.149 ± 0.035 and 0.286 ± 0.068 mm for the cases of no variability, small variability and large variability, respectively. The IMTV measurements of one of the automated algorithms were statistically similar (p > 0.05, Wilcoxon signed rank) when considering small and large variability, but non-significant when considering no variability (p < 0.05, Wilcoxon signed rank). The second automated algorithm resulted in statistically similar values in the small variability case. Two readers' manual tracings, however, produced IMTV measurements with a statistically significant difference considering all three variability levels, whereas the third reader found a statistically significant difference in both the no variability and large variability cases. Moreover, the error range between the reader and automatic IMTV values was approximately 0.15 mm, which is on the same order of small IMTV values, indicating that manual and automatic IMTV readings should be not used interchangeably in clinical practice. On the basis of our findings, we conclude that expert manual tracings should not be considered reliable in IMTV measurement and, therefore, should not be trusted as ground truth. On the other hand, our automated algorithm was found to be more reliable, indicating how automated techniques could therefore foster analysis of the carotid artery intima–media thickness irregularity.