Validation of QyScore’s® fully automated quantitative image segmentation tools against expert manual gold‐standards

Abstract

AbstractBackgroundQuantitative imaging provides valuable information for early detection, disease progression and treatment response monitoring. Manual segmentation is the current gold‐standard, but is prohibitively labour‐intensive for large scale use, such as in clinical routine, and suffers from individual variability. Thus, there is a substantial unmet need for validation of automatic segmentation techniques that perform as accurately as this labour‐intensive manual gold‐standard.MethodThe validation cohort consisted of 50 individuals with multiple diagnoses, wide age range (18‐86 yrs), balanced for sex and acquired on multiple scanners for improved generalizability of quantification results (Table 1). Three expert neuroradiologists each manually segmented the caudate, putamen, globus pallidus, thalamus, cerebellum, brainstem, and lateral ventricles on 3DT1 images using itk‐SNAP (http://www.itksnap.org/) software. A consensus expert segmentation was then derived using the STAPLE (Simultaneous Truth and Performance Level Estimation) algorithm (Wakefield et al., (2004)) for each region and compared with the fully automated segmentations produced using QyScore®, a CE‐marked and FDA‐cleared neuroimaging medical device. Performance was investigated using the Dice Similarity Coefficient (DSC) and concordance assessed with plotted linear regression.ResultMean, standard deviation and confidence intervals of the DSC demonstrated a high degree of agreement between the consensus manual gold‐standard and QyScore®’s automated segmentations (Table 2). Importantly, this agreement remained consistent following stratification by field strength, demonstrating generalizability to most clinical imaging centres. For 1.5T scanners the mean DSC was 0.81, 0.89, 0.79, 0.86, 0.94, 0.93 and 0.91 for the caudate, putamen, globus pallidus, thalamus, cerebellum, brainstem, and lateral ventricles respectively. For the 3T scanners, this was equivalent or marginally better at 0.84, 0.89, 0.82, 0.86, 0.94, 0.94, and 0.93 respectively. In addition, strong concordance was demonstrated between the volumes, expressed as a percentage of intracranial volume (%ICV), obtained by the automated medical device QyScore® and the consensus of the manual gold‐standard segmentation (Figure 1: Table 2).ConclusionQyScore® produces fast reliable automated segmentations with comparable accuracy to expert neuroradiologists. These findings support the implementation of QyScore® in clinical trials and in clinical routine to provide quantitative image analysis in support of diagnosis and monitoring of disease progression and treatment response.