Performance comparison of automated white matter lesion segmentation algorithms in the DELCODE Study

Abstract

AbstractBackgroundWhite matter hyperintensities (WMH), derived from T2‐weighted MRI, represent a sensitive imaging marker of cerebrovascular disease. To investigate WMH in large‐scale multicenter clinical studies, appropriate automated WMH segmentation algorithms are required. Therefore, prior evaluation of these methods is important, especially due to additional challenges posed by the multicenter acquisition. We assessed performance of different WMH segmentation algorithms compared to manual segmentation (reference) in the multicenter DZNE‐Longitudinal Cognitive Impairment and Dementia Study (DELCODE).MethodWMH segmentations were performed on a preliminary randomly‐selected dataset of 20 participants that included FLAIR images from different diagnostic groups across the AD continuum, various study sites and Siemens 3T scanner models. We used freely‐available and fully‐automated WMH segmentation methods, i.e. LST‐LGA, LST‐LPA, and Sysu_Media, the winner of the MICCAI WMH segmentation challenge. Performance of the algorithms was compared to reference segmentation using volumetric and spatial correspondence indices, intraclass correlation coefficient (ICC), Dice’s similarity coefficients (DSC), recall (sensitivity), and precision (positive predictive value), across the whole brain and with respect to the number of detected clusters.ResultPreliminary analyses (n=20, example in Fig.1) indicated that mean WMH volumes, estimated by the algorithms, were lower compared to reference, except for Sysu_Media. Volumetric agreement with reference segmentation was moderate‐to‐high across methods (best: Sysu_Media, ICC=0.9). Mean global DSCs were moderate (best: Sysu_Media, DSC=0.6±0.2), with better performance for larger compared to lower WMH load. Mean number of clusters, detected by the algorithms, was lower compared to reference, except for Sysu_Media that overestimated this value. Mean DSCs for clusters were also moderate for all algorithms (best: Sysu_Media, DSC=0.5±0.2). Mean precision of detected WMH load and number of clusters was higher than mean recall (sensitivity) across all methods, except for Sysu_Media.ConclusionThese preliminary findings show that performance of different automated WMH segmentations is relatively similar. Although most algorithms correctly identify WMH (higher precision), the methods underestimate total WMH load (lower sensitivity). The Sysu_Media algorithm performs best in many indices, however, it yielded lowest precision in number of clusters. Results of the final sample (n=40) will be presented, intended to serve as a method guide for conducting WMH segmentations in the multicenter DELCODE study.