Performance of five automated white matter hyperintensity segmentation methods in a multicenter dataset

Rutger Heinen,Frederik Barkhof,Geert Jan Biessels,Hugo Vrenken,Martijn D Steenwijk,Hugo J Kuijf,J Matthijs Biesbroek,Niels D Prins,Wiesje M Van Der Flier,Jeroen De Bresser

doi:10.1038/s41598-019-52966-0

Abstract

White matter hyperintensities (WMHs) are a common manifestation of cerebral small vessel disease, that is increasingly studied with large, pooled multicenter datasets. This data pooling increases statistical power, but poses challenges for automated WMH segmentation. Although there is extensive literature on the evaluation of automated WMH segmentation methods, such evaluations in a multicenter setting are lacking. We performed WMH segmentations in sixty patients scanned on six different magnetic resonance imaging (MRI) scanners (10 patients per scanner) using five freely available and fully-automated WMH segmentation methods (Cascade, kNN-TTP, Lesion-TOADS, LST-LGA and LST-LPA). Different MRI scanner vendors and field strengths were included. We compared these automated WMH segmentations with manual WMH segmentations as a reference. Performance of each method both within and across scanners was assessed using spatial and volumetric correspondence with the reference segmentations by Dice’s similarity coefficient (DSC) and intra-class correlation coefficient (ICC) respectively. We found the best performance, both within and across scanners, for kNN-TTP, followed by LST-LPA and LST-LGA, with worse performance for Lesion-TOADS and Cascade. Our findings can serve as a guide for choosing a method and also highlight the importance to further improve and evaluate consistency of methods in a multicenter setting.

Highlights

Pooling of multicenter brain magnetic resonance imaging (MRI) data is a trend in various research fields, including studies on ageing related brain diseases[1,2,3]
Achieving accurate and precise White matter hyperintensities of presumed vascular origin (WMHs) segmentations can be challenging across MRI scanners of different vendors, field strengths and scan protocols
Many different automated methods currently exist to segment WMHs

Summary

Introduction

Pooling of multicenter brain magnetic resonance imaging (MRI) data is a trend in various research fields, including studies on ageing related brain diseases[1,2,3]. A recent study provided an extensive overview of existing supervised, unsupervised and semi-supervised methods[13] Challenges for these methods include false positive (e.g. artefacts, infarcts) and false negative (often for punctate lesions) results. For WMHs of presumed vascular origin, there is a lack of studies comparing performance of these methods in multicenter, multiscanner datasets and this is an important knowledge gap[4,14]. The present study aimed to assess performance, in terms of spatial and volumetric correspondence with reference segmentations, of five automated WMH segmentation methods in a multicenter, multiscanner dataset of patients with WMHs of presumed vascular origin. We selected five methods that were fully automatic and freely available for academic research: Cascade[15,16], k-nearest neighbor classification with tissue type priors (kNN-TTP)[17], Lesion-TOpology-preserving Anatomical Segmentation (Lesion-TOADS)[11], the Lesion Segmentation Tool Lesion Prediction Algorithm (LST-LPA) and the Lesion Segmentation Tool Lesion Growth Algorithm (LST-LGA)[10]

Objectives

Methods

Results

Conclusion