Development and evaluation of a manual segmentation protocol for deep grey matter in multiple sclerosis: Towards accelerated semi-automated references

Alexandra De Sitter ,Jessica Burggraaff,Stefan Ropele,Mike P Wattjes,Antonio Gallo,Serena Ruggieri,Deborah Pareto,Christian Enzinger,Massimo Filippi,Michael Amann,Ludwig Kappos,Claudio Gasperini,Menno M Schoonheim,Frederik Barkhof,Yaou Liu,Katharina Schregel,Charles R.G Guttmann,Hugo Vrenken,Bernard M J Uitdehaag ,Jaume Sastre‐Garriga ,Miklós Palotai ,Maria Rocca ,Marios Yiannakas ,Jorge Simões ,Fabian Bartel ,Jette Lautrup Frederiksen

doi:10.1016/j.nicl.2021.102659

Abstract

BackgroundDeep grey matter (dGM) structures, particularly the thalamus, are clinically relevant in multiple sclerosis (MS). However, segmentation of dGM in MS is challenging; labeled MS-specific reference sets are needed for objective evaluation and training of new methods. ObjectivesThis study aimed to (i) create a standardized protocol for manual delineations of dGM; (ii) evaluate the reliability of the protocol with multiple raters; and (iii) evaluate the accuracy of a fast-semi-automated segmentation approach (FASTSURF). MethodsA standardized manual segmentation protocol for caudate nucleus, putamen, and thalamus was created, and applied by three raters on multi-center 3D T1-weighted MRI scans of 23 MS patients and 12 controls. Intra- and inter-rater agreement was assessed through intra-class correlation coefficient (ICC); spatial overlap through Jaccard Index (JI) and generalized conformity index (CIgen). From sparse delineations, FASTSURF reconstructed full segmentations; accuracy was assessed both volumetrically and spatially. ResultsAll structures showed excellent agreement on expert manual outlines: intra-rater JI > 0.83; inter-rater ICC ≥ 0.76 and CIgen ≥ 0.74. FASTSURF reproduced manual references excellently, with ICC ≥ 0.97 and JI ≥ 0.92. ConclusionsThe manual dGM segmentation protocol showed excellent reproducibility within and between raters. Moreover, combined with FASTSURF a reliable reference set of dGM segmentations can be produced with lower workload.

Highlights

Patients with multiple sclerosis (MS) exhibit damage of the grey matter (GM), including focal lesions and atrophy. (Du Toit et al, 2008; Bagnato et al, 2006; Geurts et al, 2005) GM atrophy can be quantified from structural brain magnetic resonance images (MRI) and has become an important and clinically relevant imaging outcome measure of MS
Atrophy of deep GM structures such as the caudate nucleus, putamen and thalamus has become of interest in MS, as it has been shown to correlate with important clinical outcome such as cognition. (Schoonheim et al, 2015; Bishop et al, 2017; Bermel et al, 2003; Houtchens et al, 2007; Pagani et al, 2005) Atrophy measures of the Deep grey matter (dGM) may serve as potential imaging biomarkers in MS
Current state-of-the-art and frequently used automated segmentation methods suffer from substantial limitations with respect to both repro ducibility and accuracy, which is partly due to the presence of MS pathological changes. (Popescu et al, 2014, 2016; Gelineau-Morel et al, 2012; Meijerman et al, 2018; Amiri et al, 2018; de Sitter et al, 2020) there are various confounds that can affect the measure ment of dGM atrophy: image registration and segmentation can be negatively affected by the presence of white matter lesions, (GelineauMorel et al, 2012; de Sitter et al, 2020) generalized or local atrophy, or subtle tissue contrast changes (Amiri et al, 2018; Westlye et al, 2009)

Summary

Introduction

Patients with multiple sclerosis (MS) exhibit damage of the grey matter (GM), including focal lesions and atrophy. (Du Toit et al, 2008; Bagnato et al, 2006; Geurts et al, 2005) GM atrophy can be quantified from structural brain magnetic resonance images (MRI) and has become an important and clinically relevant imaging outcome measure of MS. Spe cifically, we investigated the performance of a recently developed semiautomated technique called ‘FAst Segmentation Through SURface Fairing’ (FASTSURF), (Bartel et al, 2019) which was demonstrated as a proof-of-concept for the hippocampus in Alzheimer patients by Bartel et al (2019) Since this technique exhibited excellent accuracy for hippocampus, we hypothesized that FASTSURF can be used to generate accurate reference segmentations of various other brain structures, with substantially lower workload than full manual tracings. This may provide an important impetus towards improved segmentation of dGM. Objectives: This study aimed to (i) create a standardized protocol for manual delineations of dGM; (ii) evaluate the reliability of the protocol with multiple raters; and (iii) evaluate the accuracy of a fast-semi-automated segmentation approach (FASTSURF)

Objectives

Methods

Results

Conclusion