Evaluation of algorithms for Multi-Modality Whole Heart Segmentation: An open-access grand challenge.

Xiahai Zhuang ,David N Firmin ,Cheng Bian ,Chenchen Sun ,David E Newby ,Aliasghar Mortazi ,Guangbin Yang ,Zenglin Shi ,Thierry Brouard ,Sébastien Ourselin ,Lei Li ,Raad H Mohiaddin ,Christian Payer ,Mattias P Heinrich ,Guodong Zeng ,Jean-Yves Ramel ,Guoyan Zheng ,Jennifer Keegan ,Tom Macgillivray ,Ulaş Bağcı ,Martin Urschler ,Örjan Smedby ,Julien Oster ,Kawal Rhode ,Xiangyun Liao ,Pheng‐Ann Heng ,Weixin Si ,Darko Štern ,Xin Yang ,Gaëtan Galisot ,Chengjia Wang ,Chunliang Wang ,Qianqian Tong ,Guang Yang

doi:10.1016/j.media.2019.101537

Abstract

Knowledge of whole heart anatomy is a prerequisite for many clinical applications. Whole heart segmentation (WHS), which delineates substructures of the heart, can be very valuable for modeling and analysis of the anatomy and functions of the heart. However, automating this segmentation can be challenging due to the large variation of the heart shape, and different image qualities of the clinical data. To achieve this goal, an initial set of training data is generally needed for constructing priors or for training. Furthermore, it is difficult to perform comparisons between different methods, largely due to differences in the datasets and evaluation metrics used. This manuscript presents the methodologies and evaluation results for the WHS algorithms selected from the submissions to the Multi-Modality Whole Heart Segmentation (MM-WHS) challenge, in conjunction with MICCAI 2017. The challenge provided 120 three-dimensional cardiac images covering the whole heart, including 60 CT and 60 MRI volumes, all acquired in clinical environments with manual delineation. Ten algorithms for CT data and eleven algorithms for MRI data, submitted from twelve groups, have been evaluated. The results showed that the performance of CT WHS was generally better than that of MRI WHS. The segmentation of the substructures for different categories of patients could present different levels of challenge due to the difference in imaging and variations of heart shapes. The deep learning (DL)-based methods demonstrated great potential, though several of them reported poor results in the blinded evaluation. Their performance could vary greatly across different network structures and training strategies. The conventional algorithms, mainly based on multi-atlas segmentation, demonstrated good performance, though the accuracy and computational efficiency could be limited. The challenge, including provision of the annotated training data and the blinded evaluation for submitted algorithms on the test data, continues as an ongoing benchmarking resource via its homepage (www.sdspeople.fudan.edu.cn/zhuangxiahai/0/mmwhs/).

Highlights

According to the World Health Organization, cardiovascular diseases (CVDs) are the leading cause of death globally (Mendis et al, 2011)
Payer et al (2017) propose a fully automatic whole heart segmentation, based on multi-label convolutional neural network (CNN) and using volumetric kernels, which consists of two separate CNNs: one to localize the heart, referred to as localization CNN, and the other to segment the fine detail of the whole heart structure within a small region of interest (ROI), referred to as segmentation CNN
Similar conclusion can be drawn for the individual substructures as well as for the whole heart, when one compares the boxplots of segmentation Dice scores between Figs. 4 and 5

Summary

Introduction

According to the World Health Organization, cardiovascular diseases (CVDs) are the leading cause of death globally (Mendis et al, 2011). Medical imaging has revolutionized modern medicine and healthcare, and imaging and computing technologies have become increasingly important for the diagnosis and treatments of CVDs. Computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), single photon emission computed tomography (SPECT), and ultrasound (US) have been used extensively for physiologic understanding and diagnostic purposes in cardiology (Kang et al, 2012). CT and MRI are used to provide clear anatomical information of the heart. Cardiac MRI has the advantages of being free from ionizing radiation, acquiring images with good contrast between soft tissues and with relatively high spatial resolution (Nikolaou et al, 2011). Cardiac CT, though involves ionizing radiation, is fast, low cost, and generally of high quality (Roberts et al, 2008)

Objectives

Methods

Results

Discussion

Conclusion