ISLES 2016 and 2017-Benchmarking Ischemic Stroke Lesion Outcome Prediction Based on Multispectral MRI.

Stefan Winzeck,Victor Alves,Arsany Hakim,James M Brown,David Robben,Beom Joon Kim,Miguel Monteiro,Egor Krivov,Joong-Ho Won,Roland Wiest,Myunghee Cho Paik,Mobarakol Islam,Christian Lucas,José A A D S R Pinto Pinto,Yongchan Kwon,Mikhail Belyaev,Maxim Pisov,Enhao Gong,Oskar Maier,Junshen Xu,Ken Chang,Hongliang Ren,John M Pauly,Mattias P Heinrich,Pablo Arbelaezs,Paul Suetens,Hanbyul Lee,Richard Mckinley,Mauricio Reyes,Youngwon Choi,Greg Zaharchuk,Andrew Beers ,Luis Carlos Rivera ,Laura Daza ,Arlindo L Oliveira ,Jayashree Kalpathy–Cramer ,Laura Silvana Castillo ,Yanling Niu ,Carlos A Silva

doi:10.3389/fneur.2018.00679

Abstract

Performance of models highly depend not only on the used algorithm but also the data set it was applied to. This makes the comparison of newly developed tools to previously published approaches difficult. Either researchers need to implement others' algorithms first, to establish an adequate benchmark on their data, or a direct comparison of new and old techniques is infeasible. The Ischemic Stroke Lesion Segmentation (ISLES) challenge, which has ran now consecutively for 3 years, aims to address this problem of comparability. ISLES 2016 and 2017 focused on lesion outcome prediction after ischemic stroke: By providing a uniformly pre-processed data set, researchers from all over the world could apply their algorithm directly. A total of nine teams participated in ISLES 2015, and 15 teams participated in ISLES 2016. Their performance was evaluated in a fair and transparent way to identify the state-of-the-art among all submissions. Top ranked teams almost always employed deep learning tools, which were predominately convolutional neural networks (CNNs). Despite the great efforts, lesion outcome prediction persists challenging. The annotated data set remains publicly available and new approaches can be compared directly via the online evaluation system, serving as a continuing benchmark (www.isles-challenge.org).

Highlights

Defining the location and extent of a stroke lesion is an essential step toward acute stroke assessment
In order to rank participant’s submission for ISLES 2017, we focused on Dice score, as it combines both precision and sensitivity into one metric, and the Hausdorff distance (HD) metric
Outcome Prediction Methods In ISLES 2016, results showed that deep learning models outperformed Random Classification Forests (RF)

Summary

Introduction

Defining the location and extent of a stroke lesion is an essential step toward acute stroke assessment. There is a great need for advanced data analysis techniques that identify these regions and predict tissue outcome in a more reproducible and accurate way. Such tools will be available to support clinicians in their decisionmaking process (e.g., deciding for or against thrombolytic therapy). In the following years the discussions happening among interdisciplinary teams at the ISLES challenge, allowed the community to advance toward the challenge of stroke lesion prediction from MRI data This is of great interest in a clinical routine, as the responsible physician needs to decide quickly, whether the particular stroke patient could benefit from an interventional treatment (i.e., thrombectomy or thrombolysis). Objective methods that reliably predict lesions and clinical outcome only from the acute scans would be a powerful tool to support and accelerate decision making during the critical phase

Objectives

Methods

Results

Discussion

Conclusion