A Lesion-Aware Patch Sampling Approach with EfficientNet3D-UNet for Robust Multiple Sclerosis Lesion Segmentation

Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patient's neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.

Magnetic Resonance Imaging (MRI) plays a significant role in the current characterization and diagnosis of multiple sclerosis (MS) in radiological imaging. However, early detection of MS lesions from MRI still remains a challenging problem. In the present work, an information theoretic approach to cluster the voxels in MS lesions for automatic segmentation of lesions of various sizes in multi-contrast (T1, T2, PD-weighted) MR images, is applied. For accurate detection of MS lesions of various sizes, the skull-stripped brain data are rescaled and histogram manipulated prior to mapping the multi-contrast data to pseudo-color images. For automated segmentation of multiple sclerosis (MS) lesions in multi-contrast MRI, the improved jump method (IJM) clustering method has been enhanced via edge suppression for improved segmentation of white matter (WM), gray matter (GM), cerebrospinal fluid (CSF) and MS lesions if present. From this preliminary clustering, a pseudo-color to grayscale conversion is designed to equalize the intensities of the normal brain tissues, leaving the MS lesions as outliers. Binary discrete and 8-bit fuzzy labels are then assigned to segment the MS lesions throughout the full brain. For validation of the proposed method, three brains, with mild, moderate and severe hyperintense MS lesions labeled as ground truth, were selected. The MS lesions of mild, moderate and severe categories were detected with a sensitivity of 80%, and 96%, and 94%, and with the corresponding Dice similarity coefficient (DSC) of 0.5175, 0.8739, and 0.8266 respectively. The MS lesions can also be clearly visualized in a transparent pseudo-color computer rendered 3D brain.

Multiple Sclerosis (MS) lesion segmentation from MR images is important for neuroimaging analysis. MS is diffuse, multifocal, and tend to involve peripheral brain structures such as the white matter, corpus callosum, and brainstem. Recently, U-Net has made great achievements in medical image segmentation area. However, the insufficiently use of context information and feature representation, makes it fail to achieve segmentation of MS lesions accurately. To solve the problem, 3D attention context U-Net (ACU-Net) is proposed for MS lesion segmentation in this paper. The proposed ACU-Net includes 3D spatial attention block, which is used to enrich spatial details and feature representation of lesion in the decoding stage. Furthermore, in the encoding and decoding stage of the network, 3D context guided module is designed for guiding local information and surrounding information. The proposed ACU-Net was evaluated on the ISBI 2015 longitudinal MS lesion segmentation challenge dataset, and it achieved superior performance compared to latest approaches.

Evaluate the performance of a deep learning (DL)-based model for multiple sclerosis (MS) lesion segmentation and compare it to other DL and non-DL algorithms. This ambispective, multicenter study assessed the performance of a DL-based model for MS lesion segmentation and compared it to alternative DL- and non-DL-based methods. Models were tested on internal (n = 20) and external (n = 18) datasets from Latin America, and on an external dataset from Europe (n = 49). We also examined robustness by rescanning six patients (n = 6) from our MS clinical cohort. Moreover, we studied inter-human annotator agreement and discussed our findings in light of these results. Performance and robustness were assessed using intraclass correlation coefficient (ICC), Dice coefficient (DC), and coefficient of variation (CV). Inter-human ICC ranged from 0.89 to 0.95, while spatial agreement among annotators showed a median DC of 0.63. Using expert manual segmentations as ground truth, our DL model achieved a median DC of 0.73 on the internal, 0.66 on the external, and 0.70 on the challenge datasets. The performance of our DL model exceeded that of the alternative algorithms on all datasets. In the robustness experiment, our DL model also achieved higher DC (ranging from 0.82 to 0.90) and lower CV (ranging from 0.7 to 7.9%) when compared to the alternative methods. Our DL-based model outperformed alternative methods for brain MS lesion segmentation. The model also proved to generalize well on unseen data and has a robust performance and low processing times both on real-world and challenge-based data. Our DL-based model demonstrated superior performance in accurately segmenting brain MS lesions compared to alternative methods, indicating its potential for clinical application with improved accuracy, robustness, and efficiency. • Automated lesion load quantification in MS patients is valuable; however, more accurate methods are still necessary. • A novel deep learning model outperformed alternative MS lesion segmentation methods on multisite datasets. • Deep learning models are particularly suitable for MS lesion segmentation in clinical scenarios.

The aim of this study was to develop a new automated segmentation method of white matter (WM) and cortical multiple sclerosis (MS) lesions visible on magnetization-prepared 2 inversion-contrast rapid gradient echo (MP2RAGE) images acquired at 7 T MRI. The proposed prototype (MSLAST [Multiple Sclerosis Lesion Analysis at Seven Tesla]) takes as input a single image contrast derived from the 7T MP2RAGE prototype sequence and is based on partial volume estimation and topological constraints. First, MSLAST performs a skull-strip of MP2RAGE images and computes tissue concentration maps for WM, gray matter (GM), and cerebrospinal fluid (CSF) using a partial volume model of tissues within each voxel. Second, MSLAST performs (1) connected-component analysis to GM and CSF concentration maps to classify small isolated components as MS lesions; (2) hole-filling in the WM concentration map to classify areas with low WM concentration surrounded by WM (ie, MS lesions); and (3) outlier rejection to the WM mask to improve the classification of small WM lesions. Third, MSLAST unifies the 3 maps obtained from 1, 2, and 3 processing steps to generate a global lesion mask. Quantitative and qualitative assessments were performed using MSLAST in 25 MS patients from 2 research centers. Overall, MSLAST detected a median of 71% of MS lesions, specifically 74% of WM and 58% of cortical lesions, when a minimum lesion size of 6 μL was considered. The median false-positive rate was 40%. When a 15 μL minimal lesions size was applied, which is the approximation of the minimal size recommended for 1.5/3 T images, the median detection rate was 80% for WM and 63% for cortical lesions, respectively, and the median false-positive rate was 33%. We observed high correlation between MSLAST and manual segmentations (Spearman rank correlation coefficient, ρ = 0.91), although MSLAST underestimated the total lesion volume (average difference of 1.1 mL), especially in patients with high lesion loads. MSLAST also showed good scan-rescan repeatability within the same session with an average absolute volume difference and F1 score of 0.38 ± 0.32 mL and 84%, respectively. We propose a new methodology to facilitate the segmentation of WM and cortical MS lesions at 7 T MRI, our approach uses a single MP2RAGE scan and may be of special interest to clinicians and researchers.

Magnetic Resonance Imaging (MRI) is a vital tool in the diagnosis and characterization of multiple sclerosis (MS). MS lesions can be imaged with relatively high contrast using either Fluid Attenuated Inversion Recovery (FLAIR) or Double Inversion Recovery (DIR). Automated segmentation and accurate tracking of MS lesions from MRI remains a challenging problem. Here, an information theoretic approach to cluster the voxels in pseudo-colorized multispectral MR data (FLAIR, DIR, T2-weighted) is utilized to automatically segment MS lesions of various sizes and noise levels. The Improved Jump Method (IJM) clustering, assisted by edge suppression, is applied to the segmentation of white matter (WM), gray matter (GM), cerebrospinal fluid (CSF) and MS lesions, if present, into a subset of slices determined to be the best MS lesion candidates via Otsu’s method. From this preliminary clustering, the modal data values for the tissues can be determined. A Euclidean distance is then used to estimate the fuzzy memberships of each brain voxel for all tissue types and their 50/50 partial volumes. From these estimates, binary discrete and fuzzy MS lesion masks are constructed. Validation is provided by using three synthetic MS lesions brains (mild, moderate and severe) with labeled ground truths. The MS lesions of mild, moderate and severe designations were detected with a sensitivity of 83.2%, and 88.5%, and 94.5%, and with the corresponding Dice similarity coefficient (DSC) of 0.7098, 0.8739, and 0.8266, respectively. The effect of MRI noise is also examined by simulated noise and the application of a bilateral filter in preprocessing.

Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging.

Human investigators instinctively segment medical images into their anatomical components, drawing upon prior knowledge of anatomy to overcome image artifacts, noise, and lack of tissue contrast. The authors describe: 1) the development and use of a brain tissue probability model for the segmentation of multiple sclerosis (MS) lesions in magnetic resonance (MR) brain images, and 2) an empirical comparison of the performance of statistical and decision tree classifiers, applied to MS lesion segmentation. Based on MR image data obtained from healthy volunteers, the model provides prior probabilities of brain tissue distribution per unit voxel in a standardized 3-D "brain space". In comparison to purely data-driven segmentation, the use of the model to guide the segmentation of MS lesions reduced the volume of false positive lesions by 50-80%

This paper explores the use of a soft ground-truth mask (“soft mask”) to train a Fully Convolutional Neural Network (FCNN) for segmentation of Multiple Sclerosis (MS) lesions. Detection and segmentation of MS lesions is a complex task largely due to the extreme unbalanced data, with very small number of lesion pixels that can be used for training. Utilizing the anatomical knowledge that the lesion surrounding pixels may also include some lesion level information, we suggest to increase the data set of the lesion class with neighboring pixel data -with a reduced confidence weight. A soft mask is constructed by morphological dilation of the binary segmentation mask provided by a given expert, where expert-marked voxels receive label 1 and voxels of the dilated region are assigned a soft label. In the methodology proposed, the FCNN is trained using the soft mask. On the ISBI 2015 challenge dataset, this is shown to provide a better precision-recall tradeoff and to achieve a higher average Dice similarity coefficient. We also show that by using this soft mask scheme we can improve the network segmentation performance when compared to a second independent expert.

An energy minimization method for MS lesion segmentation from T1-w and FLAIR images

Multiple sclerosis: Classification revisited reveals homogeneity and recapitulation

Detecting new lesions is a key aspect of the radiological follow-up of patients with Multiple Sclerosis (MS), leading to eventual changes in their therapeutics. This paper presents our contribution to the MSSEG-2 MICCAI 2021 challenge. The challenge is focused on the segmentation of new MS lesions using two consecutive Fluid Attenuated Inversion Recovery (FLAIR) Magnetic Resonance Imaging (MRI). In other words, considering longitudinal data composed of two time points as input, the aim is to segment the lesional areas, which are present only in the follow-up scan and not in the baseline. The backbone of our segmentation method is a 3D UNet applied patch-wise to the images, and in which, to take into account both time points, we simply concatenate the baseline and follow-up images along the channel axis before passing them to the 3D UNet. Our key methodological contribution is the use of online hard example mining to address the challenge of class imbalance. Indeed, there are very few voxels belonging to new lesions which makes training deep-learning models difficult. Instead of using handcrafted priors like brain masks or multi-stage methods, we experiment with a novel modification to online hard example mining (OHEM), where we use an exponential moving average (i.e., its weights are updated with momentum) of the 3D UNet to mine hard examples. Using a moving average instead of the raw model should allow smoothing of its predictions and allow it to give more consistent feedback for OHEM.

Mitigating catastrophic forgetting in Multiple sclerosis lesion segmentation using elastic weight consolidation

Segmentation of multiple sclerosis lesions in brain MRI: A review of automated approaches

Automatic segmentation of Multiple Sclerosis (MS) lesions from Magnetic Resonance Imaging (MRI) images is essential for clinical assessment and treatment planning of MS. Recent years have seen an increasing use of Convolutional Neural Networks (CNNs) for this task. Although these methods provide accurate segmentation, their applicability in clinical settings remains limited due to a reproducibility issue across different image domains. MS images can have highly variable characteristics across patients, MRI scanners and imaging protocols; retraining a supervised model with data from each new domain is not a feasible solution because it requires manual annotation from expert radiologists. In this work, we explore an unsupervised solution to the problem of domain shift. We present a framework, Seg-JDOT, which adapts a deep model so that samples from a source domain and samples from a target domain sharing similar representations will be similarly segmented. We evaluated the framework on a multi-site dataset, MICCAI 2016, and showed that the adaptation toward a target site can bring remarkable improvements in a model performance over standard training.