Multiple Sclerosis Lesion Segmentation Research Articles

F.sh: A 3D Recurrent Residual Attention U-Net for Automated Multiple Sclerosis Lesion Segmentation

Multiple sclerosis (MS) is an autoimmune disease affecting the central nervous system, characterized by lesions in the brain and spinal cord. Accurate detection and localization of these lesions on MRI scans is crucial for diagnosis and monitoring disease progression. Manual segmentation is time-consuming and prone to inter-rater variability. This study proposes F.sh (3DR2AUNet), a novel deep learning architecture for automated MS lesion segmentation. F.sh combines 3D recurrent residual blocks, attention gates, and the U-Net structure to effectively capture lesion features. The model was trained and evaluated using a comprehensive approach, including patch-based preprocessing, data augmentation, and a composite loss function combining Binary Cross-Entropy and 3D Dice Loss. Experimental results demonstrate the superior performance of F.sh compared to baseline methods, achieving a Dice score of 0.92. The proposed approach has the potential to assist radiologists in the accurate and efficient assessment of MS lesion burden.

Consensus of algorithms for lesion segmentation in brain MRI studies of multiple sclerosis

Segmentation of multiple sclerosis (MS) lesions on brain MRI scans is crucial for diagnosis, disease and treatment monitoring but is a time-consuming task. Despite several automated algorithms have been proposed, there is still no consensus on the most effective method. Here, we applied a consensus-based framework to improve lesion segmentation on T1-weighted and FLAIR scans. The framework is designed to combine publicly available state-of-the-art deep learning models, by running multiple segmentation tasks before merging the outputs of each algorithm. To assess the effectiveness of the approach, we applied it to MRI datasets from two different centers, including a private and a public dataset, with 131 and 30 MS patients respectively, with manually segmented lesion masks available. No further training was performed for any of the included algorithms. Overlap and detection scores were improved, with Dice increasing by 4-8% and precision by 3-4% respectively for the private and public dataset. High agreement was obtained between estimated and true lesion load (ρ = 0.92 and ρ = 0.97) and count (ρ = 0.83 and ρ = 0.94). Overall, this framework ensures accurate and reliable results, exploiting complementary features and overcoming some of the limitations of individual algorithms.

Training Artificial Neural Networks to Detect Multiple Sclerosis Lesions Using Granulometric Data from Preprocessed Magnetic Resonance Images with Morphological Transformations

The symptoms of multiple sclerosis (MS) are determined by the location of demyelinating lesions in the white matter of the brain and spinal cord. Currently, magnetic resonance imaging (MRI) is the most common tool used for diagnosing MS, understanding the course of the disease, and analyzing the effects of treatments. However, undesirable components may appear during the generation of MRI scans, such as noise or intensity variations. Mathematical morphology (MM) is a powerful image analysis technique that helps to filter the image and extract relevant structures. Granulometry is an image measurement tool for measuring MM that determines the size distribution of objects in an image without explicitly segmenting each object. While several methods have been proposed for the automatic segmentation of MS lesions in MRI scans, in some cases, only simple data preprocessing, such as image resizing to standardize the input dimensions, has been performed before the algorithm training. Therefore, this paper proposes an MRI preprocessing algorithm capable of performing elementary morphological transformations in brain images of MS patients and healthy individuals in order to delete undesirable components and extract the relevant structures such as MS lesions. Also, the algorithm computes the granulometry in MRI scans to describe the size qualities of lesions. Using this algorithm, we trained two artificial neural networks (ANNs) to predict MS diagnoses. By computing the differences in granulometry measurements between an image with MS lesions and a reference image (without lesions), we determined the size characterization of the lesions. Then, the ANNs were evaluated with the validation set, and the performance results (test accuracy = 0.9753; cross-entropy loss = 0.0247) show that the proposed algorithm can support specialists in making decisions to diagnose MS and estimating the disease progress based on granulometry values.

Contrast-Enhancing Lesion Segmentation in Multiple Sclerosis: A Deep Learning Approach Validated in a Multicentric Cohort.

The detection of contrast-enhancing lesions (CELs) is fundamental for the diagnosis and monitoring of patients with multiple sclerosis (MS). This task is time-consuming and suffers from high intra- and inter-rater variability in clinical practice. However, only a few studies proposed automatic approaches for CEL detection. This study aimed to develop a deep learning model that automatically detects and segments CELs in clinical Magnetic Resonance Imaging (MRI) scans. A 3D UNet-based network was trained with clinical MRI from the Swiss Multiple Sclerosis Cohort. The dataset comprised 372 scans from 280 MS patients: 162 showed at least one CEL, while 118 showed no CELs. The input dataset consisted of T1-weighted before and after gadolinium injection, and FLuid Attenuated Inversion Recovery images. The sampling strategy was based on a white matter lesion mask to confirm the existence of real contrast-enhancing lesions. To overcome the dataset imbalance, a weighted loss function was implemented. The Dice Score Coefficient and True Positive and False Positive Rates were 0.76, 0.93, and 0.02, respectively. Based on these results, the model developed in this study might well be considered for clinical decision support.

A joint model for lesion segmentation and classification of MS and NMOSD.

Multiple sclerosis (MS) and neuromyelitis optic spectrum disorder (NMOSD) are mimic autoimmune diseases of the central nervous system with a very high disability rate. Their clinical symptoms and imaging findings are similar, making it difficult to diagnose and differentiate. Existing research typically employs the T2-weighted fluid-attenuated inversion recovery (T2-FLAIR) MRI imaging technique to focus on a single task in MS and NMOSD lesion segmentation or disease classification, while ignoring the collaboration between the tasks. To make full use of the correlation between lesion segmentation and disease classification tasks of MS and NMOSD, so as to improve the accuracy and speed of the recognition and diagnosis of MS and NMOSD, a joint model is proposed in this study. The joint model primarily comprises three components: an information-sharing subnetwork, a lesion segmentation subnetwork, and a disease classification subnetwork. Among them, the information-sharing subnetwork adopts a dualbranch structure composed of a convolution module and a Swin Transformer module to extract local and global features, respectively. These features are then input into the lesion segmentation subnetwork and disease classification subnetwork to obtain results for both tasks simultaneously. In addition, to further enhance the mutual guidance between the tasks, this study proposes two information interaction methods: a lesion guidance module and a crosstask loss function. Furthermore, the lesion location maps provide interpretability for the diagnosis process of the deep learning model. The joint model achieved a Dice similarity coefficient (DSC) of 74.87% on the lesion segmentation task and accuracy (ACC) of 92.36% on the disease classification task, demonstrating its superior performance. By setting up ablation experiments, the effectiveness of information sharing and interaction between tasks is verified. The results show that the joint model can effectively improve the performance of the two tasks.

A Hybrid Method for Multiple Sclerosis Lesion Segmentation Using Wavelet and Dense U-Net

Multiple Sclerosis (MS) is one of the debilitating disorders of the central nervous system. This disease causes lesions in the white matter of the brain tissue. It can also lead to many physical and psychological disorders in movement, vision, and memory. Lesion segmentation in MRI images to determine the number and size of lesions is one of the diagnostic problems for specialists. Using automated diagnostic tools as an aid can help professionals. Traditional image processing and deep learning methods are used to automate lesion segmentation. The U-Net is one of the most widely used deep learning architectures for MS lesion segmentation. The images are used in the Fourier domain in the U-Net network, which does not include all its features. Our proposed method combines the HAR wavelet transform and the Dense net-based U-Net. This makes local features and lesions of different sizes more prominent and leads to higher quality segmentation. The proposed method had a better Dice value than the compared methods in the experiments.

Simulation of acquisition shifts in T2 weighted fluid-attenuated inversion recovery magnetic resonance images to stress test artificial intelligence segmentation networks.

To provide a simulation framework for routine neuroimaging test data, which allows for "stress testing" of deep segmentation networks against acquisition shifts that commonly occur in clinical practice for T2 weighted (T2w) fluid-attenuated inversion recovery magnetic resonance imaging protocols. The approach simulates "acquisition shift derivatives" of MR images based on MR signal equations. Experiments comprise the validation of the simulated images by real MR scans and example stress tests on state-of-the-art multiple sclerosis lesion segmentation networks to explore a generic model function to describe the F1 score in dependence of the contrast-affecting sequence parameters echo time (TE) and inversion time (TI). The differences between real and simulated images range up to 19% in gray and white matter for extreme parameter settings. For the segmentation networks under test, the F1 score dependency on TE and TI can be well described by quadratic model functions (). The coefficients of the model functions indicate that changes of TE have more influence on the model performance than TI. We show that these deviations are in the range of values as may be caused by erroneous or individual differences in relaxation times as described by literature. The coefficients of the F1 model function allow for a quantitative comparison of the influences of TE and TI. Limitations arise mainly from tissues with a low baseline signal (like cerebrospinal fluid) and when the protocol contains contrast-affecting measures that cannot be modeled due to missing information in the DICOM header.

Improving multiple sclerosis lesion segmentation across clinical sites: A federated learning approach with noise-resilient training

Accurately measuring the evolution of Multiple Sclerosis (MS) with magnetic resonance imaging (MRI) critically informs understanding of disease progression and helps to direct therapeutic strategy. Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area. Obtaining sufficient data from a single clinical site is challenging and does not address the heterogeneous need for model robustness. Conversely, the collection of data from multiple sites introduces data privacy concerns and potential label noise due to varying annotation standards. To address this dilemma, we explore the use of the federated learning framework while considering label noise. Our approach enables collaboration among multiple clinical sites without compromising data privacy under a federated learning paradigm that incorporates a noise-robust training strategy based on label correction. Specifically, we introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions, enabling the correction of false annotations based on prediction confidence. We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites, enhancing the reliability of the correction process. Extensive experiments conducted on two multi-site datasets demonstrate the effectiveness and robustness of our proposed methods, indicating their potential for clinical applications in multi-site collaborations to train better deep learning models with lower cost in data collection and annotation.

Diagnostic effectiveness of deep learning-based MRI in predicting multiple sclerosis: A meta-analysis.

The brain and spinal cord, constituting the central nervous system (CNS), could be impacted by an inflammatory disease known as multiple sclerosis (MS). The convolutional neural networks (CNN), a machine learning method, can detect lesions early by learning patterns on brain magnetic resonance image (MRI). We performed this study to investigate the diagnostic performance of CNN based MRI in the identification, classification, and segmentation of MS lesions. PubMed, Web of Science, Embase, the Cochrane Library, CINAHL, and Google Scholar were used to retrieve papers reporting the use of CNN based MRI in MS diagnosis. The accuracy, the specificity, the sensitivity, and the Dice Similarity Coefficient (DSC) were evaluated in this study. In total, 2174 studies were identified and only 15 articles met the inclusion criteria. The 2D-3D CNN presented a high accuracy (98.81, 95% CI: 98.50-99.13), sensitivity (98.76, 95% CI: 98.42-99.10), and specificity (98.67, 95% CI: 98.22-99.12) in the identification of MS lesions. Regarding classification, the overall accuracy rate was significantly high (91.38, 95% CI: 83.23-99.54). A DSC rate of 63.78 (95% CI: 58.29-69.27) showed that 2D-3D CNN-based MRI performed highly in the segmentation of MS lesions. Sensitivity analysis showed that the results are consistent, indicating that this study is robust. This metanalysis revealed that 2D-3D CNN based MRI is an automated system that has high diagnostic performance and can promptly and effectively predict the disease.

An Efficient Multiple Sclerosis Segmentation Framework using Hybrid Dilated Convolution –based Adaptive Mobilenet Mechanism.

Multiple Sclerosis (MS) is considered a very popular neurological condition in adults. Poor walking stability is considered the primary sign of MS. The Magnetic Resonance Imaging (MRI) is ensured to be a delicate approach for discovering disease progression. Severity quantification of the disease via MS lesion volume validation with MR imaging is significant for monitoring and learning the disorder and its treatment. Numerous techniques for MS lesions segmentation demand experts seed points as input, yet do not sufficiently permit the specialists to perform effectively or intuitively. Interestingly, various approaches also consider that the points are optimally determined. Hence, it is necessary to rectify some of the troubles that existed in classical MS classification mechanisms by developing a powerful process with deep learning in this work. In the beginning, the requisite MS images are fetched from the existing online source and forwarded to the module of image segmentation. In this module, an efficient image segmentation operation is carried out employing Transformer Unet++ with MobileNetv3 (TUnet++-MNetv3). Further, the segmented images are given to the module named MS classification, where the Hybrid Dilated Convolution based Adaptive MobileNet (HDC-AMNet) is recommended to classify the MS. Moreover, the Enhanced Single Candidate Optimizer (ESCO) is adopted to optimize the parameters that exist in the MobileNet. Thus, the designed MS classification task achieves improved performance rates by contrasting it with its baseline techniques employing numerous metrics.

LST-AI: A deep learning ensemble for accurate MS lesion segmentation

Automated segmentation of brain white matter lesions is crucial for both clinical assessment and scientific research in multiple sclerosis (MS). Over a decade ago, we introduced an engineered lesion segmentation tool, LST. While recent lesion segmentation approaches have leveraged artificial intelligence (AI), they often remain proprietary and difficult to adopt. As an open-source tool, we present LST-AI, an advanced deep learning-based extension of LST that consists of an ensemble of three 3D U-Nets.LST-AI explicitly addresses the imbalance between white matter (WM) lesions and non-lesioned WM. It employs a composite loss function incorporating binary cross-entropy and Tversky loss to improve segmentation of the highly heterogeneous MS lesions. We train the network ensemble on 491 MS pairs of T1-weighted and FLAIR images, collected in-house from a 3T MRI scanner, and expert neuroradiologists manually segmented the utilized lesion maps for training. LST-AI also includes a lesion location annotation tool, labeling lesions as periventricular, infratentorial, and juxtacortical according to the 2017 McDonald criteria, and, additionally, as subcortical. We conduct evaluations on 103 test cases consisting of publicly available data using the Anima segmentation validation tools and compare LST-AI with several publicly available lesion segmentation models.Our empirical analysis shows that LST-AI achieves superior performance compared to existing methods. Its Dice and F1 scores exceeded 0.62, outperforming LST, SAMSEG (Sequence Adaptive Multimodal SEGmentation), and the popular nnUNet framework, which all scored below 0.56. Notably, LST-AI demonstrated exceptional performance on the MSSEG-1 challenge dataset, an international WM lesion segmentation challenge, with a Dice score of 0.65 and an F1 score of 0.63—surpassing all other competing models at the time of the challenge. With increasing lesion volume, the lesion detection rate rapidly increased with a detection rate of >75% for lesions with a volume between 10 mm3 and 100 mm3. Given its higher segmentation performance, we recommend that research groups currently using LST transition to LST-AI. To facilitate broad adoption, we are releasing LST-AI as an open-source model, available as a command-line tool, dockerized container, or Python script, enabling diverse applications across multiple platforms.

Retracted: Using Convolutional Neural Networks for Segmentation of Multiple Sclerosis Lesions in 3D Magnetic Resonance Imaging

Retracted: Using Convolutional Neural Networks for Segmentation of Multiple Sclerosis Lesions in 3D Magnetic Resonance Imaging

A Weakly Supervised Gradient Attribution Constraint for Interpretable Classification and Anomaly Detection.

The lack of interpretability of deep learning reduces understanding of what happens when a network does not work as expected and hinders its use in critical fields like medicine, which require transparency of decisions. For example, a healthy vs pathological classification model should rely on radiological signs and not on some training dataset biases. Several post-hoc models have been proposed to explain the decision of a trained network. However, they are very seldom used to enforce interpretability during training and none in accordance with the classification. In this paper, we propose a new weakly supervised method for both interpretable healthy vs pathological classification and anomaly detection. A new loss function is added to a standard classification model to constrain each voxel of healthy images to drive the network decision towards the healthy class according to gradient-based attributions. This constraint reveals pathological structures for patient images, allowing their unsupervised segmentation. Moreover, we advocate both theoretically and experimentally, that constrained training with the simple Gradient attribution is similar to constraints with the heavier Expected Gradient, consequently reducing the computational cost. We also propose a combination of attributions during the constrained training making the model robust to the attribution choice at inference. Our proposition was evaluated on two brain pathologies: tumors and multiple sclerosis. This new constraint provides a more relevant classification, with a more pathology-driven decision. For anomaly detection, the proposed method outperforms state-of-the-art especially on difficult multiple sclerosis lesions segmentation task with a 15 points Dice improvement.

BIANCA-MS: An optimized tool for automated multiple sclerosis lesion segmentation.

In this work we present BIANCA-MS, a novel tool for brain white matter lesion segmentation in multiple sclerosis (MS), able to generalize across both the wide spectrum of MRI acquisition protocols and the heterogeneity of manually labeled data. BIANCA-MS is based on the original version of BIANCA and implements two innovative elements: a harmonized setting, tested under different MRI protocols, which avoids the need to further tune algorithm parameters to each dataset; and a cleaning step developed to improve consistency in automated and manual segmentations, thus reducing unwanted variability in output segmentations and validation data. BIANCA-MS was tested on three datasets, acquired with different MRI protocols. First, we compared BIANCA-MS to other widely used tools. Second, we tested how BIANCA-MS performs in separate datasets. Finally, we evaluated BIANCA-MS performance on a pooled dataset where all MRI data were merged. We calculated the overlap using the DICE spatial similarity index (SI) as well as the number of false positive/negative clusters (nFPC/nFNC) in comparison to the manual masks processed with the cleaning step. BIANCA-MS clearly outperformed other available tools in both high- and low-resolution images and provided comparable performance across different scanning protocols, sets of modalities and image resolutions. BIANCA-MS performance on the pooled dataset (SI: 0.72 ± 0.25, nFPC: 13 ± 11, nFNC: 4 ± 8) were comparable to those achieved on each individual dataset (median across datasets SI: 0.72 ± 0.28, nFPC: 14 ± 11, nFNC: 4 ± 8). Our findings suggest that BIANCA-MS is a robust and accurate approach for automated MS lesion segmentation.

Multiple sclerosis lesion segmentation: revisiting weighting mechanisms for federated learning.

Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's applications remain suboptimal in neuroimage analysis tasks such as lesion segmentation in multiple sclerosis (MS), due to variance in lesion characteristics imparted by different scanners and acquisition parameters. In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation performance. In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training. The proposed method has been validated on two FL MS segmentation scenarios using public and clinical datasets. Specifically, the case-wise and voxel-wise Dice score of the proposed method under the first public dataset is 65.20 and 74.30, respectively. On the second in-house dataset, the case-wise and voxel-wise Dice score is 53.66, and 62.31, respectively. The Comparison experiments on two FL MS segmentation scenarios using public and clinical datasets have demonstrated the effectiveness of the proposed method by significantly outperforming other FL methods. Furthermore, the segmentation performance of FL incorporating our proposed aggregation mechanism can achieve comparable performance to that from centralized training with all the raw data.

Boosting multiple sclerosis lesion segmentation through attention mechanism

Magnetic resonance imaging is a fundamental tool to reach a diagnosis of multiple sclerosis and monitoring its progression. Although several attempts have been made to segment multiple sclerosis lesions using artificial intelligence, fully automated analysis is not yet available. State-of-the-art methods rely on slight variations in segmentation architectures (e.g. U-Net, etc.). However, recent research has demonstrated how exploiting temporal-aware features and attention mechanisms can provide a significant boost to traditional architectures. This paper proposes a framework that exploits an augmented U-Net architecture with a convolutional long short-term memory layer and attention mechanism which is able to segment and quantify multiple sclerosis lesions detected in magnetic resonance images. Quantitative and qualitative evaluation on challenging examples demonstrated how the method outperforms previous state-of-the-art approaches, reporting an overall Dice score of 89% and also demonstrating robustness and generalization ability on never seen new test samples of a new dedicated under construction dataset.

Edges-enhanced Convolutional Neural Network for Multiple Sclerosis Lesions Segmentation

Edges-enhanced Convolutional Neural Network for Multiple Sclerosis Lesions Segmentation

Multi-arm U-Net with dense input and skip connectivity for T2 lesion segmentation in clinical trials of multiple sclerosis

T2 lesion quantification plays a crucial role in monitoring disease progression and evaluating treatment response in multiple sclerosis (MS). We developed a 3D, multi-arm U-Net for T2 lesion segmentation, which was trained on a large, multicenter clinical trial dataset of relapsing MS. We investigated its generalization to other relapsing and primary progressive MS clinical trial datasets, and to an external dataset from the MICCAI 2016 MS lesion segmentation challenge. Additionally, we assessed the model’s ability to reproduce the separation of T2 lesion volumes between treatment and control arms; and the association of baseline T2 lesion volumes with clinical disability scores compared with manual lesion annotations. The trained model achieved a mean dice coefficient of ≥ 0.66 and a lesion detection sensitivity of ≥ 0.72 across the internal test datasets. On the external test dataset, the model achieved a mean dice coefficient of 0.62, which is comparable to 0.59 from the best model in the challenge, and a lesion detection sensitivity of 0.68. Lesion detection performance was reduced for smaller lesions (≤ 30 μL, 3–10 voxels). The model successfully maintained the separation of the longitudinal changes in T2 lesion volumes between the treatment and control arms. Such tools could facilitate semi-automated MS lesion quantification; and reduce rater burden in clinical trials.

A dense residual U-net for multiple sclerosis lesions segmentation from multi-sequence 3D MR images

Multiple Sclerosis (MS) is an autoimmune disease that causes brain and spinal cord lesions, which magnetic resonance imaging (MRI) can detect and characterize. Recently, deep learning methods have achieved remarkable results in the automated segmentation of MS lesions from MRI data. Hence, this study proposes a novel dense residual U-Net model that combines attention gate (AG), efficient channel attention (ECA), and Atrous Spatial Pyramid Pooling (ASPP) to enhance the performance of the automatic MS lesion segmentation using 3D MRI sequences. First, convolution layers in each block of the U-Net architecture are replaced by residual blocks and connected densely. Then, AGs are exploited to capture salient features passed through the skip connections. The ECA module is appended at the end of each residual block and each downsampling block of U-Net. Later, the bottleneck of U-Net is replaced with the ASSP module to extract multi-scale contextual information. Furthermore, 3D MR images of Fluid Attenuated Inversion Recovery (FLAIR), T1-weighted (T1-w), and T2-weighted (T2-w) are exploited jointly to perform better MS lesion segmentation. The proposed model is validated on the publicly available ISBI2015 and MSSEG2016 challenge datasets. This model produced an ISBI score of 92.75, a mean Dice score of 66.88%, a mean positive predictive value (PPV) of 86.50%, and a mean lesion-wise true positive rate (LTPR) of 60.64% on the ISBI2015 testing set. Also, it achieved a mean Dice score of 67.27%, a mean PPV of 65.19%, and a mean sensitivity of 74.40% on the MSSEG2016 testing set. The results show that the proposed model performs better than the results of some experts and some of the other state-of-the-art methods realized related to this particular subject. Specifically, the best Dice score and the best LTPR are obtained on the ISBI2015 testing set by using the proposed model to segment MS lesions.

Online hard example mining vs. fixed oversampling strategy for segmentation of new multiple sclerosis lesions from longitudinal FLAIR MRI

Detecting new lesions is a key aspect of the radiological follow-up of patients with Multiple Sclerosis (MS), leading to eventual changes in their therapeutics. This paper presents our contribution to the MSSEG-2 MICCAI 2021 challenge. The challenge is focused on the segmentation of new MS lesions using two consecutive Fluid Attenuated Inversion Recovery (FLAIR) Magnetic Resonance Imaging (MRI). In other words, considering longitudinal data composed of two time points as input, the aim is to segment the lesional areas, which are present only in the follow-up scan and not in the baseline. The backbone of our segmentation method is a 3D UNet applied patch-wise to the images, and in which, to take into account both time points, we simply concatenate the baseline and follow-up images along the channel axis before passing them to the 3D UNet. Our key methodological contribution is the use of online hard example mining to address the challenge of class imbalance. Indeed, there are very few voxels belonging to new lesions which makes training deep-learning models difficult. Instead of using handcrafted priors like brain masks or multi-stage methods, we experiment with a novel modification to online hard example mining (OHEM), where we use an exponential moving average (i.e., its weights are updated with momentum) of the 3D UNet to mine hard examples. Using a moving average instead of the raw model should allow smoothing of its predictions and allow it to give more consistent feedback for OHEM.