Multimodal Multimodal Magnetic Resonance Research Articles

Glioma segmentation based on dense contrastive learning and multimodal features recalibration

Accurate segmentation of different regions of gliomas from multimodal magnetic resonance (MR) images is crucial for glioma grading and precise diagnosis, but many existing segmentation methods are difficult to effectively utilize multimodal MR image information to recognize accurately the lesion regions with small size, low contrast and irregular shape. To address this issue, this work proposes a novel 3D glioma segmentation model DCL-MANet. DCL-MANet has an architecture of multiple encoders and one single decoder. Each encoder is used to extract MR image features of a given modality. To overcome the entangle problems of multimodal semantic features, a dense contrastive learning (DCL) strategy is presented to extract the modality-specific and common features. Following that, feature recalibration block (RFB) based on modality-wise attention is used to recalibrate the semantic features of each modality, enabling the model to focus on the features that are beneficial for glioma segmentation. These recalibrated features are input into the decoder to obtain the segmentation results. To verify the superiority of the proposed method, we compare it with several state-of-the-art (SOTA) methods in terms of Dice, average symmetric surface distance (ASSD), HD95 and volumetric similarity (Vs). The comparison results show that the average Dice, ASSD, HD95 and Vs of DCL-MANet on all tumor regions are improved at least by 0.66%, 3.47%, 8.94% and 1.07% respectively. For small enhance tumor (ET) region, the corresponding improvement can be up to 0.37%, 7.83%, 11.32%, and 1.35%, respectively. In addition, the ablation results demonstrate the effectiveness of the proposed DCL and RFB, and combining them can significantly increase Dice (1.59%) and Vs (1.54%) while decreasing ASSD (40.51%) and HD95 (45.16%) on ET region. The proposed DCL-MANet could disentangle multimodal features and enhance the semantics of modality-dependent features, providing a potential means to accurately segment small lesion regions in gliomas.

NaMa: Neighbor-Aware Multi-Modal Adaptive Learning for Prostate Tumor Segmentation on Anisotropic MR Images

Accurate segmentation of prostate tumors from multi-modal magnetic resonance (MR) images is crucial for diagnosis and treatment of prostate cancer. However, the robustness of existing segmentation methods is limited, mainly because these methods 1) fail to adaptively assess subject-specific information of each MR modality for accurate tumor delineation, and 2) lack effective utilization of inter-slice information across thick slices in MR images to segment tumor as a whole 3D volume. In this work, we propose a two-stage neighbor-aware multi-modal adaptive learning network (NaMa) for accurate prostate tumor segmentation from multi-modal anisotropic MR images. In particular, in the first stage, we apply subject-specific multi-modal fusion in each slice by developing a novel modality-informativeness adaptive learning (MIAL) module for selecting and adaptively fusing informative representation of each modality based on inter-modality correlations. In the second stage, we exploit inter-slice feature correlations to derive volumetric tumor segmentation. Specifically, we first use a Unet variant with sequence layers to coarsely capture slice relationship at a global scale, and further generate an activation map for each slice. Then, we introduce an activation mapping guidance (AMG) module to refine slice-wise representation (via information from adjacent slices) for consistent tumor segmentation across neighboring slices. Besides, during the network training, we further apply a random mask strategy to each MR modality to improve feature representation efficiency. Experiments on both in-house and public (PICAI) multi-modal prostate tumor datasets show that our proposed NaMa performs better than state-of-the-art methods.

Multimodal Transformer for Accelerated MR Imaging.

Accelerated multi-modal magnetic resonance (MR) imaging is a new and effective solution for fast MR imaging, providing superior performance in restoring the target modality from its undersampled counterpart with guidance from an auxiliary modality. However, existing works simply combine the auxiliary modality as prior information, lacking in-depth investigations on the potential mechanisms for fusing different modalities. Further, they usually rely on the convolutional neural networks (CNNs), which is limited by the intrinsic locality in capturing the long-distance dependency. To this end, we propose a multi-modal transformer (MTrans), which is capable of transferring multi-scale features from the target modality to the auxiliary modality, for accelerated MR imaging. To capture deep multi-modal information, our MTrans utilizes an improved multi-head attention mechanism, named cross attention module, which absorbs features from the auxiliary modality that contribute to the target modality. Our framework provides three appealing benefits: (i) Our MTrans use an improved transformers for multi-modal MR imaging, affording more global information compared with existing CNN-based methods. (ii) A new cross attention module is proposed to exploit the useful information in each modality at different scales. The small patch in the target modality aims to keep more fine details, the large patch in the auxiliary modality aims to obtain high-level context features from the larger region and supplement the target modality effectively. (iii) We evaluate MTrans with various accelerated multi-modal MR imaging tasks, e.g., MR image reconstruction and super-resolution, where MTrans outperforms state-of-the-art methods on fastMRI and real-world clinical datasets.

Deep wavelet scattering orthogonal fusion network for glioma IDH mutation status prediction

Accurately predicting the isocitrate dehydrogenase (IDH) mutation status of gliomas is greatly significant for formulating appropriate treatment plans and evaluating the prognoses of gliomas. Although existing studies can accurately predict the IDH mutation status of gliomas based on multimodal magnetic resonance (MR) images and machine learning methods, most of these methods cannot fully explore multimodal information and effectively predict IDH status for datasets acquired from multiple centers. To address this issue, a novel wavelet scattering (WS)-based orthogonal fusion network (WSOFNet) was proposed in this work to predict the IDH mutation status of gliomas from multiple centers. First, transformation-invariant features were extracted from multimodal MR images with a WS network, and then the multimodal WS features were used instead of the original images as the inputs of WSOFNet and were fully fused through an adaptive multimodal feature fusion module (AMF2M) and an orthogonal projection module (OPM). Finally, the fused features were input into a fully connected classifier to predict IDH mutation status. In addition, to achieve improved prediction accuracy, four auxiliary losses were also used in the feature extraction modules. The comparison results showed that the prediction area under the curve (AUC) of WSOFNet on a single-center dataset was 0.9966 and that on a multicenter dataset was approximately 0.9655, which was at least 3.9% higher than that of state-of-the-art methods. Moreover, the ablation experimental results also proved that the adaptive multimodal feature fusion strategy based on orthogonal projection could effectively improve the prediction performance of the model, especially for an external validation dataset.

MLP-Res-Unet：MLPs and residual blocks-based U-shaped network intervertebral disc segmentation of multi-modal MR spine images.

Intervertebral disc degeneration (IVD) is now the most prevalent disease in the world; thus, precise intervertebral disc segmentation is essential for the assessment and diagnosis of spinal diseases. Multi-modal magnetic resonance (MR) imaging is more multi-dimensional and thorough than unimodal imaging. However, manual segmentation of multi-modal MRI not only imposes a huge burden on physicians but also has a high error rate. In this study, we propose a new method that can efficiently and accurately segment intervertebral discs from multi-modal MR spine images, providing a reproducible usage scheme for the diagnosis of spinal disorders. We suggest a network structure called MLP-Res-Unet that reduces the amount of computational load and the number of parameters while maintaining performance. Our contribution is two-fold. First, a medical image segmentation network that fuses residual blocks and a multilayer perceptron (MLP) is proposed. Secondly, we design a new deep supervised method and pass the features extracted from the encoder to the decoder through the residual path to achieve a new full-scale residual connection. We evaluate the network on the MICCAI-2018 IVD dataset and obtain Dice similarity coefficient equal to 94.77 (%) and Jaccard coefficient equal to 84.74 (%), while we reduce the amount of parameters by a factor of 3.9 and computation by a factor of 2.4 compared to the IVD-Net. Experiments show that MLP-Res-Unet improves segmentation performance and creates a simpler model structure while reducing the number of parameters and computation.

Visuo-motor transformations in the intraparietal sulcus mediate the acquisition of endovascular medical skill

Performing endovascular medical interventions safely and efficiently requires a diverse set of skills that need to be practised in dedicated training sessions. Here, we used multimodal magnetic resonance (MR) imaging to determine the structural and functional plasticity and core skills associated with skill acquisition. A training group learned to perform a simulator-based endovascular procedure, while a control group performed a simplified version of the task; multimodal MR images were acquired before and after training. Using a well-controlled interaction design, we found strong multimodal evidence for the role of the intraparietal sulcus (IPS) in endovascular skill acquisition that is in line with previous work implicating the structure in visuospatial transformations including simple visuo-motor and mental rotation tasks. Our results provide a unique window into the multimodal nature of rapid structural and functional plasticity of the human brain while learning a multifaceted and complex clinical skill. Further, our results provide a detailed description of the plasticity process associated with endovascular skill acquisition and highlight specific facets of skills that could enhance current medical pedagogy and be useful to explicitly target during clinical resident training.

Glioma Segmentation-Oriented Multi-Modal MR Image Fusion With Adversarial Learning

Dear Editor, In recent years, multi-modal medical image fusion has received widespread attention in the image processing community. However, existing works on medical image fusion methods are mostly devoted to pursuing high performance on visual perception and objective fusion metrics, while ignoring the specific purpose in clinical applications. In this letter, we propose a glioma segmentation-oriented multi-modal magnetic resonance (MR) image fusion method using an adversarial learning framework, which adopts a segmentation network as the discriminator to achieve more meaningful fusion results from the perspective of the segmentation task. Experimental results demonstrate the advantage of the proposed method over some state-of-the-art medical image fusion methods.

Brain Tumor Segmentation From Multi-Modal MR Images via Ensembling UNets.

Glioma is a type of severe brain tumor, and its accurate segmentation is useful in surgery planning and progression evaluation. Based on different biological properties, the glioma can be divided into three partially-overlapping regions of interest, including whole tumor (WT), tumor core (TC), and enhancing tumor (ET). Recently, UNet has identified its effectiveness in automatically segmenting brain tumor from multi-modal magnetic resonance (MR) images. In this work, instead of network architecture, we focus on making use of prior knowledge (brain parcellation), training and testing strategy (joint 3D+2D), ensemble and post-processing to improve the brain tumor segmentation performance. We explore the accuracy of three UNets with different inputs, and then ensemble the corresponding three outputs, followed by post-processing to achieve the final segmentation. Similar to most existing works, the first UNet uses 3D patches of multi-modal MR images as the input. The second UNet uses brain parcellation as an additional input. And the third UNet is inputted by 2D slices of multi-modal MR images, brain parcellation, and probability maps of WT, TC, and ET obtained from the second UNet. Then, we sequentially unify the WT segmentation from the third UNet and the fused TC and ET segmentation from the first and the second UNets as the complete tumor segmentation. Finally, we adopt a post-processing strategy by labeling small ET as non-enhancing tumor to correct some false-positive ET segmentation. On one publicly-available challenge validation dataset (BraTS2018), the proposed segmentation pipeline yielded average Dice scores of 91.03/86.44/80.58% and average 95% Hausdorff distances of 3.76/6.73/2.51 mm for WT/TC/ET, exhibiting superior segmentation performance over other state-of-the-art methods. We then evaluated the proposed method on the BraTS2020 training data through five-fold cross validation, with similar performance having also been observed. The proposed method was finally evaluated on 10 in-house data, the effectiveness of which has been established qualitatively by professional radiologists.

An Automatic Deep Learning–Based Workflow for Glioblastoma Survival Prediction Using Preoperative Multimodal MR Images: A Feasibility Study

PurposeMost radiomic studies use the features extracted from the manually drawn tumor contours for classification or survival prediction. However, large interobserver segmentation variations lead to inconsistent features and hence introduce more challenges in constructing robust prediction models. Here, we proposed an automatic workflow for glioblastoma (GBM) survival prediction based on multimodal magnetic resonance (MR) images. Methods and MaterialsTwo hundred eighty-five patients with glioma (210 GBM, 75 low-grade glioma) were included. One hundred sixty-three of the patients with GBM had overall survival data. Every patient had 4 preoperative MR images and manually drawn tumor contours. A 3-dimensional convolutional neural network, VGG-Seg, was trained and validated using 122 patients with glioma for automatic GBM segmentation. The trained VGG-Seg was applied to the remaining 163 patients with GBM to generate their autosegmented tumor contours. The handcrafted and deep learning (DL)–based radiomic features were extracted from the autosegmented contours using explicitly designed algorithms and a pretrained convolutional neural network, respectively. One hundred sixty-three patients with GBM were randomly split into training (n = 122) and testing (n = 41) sets for survival analysis. Cox regression models were trained to construct the handcrafted and DL-based signatures. The prognostic powers of the 2 signatures were evaluated and compared. ResultsThe VGG-Seg achieved a mean Dice coefficient of 0.86 across 163 patients with GBM for GBM segmentation. The handcrafted signature achieved a C-index of 0.64 (95% confidence interval, 0.55-0.73), whereas the DL-based signature achieved a C-index of 0.67 (95% confidence interval, 0.57-0.77). Unlike the handcrafted signature, the DL-based signature successfully stratified testing patients into 2 prognostically distinct groups. ConclusionsThe VGG-Seg generated accurate GBM contours from 4 MR images. The DL-based signature achieved a numerically higher C-index than the handcrafted signature and significant patient stratification. The proposed automatic workflow demonstrated the potential of improving patient stratification and survival prediction in patients with GBM.

Non-enhanced multimodal magnetic resonance imaging in assessment of iliac vein obstruction with or without thrombosis.

To evaluate the diagnostic accuracy of a contrast-free multimodal magnetic resonance (MR) protocol (including M2DIPEAR, THRIVE, BTFE-SPAIR, and FLAIR sequences) in the detection of iliac vein obstruction with or without thrombosis. From May 1st, 2015, to May 1st, 2016, a total of 73 patients (aged 51.33 ± 4.21 years) who received both digital subtraction angiography (DSA) and the multimodal MR imaging were included. The protocol of the multimodal MR included M2DIPEAR and BTFE-SPAIR for presenting iliac vein obstruction, and THRIVE and FLAIR for revealing the co-existed thrombosis. Three observers who were blinded to clinical and DSA results independently analyzed all multimodal MR datasets. Per-patient evaluations on presence or absence of iliac vein obstruction were performed to calculate the diagnostic performance of MR imaging (DSA regarded as gold reference) in terms of overall accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Capability to display the co-existing venous thrombus was also evaluated per-MR sequence using a 3-point scale system. Iliac vein obstruction was depicted with DSA in 64 patients. In per-patient evaluation, the multimodal MR imaging yielded accuracy of 95.9% (70/73), sensitivity of 96.9% (62/64), specificity of 88.9% (8/9), positive predictive value of 98.4% (62/63), and negative predictive value of 80% (8/10), respectively. In the multimodal MR sequences, balanced turbo field echo-spectral attenuated inversion recovery (BTFE-SPAIR) sequence was superior to other sequences in depicting the iliac vein configuration, but fluid attenuated inversion recovery (FLAIR) and T1 high-resolution isovolumetric examination (THRIVE) seemed superior in detecting co-existing venous thrombosis. M2DIPEAR and BTFE-SPAIR sequence can reveal iliac vein obstruction while THRIVE and FLAIR can detect the co-existed thrombosis. The proposed multimodal MR protocol can accurately depict the iliac vein obstruction and accurately detect the co-existing venous thrombosis comparable with that of DSA.

A coarse-to-fine registration method for three-dimensional MR images.

Three-dimensional (3D) multimodal magnetic resonance (MR) image registration aims to align similar things in different MR images spatially. Such a technology is useful in auxiliary disease diagnosis and surgical treatment. However, inconsistent intensity correspondence and large initial displacement contribute to the difficulty in registering multimodal MR volumes. A coarse-to-fine method is proposed in this study for pairwise 3D MR image rigid registration. Firstly, the proposed method extracts image feature points to form unregistered point sets and performs coarse registration based on point set registration to reduce the initial displacements of offset images effectively. Then, this method calculates a grey histogram based on voxels in the adaptive region of interest and further improves registration accuracy by maximizing mutual information of coarse-registered images. Some representative registration methods are compared on the basis of three MR image datasets to evaluate the performance of the proposed method. Experimental results show that the proposed method improved more in registration success rate and accuracy compared with conventional registration methods, especially when initial displacements are large.

Sodium Cholate Bile Acid-Stabilized Ferumoxytol-Doxorubicin-Lipiodol Emulsion for Transcatheter Arterial Chemoembolization of Hepatocellular Carcinoma

PurposeTo develop bile acid-stabilized multimodal magnetic resonance (MR) imaging and computed tomography (CT)-visible doxorubicin eluting lipiodol emulsion for transarterial chemoembolization of hepatocellular carcinoma (HCC). Materials and MethodsFerumoxytol, a US Food and Drug Administration-approved iron oxide nanoparticle visible under MR imaging was electrostatically complexed with doxorubicin (DOX). An amphiphilic bile acid, sodium cholate (SC), was used to form a stable dispersion of ferumoxytol-DOX complex in lipiodol emulsion. Properties of the fabricated emulsion were characterized in various component ratios. Release kinetics of DOX were evaluated for the chemoembolization applications. Finally, in vivo multimodal MR imaging/CT imaging properties and potential therapeutic effects upon intra-arterial (IA) infusion bile acid-stabilized ferumoxytol-DOX-lipiodol emulsion were evaluated in orthotopic McA-Rh7777 HCC rat models. ResultsDOX complexed with ferumoxytol through electrostatic interaction. Amphiphilic SC bile acid at the interface between the aqueous ferumoxytol-DOX complexes and lipiodol enabled a sustained DOX release (17.2 ± 1.6% at 24 hours) at an optimized component ratio. In McA Rh7777 rat HCC model, IA-infused emulsion showed a significant contrast around tumor in both T2-weighted MR imaging and CT images (P = .044). Hematoxylin and eosin and Prussian blue staining confirmed the local deposition of IA-infused SC bile acid-stabilized emulsion in the tumor. The deposited emulsion induced significant increases in TUNEL (terminal deoxynucleotidyl transferase dUTP nick end labeling) stain-positive cancer cell apoptosis compared to those in a group treated with the nonstabilized emulsion. ConclusionsSC bile acid-stabilized ferumoxytol-DOX-lipiodol emulsion demonstrated sustained drug release and multimodal MR imaging/CT imaging capabilities. The new lipiodol-based formulation may enhance the therapeutic efficacy of chemoembolization in HCC.

The effect of glioblastoma heterogeneity on survival stratification: a multimodal MR imaging texture analysis

Background Quantitative evaluation of the effect of glioblastoma (GBM) heterogeneity on survival stratification would be critical for the diagnosis, treatment decision, and follow-up management. Purpose To evaluate the effect of GBM heterogeneity on survival stratification, using texture analysis on multimodal magnetic resonance (MR) imaging. Material and Methods A total of 119 GBM patients (65 in long-term and 54 in short-term survival group, separated by overall survival of 12 months) were selected from the Cancer Genome Atlas, who underwent the T1-weighted (T1W) contrast-enhanced (CE), T1W, T2-weighted (T2W), and FLAIR sequences. For each sequence, the co-occurrence matrix, run-length matrix, and histogram features were extracted to reflect GBM heterogeneity on different scale. The recursive feature elimination based support vector machine was adopted to find an optimal subset. Then the stratification performance of four MR sequences was evaluated, both alone and in combination. Results When each sequence used alone, the T1W-CE sequence performed best, with an area under the receiver operating characteristic curve, accuracy, sensitivity, and specificity of 0.7915, 80.67%, 78.45%, and 83.33%, respectively. When the four sequences combined, the stratification performance was basically equal to that of T1W-CE sequence. In the optimal subset of features extracted from multimodality, those from the T2W sequence weighted the most. Conclusion All the four sequences could reflect heterogeneous distribution of GBM and thereby affect the survival stratification, especially T1W-CE and T2W sequences. However, the stratification performance using only the T1W-CE sequence can be preserved with omission of other three sequences, when investigating the effect of GBM heterogeneity on survival stratification.

Weekly follow up of acute lesions in three early multiple sclerosis patients using MR spectroscopy and diffusion

ObjectPathophysiological mechanisms underlying multiple sclerosis (MS) lesion formation, including inflammation, demyelination/remyelination and axonal damage, and their temporal evolution are still not clearly understood. To this end, three acute white matter lesions were monitored using a weekly multimodal magnetic resonance (MR) protocol. Materials and methodsThree untreated patients with early relapsing-remitting MS and one healthy control subject were followed weekly for two months. MR protocol included conventional MR imaging (MRI), diffusion tensor imaging (DTI), and localized MR spectroscopy (MRS), performed on the largest gadolinium-enhancing lesion, selected at the first exam. ResultsMean diffusivity increased and fractional anisotropy decreased in lesions compared to healthy control. Cho/Cr ratios remained elevated in lesions throughout the follow-up. In contrast, temporal profiles of mI/Cr ratios varied between patients’ lesions. For patient 1, mI/Cr ratios were already elevated at the beginning of the follow-up. Patients 2 and 3 ratios increase was delayed by two and five weeks. Blood-brain barrier (BBB) recovery occurred after three weeks. ConclusionThis multimodal MR follow-up highlighted the complementary role of DTI and MRS in identifying temporal relationships between BBB disruption, inflammation, and demyelination. Diffusion metrics showed high sensitivity to detect inflammatory processes. The different temporal profiles of mI suggested a potential better specificity to monitor pathological mechanisms occurring after lesion formation, such as glial proliferation and remyelination.

High-Performance Upconversion Nanoprobes for Multimodal MR Imaging of Acute Ischemic Stroke.

Multimodal magnetic resonance (MR) imaging, including MR angiography (MRA) and MR perfusion (MRP), plays a critical role in the diagnosis and surveillance of acute ischemic stroke. However, these techniques are hindered by the low T1 relaxivity, short circulation time, and high leakage rate from vessels of clinical Magnevist. To address these problems, nontoxic polyethylene glycol (PEG)ylated upconversion nanoprobes (PEG-UCNPs) are synthesized and first adopted for excellent MRA and MRP imaging, featuring high diagnostic sensitivity toward acute ischemic stroke in high-resolution imaging. The investigations show that the agent possesses superior advantages over clinical Magnevist, such as much higher relaxivity, longer circulation time, and lower leakage rate, which guarantee much better imaging efficiency. Remarkably, an extremely small dosage (5 mg Gd kg(-1) ) of PEG-UCNPs provides high-resolution MRA imaging with the vascular system delineated much clearer than the Magnevist with clinical dosage as high as 108 mg Gd kg(-1) . On the other hand, the long circulation time of PEG-UCNPs enables the surveillance of the progression of ischemic stroke using MRA or MRP. Once translated, these PEG-UCNPs are expected to be a promising candidate for substituting the clinical Magnevist in MRA and MRP, which will significantly lengthen the imaging time window and improve the overall diagnostic efficiency.

Spatiotemporal characterization of brain infarction by sequential multimodal MR imaging following transient focal ischemia in a Rat model of intra-arterial middle cerebral artery occlusion.

To assess spatiotemporal brain infarction evolution by sequential multimodal magnetic resonance (MR) imaging in an endovascular model of acute stroke in rats. A microwire was selectively placed in the middle cerebral artery (MCA) in 16 consecutives rats during 90minutes occlusion. Longitudinal 7-T MR imaging, including angiography, diffusion, and perfusion was performed during ischemia, immediately after reperfusion, 3h and 24h after subsequent reperfusion. MCA occlusion was complete in 75% and partial in 18.7%. Hypoperfusion (mean ± SD) was observed in all animals during ischemia (-59 ± 18% of contralateral hemisphere, area 31 ± 5mm2). Infarction volume (mean ± SD) was 90 ± 64mm3 during ischemia and 57 ± 67mm3 at 24h. Brain infarction was fronto-parietal cortical in five animals (31%), striatal in four animals (25%), and cortico-striatal in seven animals (44%) at 24h. All rats survived at 24h. This model is suitable to neuroprotection studies because of possible acute and close characterization of spatiotemporal evolution of brain infarction by MR imaging techniques, and evidence of ischemic penumbra, the target of neuroprotection agents. However, optimization of the brain infarct reproducibility needs further technical and neurointerventional tools improvements. • Nitinol microwire is MRI compatible allowing spatiotemporal characterization of brain infarction in rats. • Microwire selective placement in middle cerebral artery allows complete artery occlusion in 75 %. • A diffusion/perfusion mismatch during arterial occlusion is observed in 77 % of rats.

Predicting standard-dose PET image from low-dose PET and multimodal MR images using mapping-based sparse representation

Positron emission tomography (PET) has been widely used in clinical diagnosis for diseases and disorders. To obtain high-quality PET images requires a standard-dose radionuclide (tracer) injection into the human body, which inevitably increases risk of radiation exposure. One possible solution to this problem is to predict the standard-dose PET image from its low-dose counterpart and its corresponding multimodal magnetic resonance (MR) images. Inspired by the success of patch-based sparse representation (SR) in super-resolution image reconstruction, we propose a mapping-based SR (m-SR) framework for standard-dose PET image prediction. Compared with the conventional patch-based SR, our method uses a mapping strategy to ensure that the sparse coefficients, estimated from the multimodal MR images and low-dose PET image, can be applied directly to the prediction of standard-dose PET image. As the mapping between multimodal MR images (or low-dose PET image) and standard-dose PET images can be particularly complex, one step of mapping is often insufficient. To this end, an incremental refinement framework is therefore proposed. Specifically, the predicted standard-dose PET image is further mapped to the target standard-dose PET image, and then the SR is performed again to predict a new standard-dose PET image. This procedure can be repeated for prediction refinement of the iterations. Also, a patch selection based dictionary construction method is further used to speed up the prediction process. The proposed method is validated on a human brain dataset. The experimental results show that our method can outperform benchmark methods in both qualitative and quantitative measures.

Multimodal Quantitative MR Imaging of the Thalamus in Multiple Sclerosis and Neuromyelitis Optica.

To systematically investigate structural and functional alterations of the thalamus and its subregions through a multimodal magnetic resonance (MR) imaging technique and examine its clinical relevance in multiple sclerosis (MS) and neuromyelitis optica (NMO). The institutional review board approved this study, and written informed consent was obtained from each participant. Thirty-seven patients with MS, 39 patients with NMO, and 40 healthy control subjects were recruited. Six MR imaging measurements were obtained for each participant and compared between groups in the thalamus and its seven subregions, including gray matter (GM) volume, fractional anisotropy, mean diffusivity, amplitude of low-frequency fluctuation, cross-correlation coefficient of spontaneous low frequency, and weighted functional connectivity strength. Partial correlation was used to estimate the MR imaging-clinical relationships. Both MS and NMO exhibited widespread GM atrophy (GM volume in MS, 0.244; NMO, 0.297; and control subjects, 0.329; P < .001) and diffusion abnormalities (fractional anisotropy in MS, 0.293; NMO, 0.323; and control subjects, 0.355; P < .001) in the whole thalamus and several subregions, while MS showed more severe changes than NMO. Decreased cross-correlation coefficient of spontaneous low-frequency and weighted functional connectivity strength was observed in several thalamus subregions in MS (P < .05), but no significant functional abnormalities were identified in NMO. GM volume, fractional anisotropy, and mean diffusivity, not functional changes of the thalamus and thalamic subregions, correlated with the patients' clinical variables and exhibited high discriminative power in distinguishing the three groups. Similar patterns of thalamic structural alteration were identified in MS and NMO, but MS showed more severe pathologic changes. The thalamus is a key node for functional disconnection in MS but not in NMO.

The clinical utility of multimodal MR image-guided needle biopsy in cerebral gliomas

Purpose: Our aim was to evaluate the diagnostic value of multimodal Magnetic Resonance (MR) Image in the stereotactic biopsy of cerebral gliomas, and investigate its implications. Materials and Methods: Twenty-four patients with cerebral gliomas underwent 1H Magnetic Resonance Spectroscopy (1H-MRS)- and intraoperative Magnetic Resonance Imaging (iMRI)-supported stereotactic biopsy, and 23 patients underwent only the preoperative MRI-guided biopsy. The diagnostic yield, morbidity and mortality rates were analyzed. In addition, 20 patients underwent subsequent tumor resection, thus the diagnostic accuracy of the biopsy was further evaluated. Results: The diagnostic accuracies of biopsies evaluated by tumor resection in the trial groups were better than control groups (92.3% and 42.9%, respectively, p = 0.031). The diagnostic yield in the trial groups was better than the control groups, but the difference was not statistically significant (100% and 82.6%, respectively, p = 0.05). The morbidity and mortality rates were similar in both groups. Conclusions: Multimodal MR image-guided glioma biopsy is practical and valuable. This technique can increase the diagnostic accuracy in the stereotactic biopsy of cerebral gliomas. Besides, it is likely to increase the diagnostic yield but requires further validation.

Unsupervised Segmentation of Head Tissues from Multi-modal MR Images for EEG Source Localization.

In this paper, we present and evaluate an automatic unsupervised segmentation method, hierarchical segmentation approach (HSA)-Bayesian-based adaptive mean shift (BAMS), for use in the construction of a patient-specific head conductivity model for electroencephalography (EEG) source localization. It is based on a HSA and BAMS for segmenting the tissues from multi-modal magnetic resonance (MR) head images. The evaluation of the proposed method was done both directly in terms of segmentation accuracy and indirectly in terms of source localization accuracy. The direct evaluation was performed relative to a commonly used reference method brain extraction tool (BET)-FMRIB's automated segmentation tool (FAST) and four variants of the HSA using both synthetic data and real data from ten subjects. The synthetic data includes multiple realizations of four different noise levels and several realizations of typical noise with a 20% bias field level. The Dice index and Hausdorff distance were used to measure the segmentation accuracy. The indirect evaluation was performed relative to the reference method BET-FAST using synthetic two-dimensional (2D) multimodal magnetic resonance (MR) data with 3% noise and synthetic EEG (generated for a prescribed source). The source localization accuracy was determined in terms of localization error and relative error of potential. The experimental results demonstrate the efficacy of HSA-BAMS, its robustness to noise and the bias field, and that it provides better segmentation accuracy than the reference method and variants of the HSA. They also show that it leads to a more accurate localization accuracy than the commonly used reference method and suggest that it has potential as a surrogate for expert manual segmentation for the EEG source localization problem.