Magnetic Resonance Imaging Reconstruction Research Articles

Data- and Physics-driven Deep Learning Based Reconstruction for Fast MRI: Fundamentals and Methodologies.

Magnetic Resonance Imaging (MRI) is a pivotal clinical diagnostic tool, yet its extended scanning times often compromise patient comfort and image quality, especially in volumetric, temporal and quantitative scans. This review elucidates recent advances in MRI acceleration via data and physics-driven models, leveraging techniques from algorithm unrolling models, enhancement-based methods, and plug-and-play models to the emerging full spectrum of generative model-based methods. We also explore the synergistic integration of data models with physics-based insights, encompassing the advancements in multi-coil hardware accelerations like parallel imaging and simultaneous multi-slice imaging, and the optimization of sampling patterns. We then focus on domain-specific challenges and opportunities, including image redundancy exploitation, image integrity, evaluation metrics, data heterogeneity, and model generalization. This work also discusses potential solutions and future research directions, with an emphasis on the role of data harmonization and federated learning for further improving the general applicability and performance of these methods in MRI reconstruction.

ReconFormer: Accelerated MRI Reconstruction Using Recurrent Transformer.

The accelerating magnetic resonance imaging (MRI) reconstruction process is a challenging ill-posed inverse problem due to the excessive under-sampling operation in k -space. In this paper, we propose a recurrent Transformer model, namely ReconFormer, for MRI reconstruction, which can iteratively reconstruct high-fidelity magnetic resonance images from highly under-sampled k -space data (e.g., up to 8× acceleration). In particular, the proposed architecture is built upon Recurrent Pyramid Transformer Layers (RPTLs). The core design of the proposed method is Recurrent Scale-wise Attention (RSA), which jointly exploits intrinsic multi-scale information at every architecture unit as well as the dependencies of the deep feature correlation through recurrent states. Moreover, benefiting from its recurrent nature, ReconFormer is lightweight compared to other baselines and only contains 1.1 M trainable parameters. We validate the effectiveness of ReconFormer on multiple datasets with different magnetic resonance sequences and show that it achieves significant improvements over the state-of-the-art methods with better parameter efficiency. The implementation code and pre-trained weights are available at https://github.com/guopengf/ReconFormer.

A Complex Quasi-Newton Proximal Method for Image Reconstruction in Compressed Sensing MRI.

Model-based methods are widely used for reconstruction in compressed sensing (CS) magnetic resonance imaging (MRI), using regularizers to describe the images of interest. The reconstruction process is equivalent to solving a composite optimization problem. Accelerated proximal methods (APMs) are very popular approaches for such problems. This paper proposes a complex quasi-Newton proximal method (CQNPM) for the wavelet and total variation based CS MRI reconstruction. Compared with APMs, CQNPM requires fewer iterations to converge but needs to compute a more challenging proximal mapping called weighted proximal mapping (WPM). To make CQNPM more practical, we propose efficient methods to solve the related WPM. Numerical experiments on reconstructing non-Cartesian MRI data demonstrate the effectiveness and efficiency of CQNPM.

An MTL1TV non-convex regularization model for MR Image reconstruction using the alternating direction method of multipliers

<abstract><p>The acquisition time of magnetic resonance imaging (MRI) is relatively long. To achieve high-quality and fast reconstruction of magnetic resonance (MR) images, we proposed a non-convex regularization model for MR image reconstruction with the modified transformed $ {l_1} $ total variation (MTL1TV) regularization term. We addressed this new model using the alternating direction method of multipliers (ADMM). To evaluate the proposed MTL1TV model, we performed numerical experiments on several MR images. The numerical results showed that the proposed model gives reconstructed images of improved quality compared with those obtained from state of the art models. The results indicated that the proposed model can effectively reconstruct MR images.</p></abstract>

Correlated and Multi-Frequency Diffusion Modeling for Highly Under-Sampled MRI Reconstruction.

Given the obstacle in accentuating the reconstruction accuracy for diagnostically significant tissues, most existing MRI reconstruction methods perform targeted reconstruction of the entire MR image without considering fine details, especially when dealing with highly under-sampled images. Therefore, a considerable volume of efforts has been directed towards surmounting this challenge, as evidenced by the emergence of numerous methods dedicated to preserving high-frequency content as well as fine textural details in the reconstructed image. In this case, exploring the merits associated with each method of mining high-frequency information and formulating a reasonable principle to maximize the joint utilization of these approaches will be a more effective solution to achieve accurate reconstruction. Specifically, this work constructs an innovative principle named Correlated and Multi-frequency Diffusion Model (CM-DM) for highly under-sampled MRI reconstruction. In essence, the rationale underlying the establishment of such principle lies not in assembling arbitrary models, but in pursuing the effective combinations and replacement of components. It also means that the novel principle focuses on forming a correlated and multi-frequency prior through different high-frequency operators in the diffusion process. Moreover, multi-frequency prior further constraints the noise term closer to the target distribution in the frequency domain, thereby making the diffusion process converge faster. Experimental results verify that the proposed method achieved superior reconstruction accuracy, with a notable enhancement of approximately 2dB in PSNR compared to state-of-the-art methods.

Fast and Reliable Score-Based Generative Model for Parallel MRI.

The score-based generative model (SGM) can generate high-quality samples, which have been successfully adopted for magnetic resonance imaging (MRI) reconstruction. However, the recent SGMs may take thousands of steps to generate a high-quality image. Besides, SGMs neglect to exploit the redundancy in k space. To overcome the above two drawbacks, in this article, we propose a fast and reliable SGM (FRSGM). First, we propose deep ensemble denoisers (DEDs) consisting of SGM and the deep denoiser, which are used to solve the proximal problem of the implicit regularization term. Second, we propose a spatially adaptive self-consistency (SASC) term as the regularization term of the k -space data. We use the alternating direction method of multipliers (ADMM) algorithm to solve the minimization model of compressed sensing (CS)-MRI incorporating the image prior term and the SASC term, which is significantly faster than the related works based on SGM. Meanwhile, we can prove that the iterating sequence of the proposed algorithm has a unique fixed point. In addition, the DED and the SASC term can significantly improve the generalization ability of the algorithm. The features mentioned above make our algorithm reliable, including the fixed-point convergence guarantee, the exploitation of the k space, and the powerful generalization ability.

Evaluation of a deep learning magnetic resonance imaging reconstruction method for synthetic computed tomography generation in prostate radiotherapy

Background and PurposeIn magnetic resonance imaging (MRI) only radiotherapy computed tomography (CT) is excluded. The method relies entirely on synthetic CT images generated from MRI. This study evaluates the compatibility of a commercial synthetic CT (sCT) with an accelerated commercial deep learning reconstruction (DLR) in MRI-only prostate radiotherapy. Materials and MethodsFor a group of 24 patients (cohort 1) the effects of DLR were studied in isolation. MRI data were reconstructed conventionally and with DLR from identical k-space data, and sCTs were generated for both reconstructions. The sCT quality, Hounsfield Unit (HU) and dosimetric impact were investigated. In another group of 15 patients (cohort 2) effects on sCT generation using accelerated MRI acquisition (40 % time reduction) reconstructed with DLR were investigated. ResultssCT images from both cohorts, generated from DLR MRI data, were of clinically expected image quality. The mean dose differences for targets and organs at risks in cohort 1 were <0.06 Gy, corresponding to a 0.1 % prescribed dose difference. Similar dose differences were observed in cohort 2. Gamma pass rates for cohort 1 were 100 % for criteria 3 %/3mm, 2 %/2mm and 1 %/1mm for all dose levels. Mean error and mean absolute error inside the body, between sCTs, averaged over all cohort 1 subjects, were −1.1 ± 0.6 [−2.4 0.2] and 2.9 ± 0.4 [2.3 3.9] HU, respectively. ConclusionsDLR was suitable for sCT generation with clinically negligible differences in HU and calculated dose compared to the conventional MRI reconstruction method. For sCT generation DLR enables scan time reduction, without compromised sCT quality.

Adversarial Learning for MRI Reconstruction and Classification of Cognitively Impaired Individuals.

Game theory-inspired deep learning using a generative adversarial network provides an environment to competitively interact and accomplish a goal. In the context of medical imaging, most work has focused on achieving single tasks such as improving image resolution, segmenting images, and correcting motion artifacts. We developed a dual-objective adversarial learning framework that simultaneously 1) reconstructs higher quality brain magnetic resonance images (MRIs) that 2) retain disease-specific imaging features critical for predicting progression from mild cognitive impairment (MCI) to Alzheimer's disease (AD). We obtained 3-Tesla, T1-weighted brain MRIs of participants from the Alzheimer's Disease Neuroimaging Initiative (ADNI, N=342) and the National Alzheimer's Coordinating Center (NACC, N = 190) datasets. We simulated MRIs with missing data by removing 50% of sagittal slices from the original scans (i.e., diced scans). The generator was trained to reconstruct brain MRIs using the diced scans as input. We introduced a classifier into the GAN architecture to discriminate between stable (i.e., sMCI) and progressive MCI (i.e., pMCI) based on the generated images to facilitate encoding of disease-related information during reconstruction. The framework was trained using ADNI data and externally validated on NACC data. In the NACC cohort, generated images had better image quality than the diced scans (Structural similarity (SSIM) index: 0.553 ± 0.116 versus 0.348 ± 0.108). Furthermore, a classifier utilizing the generated images distinguished pMCI from sMCI more accurately than with the diced scans (F1-score: 0.634 ± 0.019 versus 0.573 ± 0.028). Competitive deep learning has potential to facilitate disease-oriented image reconstruction in those at risk of developing Alzheimer's disease.

Learning From Oversampling: A Systematic Exploitation of Oversampling to Address Data Scarcity Issues in Deep Learning- Based Magnetic Resonance Image Reconstruction

Learning From Oversampling: A Systematic Exploitation of Oversampling to Address Data Scarcity Issues in Deep Learning- Based Magnetic Resonance Image Reconstruction

HARA-GAN: Hybrid Attention and Relative Average Discriminator Based Generative Adversarial Network for MR Image Reconstruction

HARA-GAN: Hybrid Attention and Relative Average Discriminator Based Generative Adversarial Network for MR Image Reconstruction

Adaptive Local Neighborhood-Based Neural Networks for MR Image Reconstruction From Undersampled Data

Adaptive Local Neighborhood-Based Neural Networks for MR Image Reconstruction From Undersampled Data

Numerical and Clinical Evaluation of the Robustness of Open-source Networks for Parallel MR Imaging Reconstruction.

Deep neural networks (DNNs) for MRI reconstruction often require large datasets for training. Still, in clinical settings, the domains of datasets are diverse, and how robust DNNs are to domain differences between training and testing datasets has been an open question. Here, we numerically and clinically evaluate the generalization of the reconstruction networks across various domains under clinically practical conditions and provide practical guidance on what points to consider when selecting models for clinical application. We compare the reconstruction performance between four network models: U-Net, the deep cascade of convolutional neural networks (DC-CNNs), Hybrid Cascade, and variational network (VarNet). We used the public multicoil dataset fastMRI for training and testing and performed a single-domain test, where the domains of the dataset used for training and testing were the same, and cross-domain tests, where the source and target domains were different. We conducted a single-domain test (Experiment 1) and cross-domain tests (Experiments 2-4), focusing on six factors (the number of images, sampling pattern, acceleration factor, noise level, contrast, and anatomical structure) both numerically and clinically. U-Net had lower performance than the three model-based networks and was less robust to domain shifts between training and testing datasets. VarNet had the highest performance and robustness among the three model-based networks, followed by Hybrid Cascade and DC-CNN. Especially, VarNet showed high performance even with a limited number of training images (200 images/10 cases). U-Net was more robust to domain shifts concerning noise level than the other model-based networks. Hybrid Cascade showed slightly better performance and robustness than DC-CNN, except for robustness to noise-level domain shifts. The results of the clinical evaluations generally agreed with the results of the quantitative metrics. In this study, we numerically and clinically evaluated the robustness of the publicly available networks using the multicoil data. Therefore, this study provided practical guidance for clinical applications.

Image Reconstruction for Accelerated MR Scan With Faster Fourier Convolutional Neural Networks.

High quality image reconstruction from undersampled k -space data is key to accelerating MR scanning. Current deep learning methods are limited by the small receptive fields in reconstruction networks, which restrict the exploitation of long-range information, and impede the mitigation of full-image artifacts, particularly in 3D reconstruction tasks. Additionally, the substantial computational demands of 3D reconstruction considerably hinder advancements in related fields. To tackle these challenges, we propose the following: 1) A novel convolution operator named Faster Fourier Convolution (FasterFC), aims at providing an adaptable broad receptive field for spatial domain reconstruction networks with fast computational speed. 2) A split-slice strategy that substantially reduces the computational load of 3D reconstruction, enabling high-resolution, multi-coil, 3D MR image reconstruction while fully utilizing inter-layer and intra-layer information. 3) A single-to-group algorithm that efficiently utilizes scan-specific and data-driven priors to enhance k -space interpolation effects. 4) A multi-stage, multi-coil, 3D fast MRI method, called the faster Fourier convolution based single-to-group network (FAS-Net), comprising a single-to-group k -space interpolation algorithm and a FasterFC-based image domain reconstruction module, significantly minimizes the computational demands of 3D reconstruction through split-slice strategy. Experimental evaluations conducted on the NYU fastMRI and Stanford MRI Data datasets reveal that the FasterFC significantly enhances the quality of both 2D and 3D reconstruction results. Moreover, FAS-Net, characterized as a method that can achieve high-resolution (320, 320, 256), multi-coil, (8 coils), 3D fast MRI, exhibits superior reconstruction performance compared to other state-of-the-art 2D and 3D methods.

Joint MR image reconstruction and super-resolution via mutual co-attention network

Abstract In the realm of medical diagnosis, recent strides in deep neural network-guided magnetic resonance imaging (MRI) restoration have shown promise. Nevertheless, persistent drawbacks overshadow these advancements. Challenges persist in balancing acquisition speed and image quality, while existing methods primarily focus on singular tasks like MRI reconstruction or super-resolution (SR), neglecting the interplay between these tasks. To tackle these challenges, this paper introduces the mutual co-attention network (MCAN) specifically designed to concurrently address both MRI reconstruction and SR tasks. Comprising multiple mutual cooperation attention blocks (MCABs) in succession, MCAN is tailored to maintain consistency between local physiological details and global anatomical structures. The intricately crafted MCAB includes a feature extraction block, a local attention block and a global attention block. Additionally, to ensure data fidelity without compromising acquired data, we propose the channel-wise data consistency block. Thorough experimentation on the IXI and fastMRI dataset showcases MCAN’s superiority over existing state-of-the-art methods. Both quantitative metrics and visual quality assessments validate the enhanced performance of MCAN in MRI restoration. The findings underscore MCAN’s potential in significantly advancing therapeutic applications. By mitigating the trade-off between acquisition speed and image quality while simultaneously addressing both MRI reconstruction and SR tasks, MCAN emerges as a promising solution in the domain of magnetic resonance image restoration.

Deep learning-based magnetic resonance imaging reconstruction for improving the image quality of reduced-field-of-view diffusion-weighted imaging of the pancreas.

It has been reported that deep learning-based reconstruction (DLR) can reduce image noise and artifacts, thereby improving the signal-to-noise ratio and image sharpness. However, no previous studies have evaluated the efficacy of DLR in improving image quality in reduced-field-of-view (reduced-FOV) diffusion-weighted imaging (DWI) [field-of-view optimized and constrained undistorted single-shot (FOCUS)] of the pancreas. We hypothesized that a combination of these techniques would improve DWI image quality without prolonging the scan time but would influence the apparent diffusion coefficient calculation. To evaluate the efficacy of DLR for image quality improvement of FOCUS of the pancreas. This was a retrospective study evaluated 37 patients with pancreatic cystic lesions who underwent magnetic resonance imaging between August 2021 and October 2021. We evaluated three types of FOCUS examinations: FOCUS with DLR (FOCUS-DLR+), FOCUS without DLR (FOCUS-DLR-), and conventional FOCUS (FOCUS-conv). The three types of FOCUS and their apparent diffusion coefficient (ADC) maps were compared qualitatively and quantitatively. FOCUS-DLR+ (3.62, average score of two radiologists) showed significantly better qualitative scores for image noise than FOCUS-DLR- (2.62) and FOCUS-conv (2.88) (P < 0.05). Furthermore, FOCUS-DLR+ showed the highest contrast ratio (CR) between the pancreatic parenchyma and adjacent fat tissue for b-values of 0 and 600 s/mm2 (0.72 ± 0.08 and 0.68 ± 0.08) and FOCUS-DLR- showed the highest CR between cystic lesions and the pancreatic parenchyma for the b-values of 0 and 600 s/mm2 (0.62 ± 0.21 and 0.62 ± 0.21) (P < 0.05), respectively. FOCUS-DLR+ provided significantly higher ADCs of the pancreas and lesion (1.44 ± 0.24 and 3.00 ± 0.66) compared to FOCUS-DLR- (1.39 ± 0.22 and 2.86 ± 0.61) and significantly lower ADCs compared to FOCUS-conv (1.84 ± 0.45 and 3.32 ± 0.70) (P < 0.05), respectively. This study evaluated the efficacy of DLR for image quality improvement in reduced-FOV DWI of the pancreas. DLR can significantly denoise images without prolonging the scan time or decreasing the spatial resolution. The denoising level of DWI can be controlled to make the images appear more natural to the human eye. However, this study revealed that DLR did not ameliorate pancreatic distortion. Additionally, physicians should pay attention to the interpretation of ADCs after DLR application because ADCs are significantly changed by DLR.

Two-and-a-half order score-based model for solving 3D ill-posed inverse problems

Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) are crucial technologies in the field of medical imaging. Score-based models demonstrated effectiveness in addressing different inverse problems encountered in the field of CT and MRI, such as sparse-view CT and fast MRI reconstruction. However, these models face challenges in achieving accurate three dimensional (3D) volumetric reconstruction. The existing score-based models predominantly concentrate on reconstructing two-dimensional (2D) data distributions, resulting in inconsistencies between adjacent slices in the reconstructed 3D volumetric images. To overcome this limitation, we propose a novel two-and-a-half order score-based model (TOSM). During the training phase, our TOSM learns data distributions in 2D space, simplifying the training process compared to working directly on 3D volumes. However, during the reconstruction phase, the TOSM utilizes complementary scores along three directions (sagittal, coronal, and transaxial) to achieve a more precise reconstruction. The development of TOSM is built on robust theoretical principles, ensuring its reliability and efficacy. Through extensive experimentation on large-scale sparse-view CT and fast MRI datasets, our method achieved state-of-the-art (SOTA) results in solving 3D ill-posed inverse problems, averaging a 1.56 dB peak signal-to-noise ratio (PSNR) improvement over existing sparse-view CT reconstruction methods across 29 views and 0.87 dB PSNR improvement over existing fast MRI reconstruction methods with × 2 acceleration. In summary, TOSM significantly addresses the issue of inconsistency in 3D ill-posed problems by modeling the distribution of 3D data rather than 2D distribution which has achieved remarkable results in both CT and MRI reconstruction tasks.

DIR3D: Cascaded Dual-Domain Inter-Scale Mutual Reinforcement 3D Network for highly accelerated 3D MR image reconstruction

Deep Learning has been successfully applied to reconstruct Magnetic Resonance (MR) images from undersampled k-space data to achieve the acceleration of MRI. However, most existing works focused on 2D MRI scans and perform reconstruction in a slice-by-slice manner due to the tremendous computation and memory cost of 3D MRI reconstruction, resulting in a blank in the research of 3D MRI acceleration. Providing more accurate and diagnostic information, 3D MRI requires excessively long scan time. To accelerate 3D MRI, in this work, we design a lightweight and powerful cascaded 3D network DIR3D utilizing a previously proposed computation-friendly data processing strategy. Specifically, we propose an efficient block called Dual-Domain Inter-Scale Mutual Reinforcement Block (DIRB) to fuse multi-scale features locally and globally with neglectable computation and memory costs, which enhances the representative ability of the network. To allow more flexibility, we further redesign the commonly used Data Consistency (DC) layer by introducing a learnable adaptor which enables the network to perform point-wise adaptive merge of the reconstructed and sampled k-space data while ensuring data consistency. We conduct comprehensive experiments on the Stanford MRIData and evaluate our DIR3D from multiple perspectives. When achieving the same acceleration factors, our proposed DIR3D consistently outperforms other state-of-the-art 2D methods at multiple subsampling masks, especially for highly undersampled data, which provides strong evidence for the superiority of our DIR3D for 3D MRI acceleration. Additionally, at the inference stage, our DIR3D can achieve a much higher reconstruction efficiency.

Improved Sensitivity Encoding Parallel Magnetic Resonance Imaging Reconstruction Algorithm Based on Efficient Sum of Outer Products Dictionary Learning

Improved Sensitivity Encoding Parallel Magnetic Resonance Imaging Reconstruction Algorithm Based on Efficient Sum of Outer Products Dictionary Learning

Attention hybrid variational net for accelerated MRI reconstruction

The application of compressed sensing (CS)-enabled data reconstruction for accelerating magnetic resonance imaging (MRI) remains a challenging problem. This is due to the fact that the information lost in k-space from the acceleration mask makes it difficult to reconstruct an image similar to the quality of a fully sampled image. Multiple deep learning-based structures have been proposed for MRI reconstruction using CS, in both the k-space and image domains, and using unrolled optimization methods. However, the drawback of these structures is that they are not fully utilizing the information from both domains (k-space and image). Herein, we propose a deep learning-based attention hybrid variational network that performs learning in both the k-space and image domains. We evaluate our method on a well-known open-source MRI dataset (652 brain cases and 1172 knee cases) and a clinical MRI dataset of 243 patients diagnosed with strokes from our institution to demonstrate the performance of our network. Our model achieves an overall peak signal-to-noise ratio/structural similarity of 40.92 ± 0.29/0.9577 ± 0.0025 (fourfold) and 37.03 ± 0.25/0.9365 ± 0.0029 (eightfold) for the brain dataset, 31.09 ± 0.25/0.6901 ± 0.0094 (fourfold) and 29.49 ± 0.22/0.6197 ± 0.0106 (eightfold) for the knee dataset, and 36.32 ± 0.16/0.9199 ± 0.0029 (20-fold) and 33.70 ± 0.15/0.8882 ± 0.0035 (30-fold) for the stroke dataset. In addition to quantitative evaluation, we undertook a blinded comparison of image quality across networks performed by a subspecialty trained radiologist. Overall, we demonstrate that our network achieves a superior performance among others under multiple reconstruction tasks.

Stable Deep MRI Reconstruction Using Generative Priors.

Data-driven approaches recently achieved remarkable success in magnetic resonance imaging (MRI) reconstruction, but integration into clinical routine remains challenging due to a lack of generalizability and interpretability. In this paper, we address these challenges in a unified framework based on generative image priors. We propose a novel deep neural network based regularizer which is trained in a generative setting on reference magnitude images only. After training, the regularizer encodes higher-level domain statistics which we demonstrate by synthesizing images without data. Embedding the trained model in a classical variational approach yields high-quality reconstructions irrespective of the sub-sampling pattern. In addition, the model shows stable behavior when confronted with out-of-distribution data in the form of contrast variation. Furthermore, a probabilistic interpretation provides a distribution of reconstructions and hence allows uncertainty quantification. To reconstruct parallel MRI, we propose a fast algorithm to jointly estimate the image and the sensitivity maps. The results demonstrate competitive performance, on par with state-of-the-art end-to-end deep learning methods, while preserving the flexibility with respect to sub-sampling patterns and allowing for uncertainty quantification.