Mutli-modal straight flow matching for accelerated MR imaging

Abstract

Diffusion models have garnered great interest lately in Magnetic Resonance (MR) image reconstruction. A key component of generating high-quality samples from noise is iterative denoising for thousands of steps. However, the complexity of inference steps has limited its applications. To solve the challenge in obtaining high-quality reconstructed images with fewer inference steps and computational complexity, we introduce a novel straight flow matching, based on a neural ordinary differential equation (ODE) generative model. Our model creates a linear path between undersampled images and reconstructed images, which can be accurately simulated with a few Euler steps. Furthermore, we propose a multi-modal straight flow matching model, which uses relatively easily available modalities as supplementary information to guide the reconstruction of target modalities. We introduce the low frequency fusion layer and the high frequency fusion layer into our multi-modal model, which has been proved to produce promising results in fusion tasks. The proposed multi-modal straight flow matching (MMSflow) achieves state-of-the-art performances in task of reconstruction in fastMRI and Brats-2020 and improves the sampling rate by an order of magnitude than other methods based on stochastic differential equations (SDE).