Frequency Learning via Multi-scale Fourier Transformer for MRI Reconstruction.

Abstract

Since Magnetic Resonance Imaging(MRI) requires a long acquisition time, various methods were proposed to reduce the time, but they ignored the frequency information and non-local similarity, so that they failed to reconstruct images with a clear structure. In this paper, we propose Frequency Learning via Multi-scale Fourier Transformer for MRI Reconstruction(FMTNet), which focuses on repairing the low-frequency and high-frequency information. Specifically, FMTNet is composed of a high-frequency learning branch(HFLB) and a low-frequency learning branch(LFLB). Meanwhile, we propose a Multi-scale Fourier Transformer(MFT) as the basic module to learn the non-local information. Unlike normal Transformers, MFT adopts Fourier convolution to replace self-attention to efficiently learn global information. Moreover, we further introduce a multi-scale learning and cross-scale linear fusion strategy in MFT to interact information between features of different scales and strengthen the representation of features. Compared with normal Transformers, the proposed MFT occupies fewer computing resources. Based on MFT, we design a Residual Multi-scale Fourier Transformer module as the main component of HFLB and LFLB. We conduct several experiments under different acceleration rates and different sampling patterns on different datasets, and the experiment results show that our method is superior to the previous state-of-the-art method. The code and dataset will be available at: https://github.com/Joyies/FMTNet.