IMPORTANT-Net: Integrated MRI multi-parametric increment fusion generator with attention network for synthesizing absent data

Abstract

Magnetic resonance imaging (MRI) is highly sensitive for lesion detection. Sequences obtained with different settings can capture specific characteristics of lesions. Such multi-parametric MRI information has been shown to aid radiologist performance in lesion classification, as well as improving the performance of artificial intelligence models in various tasks. However, obtaining multi-parametric MRI makes the examination costly from both financial and time perspectives, and there may also be safety concerns for special populations, thus making acquisition of the full spectrum of MRI sequences less durable. In this study, a sophisticated Integrated MRI Multi-Parametric increment fusiOn generatoR wiTh AtteNTion Network (IMPORTANT-Net) is developed to generate absent sequences/parameters. First, the parameter reconstruction module is used to encode and restore the existing MRI parameters to obtain the corresponding latent representation information at any scale level. Then the multi-parametric fusion with attention module enables the interaction of the encoded information from different parameters through a set of algorithmic strategies, and applies different weights to the information through the attention mechanism after information fusion to obtain refined representation information. Finally, a increment fusion scheme embedded in a V-shape generation module is used to combine the hierarchical representations to generate specified absent MRI parameter. Results showed that our IMPORTANT-Net is capable of synthesizing absent MRI, outperforms comparable state-of-the-art networks and more importantly benefit downstream tasks. The codes are available at https://github.com/Netherlands-Cancer-Institute/MRI_IMPORTANT_NET