Cross-Model Transformer Method for Medical Image Synthesis

Zebin Hu,Zekuan Yu,Zhendong Li,Hao Liu,Long Wang

doi:10.1155/2021/5624909

Abstract

Acquiring complementary information about tissue morphology from multimodal medical images is beneficial to clinical disease diagnosis, but it cannot be widely used due to the cost of scans. In such cases, medical image synthesis has become a popular area. Recently, generative adversarial network (GAN) models are applied to many medical image synthesis tasks and show prior performance, since they enable to capture structural details clearly. However, GAN still builds the main framework based on convolutional neural network (CNN) that exhibits a strong locality bias and spatial invariance through the use of shared weights across all positions. Therefore, the long-range dependencies have been destroyed in this processing. To address this issue, we introduce a double-scale deep learning method for cross-modal medical image synthesis. More specifically, the proposed method captures locality feature via local discriminator based on CNN and utilizes long-range dependencies to learning global feature through global discriminator based on transformer architecture. To evaluate the effectiveness of double-scale GAN, we conduct folds of experiments on the standard benchmark IXI dataset and experimental results demonstrate the effectiveness of our method.

Highlights

Magnetic resonance imaging (MRI) is a versatile and noninvasive imaging technique widely used in clinical applications
(1) We introduce a double-scale discriminator generative adversarial network (GAN) for medical image synthesis. (2) e global discriminator of our model is designed on vision transformer that utilizes longrange dependencies between distant patches and captures global features
To validate the effectiveness of the proposed synthesis method, we compare it with three stateof-the-art cross-modality synthesis methods: (1) pix2pix [26]: this method is based on a convolutional GAN model and UNet backbone, which synthesizes the whole image by focusing on the pixelwise similarity

Summary

Introduction

Magnetic resonance imaging (MRI) is a versatile and noninvasive imaging technique widely used in clinical applications. Acquiring multimodal MR imaging is often challenging due to numerous factors, such as uncooperative patients, limited availability of scanning time, and the expensive cost of prolonged exams [2, 3]. To address this issue, cross-modal medical image synthesis has been widely used, as it enables to synthesis unattained images in multimodal protocols from the subset of available images [4,5,6,7]

Methods

Results

Conclusion