SeRL: Style-embedding representation learning for unsupervised CT images synthesis from unpaired MR images

Abstract

Hepatocellular carcinoma (HCC) is the third leading cause of cancer-related deaths worldwide, and the fastest-growing cause of cancer deaths in the United States. Computed tomography (CT) and magnetic resonance (MR) imaging are the key image-based examination methods for HCC patients. Patients may alternatively undergo CT and MR exams given a radiation dose of the former and the cost of the latter. Due to organ representation differences between CT and MR, it’s a great challenge to detect the tumors from a time series of data consisting of both CT and MR by a single model. The annotations for these time series data will consume much time and labor for the physicians. Thus, we propose our style-embedding representation learning (SeRL) for unsupervised and unpaired abdomen CT and MR translation. Different from current medical image translation models, the style-representation information from real CT and real MR images has been embedded in the translation process to bypass some local minima during the convergence process and improve the synthesis results. Our patch-corrosion augmentation method enhances the style-embedding representation learning by bringing more diversity to the training data. Combined with the self-attention module, SeRL eliminates the noise caused by low grayscale pixel values during translation. Results on the unpaired HCC patient's CT and MR images show that our proposed SeRL is able to generate high quality CT images from MR ones. Evaluations such as Frechet Inception Distance (FID), Sliced Wasserstein Distance (SWD), and liver segmentation dice score are utilized to demonstrate our advantages over other state-of-the-art unsupervised methods.