MR image harmonization based on cross‐center style transfer using deep generative adversarial network

Abstract

AbstractBackgroundDue to heterogeneities in protocols such as magnetic field strength, coil channels, and scanner manufacturers, data shift problems occur. This causes a serious performance drop in predictions tested to other hospitals. We proposed a multi center‐cross transfer network aimed to adapt institutions with minimal loss of predictions for Alzheimer disease.MethodThree datasets collected from hospitals with different manufacturers and protocols were included in this study (Table1). We used a deep generative adversarial network (GAN) consisting of 4 models: The generator learns to generate images by inputting a source image and a style code. The style encoder and mapping network extracts the style code of images and transforms into style spaces of multiple domains. The discriminator distinguishes between real and fake images from multiple domains. The integrated model was adjusted while training based on four losses: adversarial loss, cycle‐consistency loss, style‐reconstruction loss, and style‐diversification loss. To resolve the data imbalance problem, 1200 slices were randomly extracted per institution. For validation, we evaluated how well the proposed deep learning model transforms images while maintaining image quality and variation. Next, classification was performed using a convolutional neural network model (Resnet 50) to check how successfully transfers styles of an institution to another one.ResultWe quantitatively evaluated transfering styles between the desired source and reference institution (Figures 1 and 2). Through interpolation and consistency loss coefficient differences, the source to reference transformation process can be checked continuously (Figure3). The proposed model performed well with SSIM of 0.897, LPIPS of 0.862, which are metrics for assessing quality and variation of image transfering model. To predict harmonization ability, we compared classification performance between each pair of datasets before and after style transfering, respectively. The accuracies for each pair shows dramatic decrese in performance of discriminating different centers: ADNI‐SMC (91.20 to 64.35), ADNI‐GMC (96.52 to 66.66) and GMC‐SMC (87.84 to 57.89).ConclusionWe proposed a multi center‐cross transfer network that freely converts protocol at any desired institution. This work will be able to predict Alzheimer’s biomarkers robustly by harmonizing all data from multiple hospitals, while preserving disease properties encoded in the dataset.