Unsupervised Domain Adaptation for 3D Medical Image with High Efficiency

Abstract

Domain adaptation is a fundamental problem in the 3D medical image process. The current methods mainly cut the 3D image into 2D slices and then use 2D CNN for processing, which may ignore the inter-slice information of the 3D medical image. Methods based on 3D CNN can capture the inter-slice information but lead to extensive memory consumption and lots of training time. In this paper, we aim to model the inter-slice information in 2D CNN to realize the unsupervised 3D medical image’s domain adaptation without additional cost from 3D convolutions. To better capture the inter-slice information, we train the model from the adjacent (local) slices and the global slice sequence perspective. We first propose the Slice Subtract Module method (SSM), which can easily embed into 2D CNN and model the adjacent slices by introducing very limited extra computation cost. We then train the adaptation model with a distance consistency loss supervised by the LSTM component to model the global slice sequence. Extensive experiments on BraTS2019 and Chaos datasets show that our method can effectively improve the quality of domain adaptation and achieve state-of-the-art accuracy on segmentation tasks with little computation increased while remaining parameters unchanged.