Division and Fusion: Rethink Convolutional Kernels for 3D Medical Image Segmentation

Abstract

There has been a debate of using 2D and 3D convolution on volumetric medical image segmentation. The problem is that 2D convolution loses 3D spatial relationship of image features, while 3D convolution layers are hard to train from scratch due to the limited size of medical image dataset. Employing more trainable parameters and complicated connections may improve the performance of 3D CNN, however, inducing extra computational burden at the same time. It is meaningful to improve performance of current 3D medical image processing without requiring extra inference computation and memory resources. In this paper, we propose a general solution, Division-Fusion (DF)-CNN for free performance improvement on any available 3D medical image segmentation approach. During the division phase, different view-based kernels are divided from a single 3D kernel to extract multi-view context information that strengthens the spatial information of feature maps. During the fusion phase, all kernels are fused into one 3D kernel to reduce the parameters of deployed model. We extensively evaluated our DF mechanism on prostate ultrasound volume segmentation. The results demonstrate a consistent improvement over different benchmark models with a clear margin.