Towards semi-supervised multi-modal rectal cancer segmentation: A large-scale dataset and a multi-teacher uncertainty-aware network

Abstract

Rectal cancer is one of the most common malignant tumors of the digestive tract. Recently, deep learning has attracted significant attention in computer-aided cancerous region segmentation, offering better adjuvant treatment options for colorectal cancer patients. However, the size of existing rectal cancer datasets is insufficient, which severely limits the training of data-hungry segmentation models based on deep learning. Moreover, the existing datasets only contain image modality, while non-image modality information such as blood test reports is also important in practical clinical diagnosis. Hence, in this paper, we first collect the largest public dataset named RC4 for rectal cancer segmentation by far. RC4 contains 51,097 MRI slices from 657 patients of colorectal cancer, including MRI data and non-image data. We further propose a multi-modal multi-teacher uncertainty-aware network (M-MTUNet) for semi-supervised rectal cancer segmentation. Inspired by the human learning process, multi-teacher distillation is designed to transfer a variety of knowledge to a single student, which provides the student model with richer information. Besides, we propose a multi-modal uncertainty-aware consistency loss, which facilitates the knowledge transfer process from the two teacher networks to the student network and reduces the learning uncertainty within the student network. Extensive experiments demonstrate that M-MTUNet outperforms the current widely-used fully- and semi-supervised segmentation models.