Multi-organ CT segmentation using shifted-window multilayer perceptron mixer

Abstract

This work proposes a novel U-shaped neural network, Shifted-window MLP (Swin-MLP), that incorporates a Convolutional Neural Network (CNN) and Multilayer Linear Perceptron-Mixer (MLP-Mixer) for automatic CT multi-organ segmentation. The network has a structure like V-net: 1) a Shifted-window MLP-Mixer encoder learns semantic features from the input CT scans, and 2) a decoder, which mirrors the architecture of the encoder, then reconstructs segmentation maps from the encoder’s features. Novel to the proposed network, we apply a Shifted-window MLP-Mixer rather than convolutional layers to better model both global and local representations of the input scans. We evaluate the proposed network using an institutional pelvic dataset comprising 120 CT scans, and a public abdomen dataset containing 30 scans. The network’s segmentation accuracy is evaluated in two domains: 1) volume-based accuracy is measured by Dice Similarity Coefficient (DSC), segmentation sensitivity, and precision; 2) surface-based accuracy is measured by Hausdorff Distance (HD), Mean Surface Distance (MSD), and Residual Mean Square distance (RMS). The average DSC achieved by MLP-Vnet on the pelvic dataset is 0.866; sensitivity is 0.883, precision is 0.856, HD is 11.523 millimeter (mm), MSD is 3.926 mm, and RMS is 6.262 mm. The average DSC on the public abdomen dataset is 0.903, and HD is 5.275 mm. The proposed MLP-Mixer-Vnet demonstrates significant improvement over CNN-based networks. The automatic multi-organ segmentation tool may potentially facilitate the current radiotherapy treatment planning workflow.