M-Net: A Novel U-Net With Multi-Stream Feature Fusion and Multi-Scale Dilated Convolutions for Bile Ducts and Hepatolith Segmentation

Abstract

Automatically segmenting bile ducts and hepatolith in abdominal CT scans is helpful to assist hepatobiliary surgeons for minimally invasive surgery. High-deformation characteristics of bile ducts and small-size characteristics of hepatolith make this segmentation task challenging. To the best of our knowledge, we make the first attempt to simultaneously segment bile ducts and hepatolith in this paper. Inspired by U-Net, a novel two-dimensional end-to-end fully convolutional network named M-Net is designed to implement this segmentation task. The M-Net is composed of four streams involving two encoder-decoder processes. Multi-scale dilated convolutions are designed to extract abundant semantic features and multi-scale context information at different scales. To make full advantages of multi-scale feature maps, a multi-stream feature fusion strategy is proposed to transfer the most abundant semantic features produced in the first stream to the other streams. To further improve the segmentation performance, a novel loss function is defined to focus the M-Net on hard pixels (difficultly distinguished) in the edges of bile ducts and hepatolith, which is based on the online bootstrapped method and cross entropy. By discarding pixels (easy to distinguish) with higher probability of class, the decline of loss is focused on hard pixels so that the training become more efficient and directional. Experimental results indicate that our proposed M-Net is superior to the state-of-the-art deep-learning methods for simultaneously segmenting bile ducts and hepatolith in the abdominal CT scans. The M-Net can simultaneously segment bile ducts and hepatolith in abdominal CT scans at a high performance with 98.678% Recall, 84.427% Precision, 89.831% DICE and 90.998% F1-score for bile ducts, and 99.894% Recall, 55.132% Precision, 71.248% DICE and 71.051% F1-score for hepatolith.