TSRNet: Tongue image segmentation with global and local refinement

Abstract

Tongue Image Segmentation is an essential task for intelligent Traditional Chinese Medicine (TCM), as the tongue is sensitive to the physiological conditions and pathological changes of patients and can help physicians determine strategies for the syndrome differentiation. However, it is a big challenge to acquire an accurate tongue segmentation mask, due to the varied shape and texture of the tongue. This paper proposes a novel tongue segmentation network based on an encoder–decoder framework with global and local refinement, named TSRNet. In the global refinement module, we design an effective module for fusing features from an autoencoder, which is pre-trained on tongue images with segmentation labels, so that the network can make use of the prior knowledge. Moreover, in the local refinement module, we perform patch sampling according to the coarse prediction boundary and correct errors through a patch segmentation module. Both two modules are plugged into the decoder to obtain better tongue segmentation results by training end-to-end. Experimental results compared with state-of-the-art models on two real-world tongue datasets demonstrate the effectiveness of the proposed TSRNet.