A weakly supervised tooth‐mark and crack detection method in tongue image

Abstract

AbstractTongue diagnosis is one of the primary clinical diagnostic methods in Traditional Chinese Medicine. Recognizing the tooth‐marked tongue and the crackled tongue plays an essential role in evaluating the status of patients. Previous methods mainly focus on identifying whether a tongue image is a tooth‐marked tongue (cracked tongue) or not, while cannot provide more details. In this study, we propose a weakly supervised method for training the tooth‐mark and crack detection model by leveraging fully bounding‐box level annotated and coarse image‐level annotated tongue images. The proposed model is extended from the YOLO object detection model, and we add several classification branches for recognizing the tooth‐marked tongue and cracked tongue. The classification branch aims to predict the coarse label for both coarse‐labeled data and fully annotated data. The detection branch is used to locate the position of tooth marks and cracks from the fully annotated data. Finally, we utilize a multitask loss function for training the model. Experimental results on a challenging tongue image dataset demonstrate the effectiveness of our proposed weakly supervised method.