A novel tongue feature extraction method on mobile devices

Abstract

In Traditional Chinese Medicine (TCM), tongue diagnosis is essential for symptom differentiation and treatment selection. Compared with traditional tongue diagnostic instruments, deploying a tongue diagnosis system on mobile devices is more convenient to monitor general health and facilitates the development of telemedicine. However, limited by both the quality and quantity of tongue images taken by mobile devices, extracting tongue features of the images maintains a great challenge. In this paper, we present a tongue feature extraction method on mobile devices, containing a high-accuracy tongue segmentation method based on Moment Invariants with Data Augmentation (DAMI) and an efficient and lightweight feature classification model with an attention mechanism. Meanwhile, we construct a novel tongue image dataset from mobile devices for extracting tongue features, significantly, first including sublingual images which are beneficial to extracting sublingual vein features. Extensive experiments on two datasets demonstrate the effectiveness and robustness of our method. Furthermore, our method greatly reduces the computing and storage demands compared with other current methods, providing a good prerequisite for deployment on mobile devices. Finally, to demonstrate the potential application of our proposed method, we develop a TCM intelligence tongue diagnosis application, which can be accessed through the WeChat Mini Program or web version, exhibiting its great potential in clinical diagnosis and health monitoring.