Tongue Image Segmentation Research Articles

The Relationship Between Computerized Face and Tongue Image Segmentation and Metabolic Parameters in Patients with Type 2 Diabetes Based on Machine Learning.

We aim to examine and reestablish the correlational and linear regression relationships, as well as the predictive value, between the significant facial and tongue features and the metabolic parameters in type 2 diabetes mellitus (T2DM). From March to May 2024, we studied 269 patients with T2DM in the endocrinology department of Shanghai Pudong Hospital. The patients' facial and tongue characteristics were sampling by a tongue imaging device equipped with artificial intelligence (AI) (XiMaLife, Sinology, China) of automated and advanced machine learning algorithms. Then, the imaging features were examined in relation to the blood examination. Multiple facial and tongue features, as well as dimensional facial and tongue color parameters, were significantly correlated with glycated hemoglobin A1c (HbA1c) (r < 0.3, p < 0.05), glycated albumin (GA) (-0.20 < 0.30, p < 0.05), C-peptide (-0.20.20, p < 0.05), plasma insulin (r < 0.30, p < 0.05), fasting plasma glucose (FPG) (r < 0.3, p < 0.05), significant hepatic and renal function indicators (-0.30 < r < 0.20, p<0.05), cardiac injury markers (-0.30 < r < 0.30, p < 0.05), tumor markers (-0.5 < r < 0.5, p < 0.05), thyroid function (-0.15 < r < 0.55, p < 0.05), and blood cell count, including white blood cells (r < 0.2, p < 0.05), and hemoglobin (Hb) (-0.30 < r < 0.3, 0.0001. The correlational results demonstrated that the tongue's characteristics and signsmay be linked with the dynamic of the metabolic status of T2DM. In order to examine the causal relationships, we performed linear regression analyses, which revealed that various facial and tongue imaging parameters partially determined the metabolic indicators. The predictive value of imaging features was evaluated by receiver operating characteristic curve (ROC) to assess metabolic status in T2DM. This study demonstrated that metabolic status, renal and hepatic, cardiac, and thyroid function, the proportion of blood cells, and Hb in T2DM were intimately associated with facial and tongue features. The precise analysis of facial and tongue features through AI and advanced machine learning could be used to predict T2DM's conditions and progression.

S5Utis: Structured State-Space Sequence SegNeXt UNet-like Tongue Image Segmentation in Traditional Chinese Medicine.

Intelligent Traditional Chinese Medicine can provide people with a convenient way to participate in daily health care. The ease of acceptance of Traditional Chinese Medicine is also a major advantage in promoting health management. In Traditional Chinese Medicine, tongue imaging is an important step in the examination process. The segmentation and processing of the tongue image directly affects the results of intelligent Traditional Chinese Medicine diagnosis. As intelligent Traditional Chinese Medicine continues to develop, remote diagnosis and patient participation will play important roles. Smartphone sensor cameras can provide irreplaceable data collection capabilities in enhancing interaction in smart Traditional Chinese Medicine. However, these factors lead to differences in the size and quality of the captured images due to factors such as differences in shooting equipment, professionalism of the photographer, and the subject's cooperation. Most current tongue image segmentation algorithms are based on data collected by professional tongue diagnosis instruments in standard environments, and are not able to demonstrate the tongue image segmentation effect in complex environments. Therefore, we propose a segmentation algorithm for tongue images collected in complex multi-device and multi-user environments. We use convolutional attention and extend state space models to the 2D environment in the encoder. Then, cross-layer connection fusion is used in the decoder part to fuse shallow texture and deep semantic features. Through segmentation experiments on tongue image datasets collected by patients and doctors in real-world settings, our algorithm significantly improves segmentation performance and accuracy.

RTC_TongueNet: An improved tongue image segmentation model based on DeepLabV3.

Tongue segmentation as a basis for automated tongue recognition studies in Chinese medicine, which has defects such as network degradation and inability to obtain global features, which seriously affects the segmentation effect. This article proposes an improved model RTC_TongueNet based on DeepLabV3, which combines the improved residual structure and transformer and integrates the ECA (Efficient Channel Attention Module) attention mechanism of multiscale atrous convolution to improve the effect of tongue image segmentation. In this paper, we improve the backbone network based on DeepLabV3 by incorporating the transformer structure and an improved residual structure. The residual module is divided into two structures and uses different residual structures under different conditions to speed up the frequency of shallow information mapping to deep network, which can more effectively extract the underlying features of tongue image; introduces ECA attention mechanism after concat operation in ASPP (Atrous Spatial Pyramid Pooling) structure to strengthen information interaction and fusion, effectively extract local and global features, and enable the model to focus more on difficult-to-separate areas such as tongue edge, to obtain better segmentation effect. The RTC_TongueNet network model was compared with FCN (Fully Convolutional Networks), UNet, LRASPP (Lite Reduced ASPP), and DeepLabV3 models on two datasets. On the two datasets, the MIOU (Mean Intersection over Union) and MPA (Mean Pixel Accuracy) values of the classic model DeepLabV3 were higher than those of FCN, UNet, and LRASPP models, and the performance was better. Compared with the DeepLabV3 model, the RTC_TongueNet network model increased MIOU value by 0.9% and MPA value by 0.3% on the first dataset; MIOU increased by 1.0% and MPA increased by 1.1% on the second dataset. RTC_TongueNet model performed best on both datasets. In this study, based on DeepLabV3, we apply the improved residual structure and transformer as a backbone to fully extract image features locally and globally. The ECA attention module is combined to enhance channel attention, strengthen useful information, and weaken the interference of useless information. RTC_TongueNet model can effectively segment tongue images. This study has practical application value and reference value for tongue image segmentation.

DeepDSP: Deep Convolutional Network with Double Spatial Pyramid for tongue image segmentation

DeepDSP: Deep Convolutional Network with Double Spatial Pyramid for tongue image segmentation

Deep-DSP: deep convolutional network with double spatial pyramid for tongue image segmentation

Deep-DSP: deep convolutional network with double spatial pyramid for tongue image segmentation

Semi-supervised tongue image segmentation method for traditional chinese medicine based on mutual learning with dual models

Semi-supervised tongue image segmentation method for traditional chinese medicine based on mutual learning with dual models

TSRNet: Tongue image segmentation with global and local refinement

Tongue Image Segmentation is an essential task for intelligent Traditional Chinese Medicine (TCM), as the tongue is sensitive to the physiological conditions and pathological changes of patients and can help physicians determine strategies for the syndrome differentiation. However, it is a big challenge to acquire an accurate tongue segmentation mask, due to the varied shape and texture of the tongue. This paper proposes a novel tongue segmentation network based on an encoder–decoder framework with global and local refinement, named TSRNet. In the global refinement module, we design an effective module for fusing features from an autoencoder, which is pre-trained on tongue images with segmentation labels, so that the network can make use of the prior knowledge. Moreover, in the local refinement module, we perform patch sampling according to the coarse prediction boundary and correct errors through a patch segmentation module. Both two modules are plugged into the decoder to obtain better tongue segmentation results by training end-to-end. Experimental results compared with state-of-the-art models on two real-world tongue datasets demonstrate the effectiveness of the proposed TSRNet.

Tongue image segmentation algorithm based on deep convolutional neural network and attention mechanism

The key step in the intelligence of tongue diagnosis is the segmentation of the tongue image, and the accuracy of the segmented edges has a significant impact on the subsequent medical judgment. Deep learning can predict the class of pixel points to achieve pixel-level segmentation of images, so it can be used to handle tongue segmentation tasks. However, different models have different segmentation effects, and they did not learn the connection between space and channels, resulting in inaccurate tongue segmentation. This paper first discussed the choice of model and loss function and then compared the results of different options to find the better model. Associating the red feature of the tongue is very conducive to segmentation as a feature, this paper tested many methods to try to get the color features of the original image to be paid attention to. Finally, this paper proposed an improved Encoder-Decoder network model to solve the problem based on the results. Start with Resnet as the backbone network, then introduce the U-Net model, and then we fused the attention layer, obtained from the source image through convolution and CBAM attention mechanism, and the feature layer obtained from the last upsampling in U-Net. Experimental results show that: The new, improved algorithm results are 2-3 percentage points higher than the popular algorithm, making it more suitable for tongue segmentation tasks.

A survey of artificial intelligence in tongue image for disease diagnosis and syndrome differentiation.

The rapid development of artificial intelligence technology has gradually extended from the general field to all walks of life, and intelligent tongue diagnosis is the product of a miraculous connection between this new discipline and traditional disciplines. We reviewed the deep learning methods and machine learning applied in tongue image analysis that have been studied in the last 5 years, focusing on tongue image calibration, detection, segmentation, and classification of diseases, syndromes, and symptoms/signs. Introducing technical evolutions or emerging technologies were applied in tongue image analysis; as we have noticed, attention mechanism, multiscale features, and prior knowledge were successfully applied in it, and we emphasized the value of combining deep learning with traditional methods. We also pointed out two major problems concerned with data set construction and the low reliability of performance evaluation that exist in this field based on the basic essence of tongue diagnosis in traditional Chinese medicine. Finally, a perspective on the future of intelligent tongue diagnosis was presented; we believe that the self-supervised method, multimodal information fusion, and the study of tongue pathology will have great research significance.

Tongue image segmentation and tongue color classification based on deep learning

ObjectiveTo propose two novel methods based on deep learning for computer-aided tongue diagnosis, including tongue image segmentation and tongue color classification, improving their diagnostic accuracy. MethodsLabelMe was used to label the tongue mask and Snake model to optimize the labeling results. A new dataset was constructed for tongue image segmentation. Tongue color was marked to build a classified dataset for network training. In this research, the Inception + Atrous Spatial Pyramid Pooling (ASPP) + UNet (IAUNet) method was proposed for tongue image segmentation, based on the existing UNet, Inception, and atrous convolution. Moreover, the Tongue Color Classification Net (TCCNet) was constructed with reference to ResNet, Inception, and Triple-Loss. Several important measurement indexes were selected to evaluate and compare the effects of the novel and existing methods for tongue segmentation and tongue color classification. IAUNet was compared with existing mainstream methods such as UNet and DeepLabV3+ for tongue segmentation. TCCNet for tongue color classification was compared with VGG16 and GoogLeNet. ResultsIAUNet can accurately segment the tongue from original images. The results showed that the Mean Intersection over Union (MIoU) of IAUNet reached 96.30%, and its Mean Pixel Accuracy (MPA), mean Average Precision (mAP), F1-Score, G-Score, and Area Under Curve (AUC) reached 97.86%, 99.18%, 96.71%, 96.82%, and 99.71%, respectively, suggesting IAUNet produced better segmentation than other methods, with fewer parameters. Triplet-Loss was applied in the proposed TCCNet to separate different embedded colors. The experiment yielded ideal results, with F1-Score and mAP of the TCCNet reached 88.86% and 93.49%, respectively. ConclusionIAUNet based on deep learning for tongue segmentation is better than traditional ones. IAUNet can not only produce ideal tongue segmentation, but have better effects than those of PSPNet, SegNet, UNet, and DeepLabV3+, the traditional networks. As for tongue color classification, the proposed network, TCCNet, had better F1-Score and mAP values as compared with other neural networks such as VGG16 and GoogLeNet.

A novel tongue segmentation method based on improved U-Net

Background and objectiveAccurate segmentation of tongue image is a prerequisite to intelligent tongue diagnosis. Current tongue segmentation algorithms have good performances in tongue image segmentation in a standard environment. However, tongue segmentation is still challenging to be used in an open environment due to factors such as the complex external open environment and relatively simple equipment. This study aims to construct a novel deep neural network tongue segmentation system suitable for mobile devices, which performs tongue segmentation rapidly in an open environment. MethodsFirstly, a new OET-NET is constructed based on U-Net combined with a residual soft connection module and salient image fusion module to train the tongue image from a different device. Secondly, a new loss function is established based on the Focal loss. Finally, the number of network parameters, the inferred time are treated as indicators to evaluate the model. At the same time, MIoU, precision, recall, F1-Score and FLOPs are used to compare with different tongue segmentation frameworks. ResultsAccording to the training results, the proposed OET-NET’s parameter number is 7.75 MB, the required time of tongue segmentation is about 59 ms/piece, the MIoU is 96.98% and the FLOPs is 15.50 MFLOPs. Compared with U-Net, OET-NET increased the total number of parameters by 0.39 MB and the inference time by 1 ms/piece, but achieved the best segmentation effect compared with the reference models. ConclusionsAccording to less time consumption and less space, the precision of segmentation results is higher than that of other segmentation models. OET-NET can quickly and accurately extract tongue bodies from the open environment, and its relatively smaller number of model parameters made it suitable for mobile devices. It is potential for OET-NET to be applied to tongue image segmentation on mobile terminals.

Tongue Segmentation for Disease Diagnosis a Precise and Fast method using Double U-Net Architecture

Abstract: A robust automatic tongue diagnosis system greatly relies on accurate segmentation of the tongue body from the image. It is a challenge to accurately segment tongue body from an image having close interference of teeth, lips and face. Deep Learning methods such as Deep Convolution Networks like FCN (fully connected Networks), U-NET, Res-Net (Residual Network) have superseded the performance of conventional techniques like Active Contour, Gradient V Flow, Level Set etc. . In this paper looking into the tremendous capability of Deep neural Networks, we have employed Double U-NET for tongue image segmentation of images captured by mobile device and compared the results with that of U-NET, and Res U-NET architectures. Qualitative as well as quantitative analysis of the three reveals a superior performance of Double U-net especially in images with additional dominant features of the face such as lips, teeth, and spaces with the tongue image. Keywords: Double U-Net, Residual U-Net, U-Net, tongue segmentation, automatic tongue diagnosis

Study on TCM Tongue Image Segmentation Model Based on Convolutional Neural Network Fused with Superpixel.

Tongue image segmentation is a base work of TCM tongue processing. Nowadays, deep learning methods are widely used on tongue segmentation, which has better performance than conventional methods. However, when the tongue color is close to the color of the adjoining area, the contour of tongue segmentation by deep learning may be coarse which could influence the subsequent analysis. Here a novel tongue image segmentation model based on a convolutional neural network fused with superpixel was proposed to solve the problem. Methods. On the basis of a convolutional neural network fused with superpixel, the novel tongue image segmentation model SpurNet was proposed in this study. The residual structure of ResNet18 was introduced as the feature extraction layer on the encoding path, to construct the first stage processing module UrNet of SpurNet. The superpixel segmentation was fused with UrNet to form the second stage process of SpurNet. To verify the effect of SpurNet. The models before and after fusion with superpixel, classical image segmentation models FCN and DeepLab were compared with SpurNet on the dataset of 367 manually labeled tongue images. Results. The SpurNet model performance test with 10-fold cross-validation showed PA of 0.9145 ± 0.0043, MPA of 0.9168 ± 0.0048, MIoU of 0.8417 ± 0.0072 and FWIoU of 0.8454 ± 0.0072. Relative to FCN, DeepLab and their superpixel fused models, the SpurNet model was superior in tongue image segmentation and could increase PA by 1.91%–3.17%, MPA by 1.38%–2.61%, MIoU by 3.09%–5.07%, and FWIoU by 3.11%–5.08%. Compared to UrNet, the first stage processing module, the SpurNet model also increased the PA, MPA, MIoU and FWIoU by 0.15%, 0.09%, 0.24% and 0.24%, respectively. Conclusion. The SpurNet model, after fusing with superpixel image segmentation, can better accomplish the task of tongue image segmentation, more accurately process the margins of tongue and resolve the over-segmentation and under-segmentation. The thought of this study is a new exploration in the field of tongue image segmentation, which could provide a reference for the modern research on TCM tongue images.

An improved tongue image segmentation algorithm based on Deeplabv3+ framework

Tongue image segmentation is the key step of traditional Chinese medicine (TCM) intelligent tongue image analysis. The subsequent tongue image quality analysis is directly affected by the precision of segmentation. Deeplabv3+ network has become an excellent algorithm in the field of tongue images segmentation by virtue of its ability to extract multi-scale information and its codec structure. However, there are unclear edge segmentation of the tongue body and missegmentation of small areas in some tongue images. In view of the above phenomenon, an improved algorithm is proposed. Firstly, the network structure is optimized, so that the ability of the network to extract multi-scale information and low-level information is improved. Secondly, a loss function based on edge information is proposed, which makes the network pay more attention to the separation of tongue edges in the process of training. Finally, the segmentation results are post-processed by using the prior knowledge of tongue image, so as to eliminate the phenomenon of misjudgement. The experimental results show that the algorithm significantly improves the ambiguity of image segmentation, and the MIOU value is still increased to 99.13% when the MIOU value has reached 98.77%.

Research and application of tongue and face diagnosis based on deep learning.

ObjectiveTo explore the technical research and application characteristics of deep learning in tongue-facial diagnosis.MethodsThrough summarizing the merits and demerits of current image processing techniques used in the traditional medical tongue and face diagnosis, the research status of deep learning in tongue image preprocessing, segmentation, and classification was analyzed and reviewed, and the algorithm was compared and verified with the real tongue and face image. Images of the face and tongue used for diagnosis in conventional medicine were systematically reviewed, from acquisition and pre-processing to segmentation, classification, algorithm comparison, result from analysis, and application.ResultsDeep learning improved the speed and accuracy of tongue and face diagnostic image data processing. Among them, the average intersection ratio of U-net and Seg-net models exceeded 0.98, and the segmentation speed ranged from 54 to 58 ms.ConclusionThere is no unified standard for lingual-facial diagnosis objectification in terms of image acquisition conditions and image processing methods, thus further research is indispensable. It is feasible to use the images acquired by mobile in the field of medical image analysis by reducing the influence of environmental and other factors on the quality of lingual-facial diagnosis images and improving the efficiency of image processing.

An Efficient Tongue Segmentation Model Based on U-Net Framework

In the intelligently processing of the tongue image, one of the most important tasks is to accurately segment the tongue body from a whole tongue image, and the good quality of tongue body edge processing is of great significance for the relevant tongue feature extraction. To improve the performance of the segmentation model for tongue images, we propose an efficient tongue segmentation model based on U-Net. Three important studies are launched, including optimizing the model’s main network, innovating a new network to specially handle tongue edge cutting and proposing a weighted binary cross-entropy loss function. The purpose of optimizing the tongue image main segmentation network is to make the model recognize the foreground and background features for the tongue image as well as possible. A novel tongue edge segmentation network is used to focus on handling the tongue edge because the edge of the tongue contains a number of important information. Furthermore, the advantageous loss function proposed is to be adopted to enhance the pixel supervision corresponding to tongue images. Moreover, thanks to a lack of tongue image resources on Traditional Chinese Medicine (TCM), some special measures are adopted to augment training samples. Various comparing experiments on two datasets were conducted to verify the performance of the segmentation model. The experimental results indicate that the loss rate of our model converges faster than the others. It is proved that our model has better stability and robustness of segmentation for tongue image from poor environment. The experimental results also indicate that our model outperforms the state-of-the-art ones in aspects of the two most important tongue image segmentation indexes: IoU and Dice. Moreover, experimental results on augmentation samples demonstrate our model have better performances.

Automatic Segmentation of Tongue Image Based on Res-Unet Approach

Tongue diagnosis occupies an important position in the field of traditional Chinese medicine and has been developed for thousands of years. Doctors diagnose disease based on tongue images of patients stored in hospital databases. Hence, segmenting the tongue area of the tongue image facilitates the diagnosis and saves space for storing the tongue image. In order to solve such a challenging problem, we put forward a method combing Unet and Res-net for tongue image segmentation and implements the end-to-end form. In our Res-Unet architecture, including four encoder blocks and four decoder blocks, and the residual network (Res-net) block used as the backbone for each block. The upsampling layer restores the features extracted by the sampling layer. We use our own datasets named TongueSet1 (TS1) and Tongueset2 (TS2) that collected from the hospital. The collection methods of these two datasets are different; TS1 is collected by professionals while TS2 is taken by nurses. This method obtained the latest results on both data sets. We used accuracy (acc) and mean intersection (mIoU) as the evaluation indicators of our model. Among them, the acc and mIoU of the model on TongueSet1 reached 0.984 and 0.925, on TongueSet2 reached 0.985 and 0.925.

Automatic Segmentation of Color Tongue Image Using Deep Asymmetric Convolution Skip Net

Segmentation of the tongue body from color images is vital for tongue diagnoses in traditional Chinese medicine. In tongue images, the tongue body is easily confused with the skin and lips, and the shadow also causes incorrect segmentation. To address these issues, we proposed a novel ACS-Net for tongue image segmentation and implemented the end-to-end form. In our ACS-Net architecture, the following innovations proposed: (1) ordinary convolution was replaced with ACB Module, (2) decoder block restores the features extracted by the encoder block, (3) skip connections are implemented between and within blocks. We use our own datasets named S1 and S2 that collected from the partner hospital. The collection methods of these two datasets are different: S1 was collected by professionals while S2 was taken by nurses. The method achieved state-of-the-art results on both two datasets, we use two metrics to reflect the segmentation performance, which are accuracy (acc) and mean intersection over Union (mIoU), in which the acc reaches 0.984 on S1 and 0.981 on S2; the mIoU reaches 0.925 on S1 and 0.958 on S2.

Reconstruction enhanced probabilistic model for semisupervised tongue image segmentation

SummaryTongue segmentation is a key step of automatic tongue diagnosis, and the major challenges for the effective segmentation lie in the large appearance variations of tongue caused by different diseases, for example, tongue coating and tongue texture. Moreover, the limited labeled data also hinders traditional supervised methods from their powerful learning ability. To alleviate these challenges, in this work, we propose a reconstruction enhanced probabilistic model for semisupervised tongue segmentation, named SemiTongue, in which, image reconstruction constraint combined with adversarial learning is used to improve the accuracy of tongue segmentation. Specifically, based on a shared feature encoder that served as an inference model, two separate branches in SemiTongue as the generative model, which are composed of a segmentation decoder and a reconstruction decoder, are utilized to generate the tongue segmentation and reconstruct original tongue image respectively. Then, a discriminator is employed to differentiate the generated segmentation map from the ground truth segmentation distribution. Moreover, semisupervised learning is conducted through discriminator by discovering the reliable region in the generated segmentation map of unlabeled images, which is further utilized to supervise the segmentation branch. Experimental results compared with state‐of‐the‐art methods on real‐world datasets demonstrate the effectiveness of SemiTongue.

TISNet-Enhanced Fully Convolutional Network with Encoder-Decoder Structure for Tongue Image Segmentation in Traditional Chinese Medicine.

Extracting the tongue body accurately from a digital tongue image is a challenge for automated tongue diagnoses, as the blurred edge of the tongue body, interference of pathological details, and the huge difference in the size and shape of the tongue. In this study, an automated tongue image segmentation method using enhanced fully convolutional network with encoder-decoder structure was presented. In the frame of the proposed network, the deep residual network was adopted as an encoder to obtain dense feature maps, and a Receptive Field Block was assembled behind the encoder. Receptive Field Block can capture adequate global contextual prior because of its structure of the multibranch convolution layers with varying kernels. Moreover, the Feature Pyramid Network was used as a decoder to fuse multiscale feature maps for gathering sufficient positional information to recover the clear contour of the tongue body. The quantitative evaluation of the segmentation results of 300 tongue images from the SIPL-tongue dataset showed that the average Hausdorff Distance, average Symmetric Mean Absolute Surface Distance, average Dice Similarity Coefficient, average precision, average sensitivity, and average specificity were 11.2963, 3.4737, 97.26%, 95.66%, 98.97%, and 98.68%, respectively. The proposed method achieved the best performance compared with the other four deep-learning-based segmentation methods (including SegNet, FCN, PSPNet, and DeepLab v3+). There were also similar results on the HIT-tongue dataset. The experimental results demonstrated that the proposed method can achieve accurate tongue image segmentation and meet the practical requirements of automated tongue diagnoses.