Tongue Image Research Articles

A Comprehensive Approach to Vitamin Deficiency Detection through Image Analysis of Skin, Tongue, Eyes and Nail Images using Convolutional Neural Networks

This research presents a free artificial intelligence-based smartphone application designed to detect vitamin deficiencies among individuals by utilising images of specific parts of the body. Current approaches for detecting vitamin deficiencies requires an expensive laboratory analysis. Several vitamin deficiencies can exhibit one or more easily identified signs and indicators that manifest in various parts of the human body. Through the examination of images of their eyes, lips, tongue, and nails, users of the application can determine whether they may be lacking in any vitamin without having to give blood samples. Using nutritional micro-correction, the program then suggests a list of sources of nutrients for tackling the identified deficiency and its expected consequences. Through collecting and validating visual data of individuals, medical professionals can also help the platform improve its detection and accuracy capabilities. Allowing more advanced picture analysis and feature extraction skills that could eventually outperform human medical condition diagnosis. In addition to helping individuals solve a global issue that affects millions of people due to a lack of nutritional understanding, our software will eventually assist medical professionals in making more accurate diagnoses

Research on the model of automatic recognition and natural language question-answer system for traditional Chinese medicine tongue images based on LLMs

Large Language Models (LLMs) have recently demonstrated their potential in clinical applications by providing valuable medical knowledge and recommendations. Traditional Chinese tongue diagnosis, one of the "Four Diagnoses," is an essential method for traditional Chinese medicine diagnosis. This paper builds upon tongue image classification technology and utilizes natural language processing and image recognition techniques to enhance the discrimination and analysis of traditional Chinese tongue images through learning and inference. We propose a method to integrate LLMs into the tongue image automatic recognition model and use them in interactive question-answering by summarizing and reorganizing information in natural language text format.

Enhancing deep learning classification performance of tongue lesions in imbalanced data: mosaic-based soft labeling with curriculum learning

BackgroundOral potentially malignant disorders (OPMDs) are associated with an increased risk of cancer of the oral cavity including the tongue. The early detection of oral cavity cancers and OPMDs is critical for reducing cancer-specific morbidity and mortality. Recently, there have been studies to apply the rapidly advancing technology of deep learning for diagnosing oral cavity cancer and OPMDs. However, several challenging issues such as class imbalance must be resolved to effectively train a deep learning model for medical imaging classification tasks. The aim of this study is to evaluate a new technique of artificial intelligence to improve the classification performance in an imbalanced tongue lesion dataset.MethodsA total of 1,810 tongue images were used for the classification. The class-imbalanced dataset consisted of 372 instances of cancer, 141 instances of OPMDs, and 1,297 instances of noncancerous lesions. The EfficientNet model was used as the feature extraction model for classification. Mosaic data augmentation, soft labeling, and curriculum learning (CL) were employed to improve the classification performance of the convolutional neural network.ResultsUtilizing a mosaic-augmented dataset in conjunction with CL, the final model achieved an accuracy rate of 0.9444, surpassing conventional oversampling and weight balancing methods. The relative precision improvement rate for the minority class OPMD was 21.2%, while the relative F1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${F}_{1}$$\\end{document} score improvement rate of OPMD was 4.9%.ConclusionsThe present study demonstrates that the integration of mosaic-based soft labeling and curriculum learning improves the classification performance of tongue lesions compared to previous methods, establishing a foundation for future research on effectively learning from imbalanced data.

Tongue position in Mandarin Chinese voiceless stops.

The current study explores whether Mandarin initial and medial voiceless unaspirated and voiceless aspirated stops differ in their tongue positions and post-vocalic voicing during closure. Ultrasound tongue imaging and acoustic data from five Mandarin speakers revealed (1) no consistent pattern for tongue positions among speakers, and (2) no difference in degree of voicing during closure between the two stop series. These findings suggest that tongue position is not a reliable articulatory correlate for Mandarin laryngeal contrasts. This further suggests that aspiration is not correlated with tongue position differences, unlike the reported correlation between voicing and tongue root advancement.

CTM Tongue Image Consulting System based on Deep Learning Technology

Internet is an important development step in information times. With the tide of Internet development, internet plus health will become a trend of new times. The Chinese Traditional Medicine (CTM) tongue image consulting system based on deep learning technology has created a more perfect and intelligent personal health management system by connecting smart devices to mobile platforms. The system will serve customers perfectly through health food therapy, medical consultation, etc.

Two-stream vision transformer based multi-label recognition for TCM prescriptions construction

Traditional Chinese medicine (TCM) observation diagnosis images (including facial and tongue images) provide essential human body information, holding significant importance in clinical medicine for diagnosis and treatment. TCM prescriptions, known for their simplicity, non-invasiveness, and low side effects, have been widely applied worldwide. Exploring automated herbal prescription construction based on visual diagnosis holds vital value in delving into the correlation between external features and herbal prescriptions and offering medical services in mobile healthcare systems. To effectively integrate multi-perspective visual diagnosis images and automate prescription construction, this study proposes a multi-herb recommendation framework based on Visual Transformer and multi-label classification. The framework comprises three key components: image encoder, label embedding module, and cross-modal fusion classification module. The image encoder employs a dual-stream Visual Transformer to learn dependencies between different regions of input images, capturing both local and global features. The label embedding module utilizes Graph Convolutional Networks to capture associations between diverse herbal labels. Finally, two Multi-Modal Factorized Bilinear modules are introduced as effective components to fuse cross-modal vectors, creating an end-to-end multi-label image-herb recommendation model. Through experimentation with real facial and tongue images and generating prescription data closely resembling real samples. The precision is 50.06 %, the recall rate is 48.33 %, and the F1-score is 49.18 %. This study validates the feasibility of automated herbal prescription construction from the perspective of visual diagnosis. Simultaneously, it provides valuable insights for constructing herbal prescriptions automatically from more physical information.

Accurate estimation of biological age and its application in disease prediction using a multimodal image Transformer system.

Aging in an individual refers to the temporal change, mostly decline, in the body's ability to meet physiological demands. Biological age (BA) is a biomarker of chronological aging and can be used to stratify populations to predict certain age-related chronic diseases. BA can be predicted from biomedical features such as brain MRI, retinal, or facial images, but the inherent heterogeneity in the aging process limits the usefulness of BA predicted from individual body systems. In this paper, we developed a multimodal Transformer-based architecture with cross-attention which was able to combine facial, tongue, and retinal images to estimate BA. We trained our model using facial, tongue, and retinal images from 11,223 healthy subjects and demonstrated that using a fusion of the three image modalities achieved the most accurate BA predictions. We validated our approach on a test population of 2,840 individuals with six chronic diseases and obtained significant difference between chronological age and BA (AgeDiff) than that of healthy subjects. We showed that AgeDiff has the potential to be utilized as a standalone biomarker or conjunctively alongside other known factors for risk stratification and progression prediction of chronic diseases. Our results therefore highlight the feasibility of using multimodal images to estimate and interrogate the aging process.

In-depth metaproteomics analysis of tongue coating for gastric cancer: a multicenter diagnostic research study

BackgroundOur previous study revealed marked differences in tongue images between individuals with gastric cancer and those without gastric cancer. However, the biological mechanism of tongue images as a disease indicator remains unclear. Tongue coating, a major factor in tongue appearance, is the visible layer on the tongue dorsum that provides a vital environment for oral microorganisms. While oral microorganisms are associated with gastric and intestinal diseases, the comprehensive function profiles of oral microbiota remain incompletely understood. Metaproteomics has unique strength in revealing functional profiles of microbiota that aid in comprehending the mechanism behind specific tongue coating formation and its role as an indicator of gastric cancer.MethodsWe employed pressure cycling technology and data-independent acquisition (PCT-DIA) mass spectrometry to extract and identify tongue-coating proteins from 180 gastric cancer patients and 185 non-gastric cancer patients across 5 independent research centers in China. Additionally, we investigated the temporal stability of tongue-coating proteins based on a time-series cohort. Finally, we constructed a machine learning model using the stochastic gradient boosting algorithm to identify individuals at high risk of gastric cancer based on tongue-coating microbial proteins.ResultsWe measured 1432 human-derived proteins and 13,780 microbial proteins from 345 tongue-coating samples. The abundance of tongue-coating proteins exhibited high temporal stability within an individual. Notably, we observed the downregulation of human keratins KRT2 and KRT9 on the tongue surface, as well as the downregulation of ABC transporter COG1136 in microbiota, in gastric cancer patients. This suggests a decline in the defense capacity of the lingual mucosa. Finally, we established a machine learning model that employs 50 microbial proteins of tongue coating to identify individuals at a high risk of gastric cancer, achieving an area under the curve (AUC) of 0.91 in the independent validation cohort.ConclusionsWe characterized the alterations in tongue-coating proteins among gastric cancer patients and constructed a gastric cancer screening model based on microbial-derived tongue-coating proteins. Tongue-coating proteins are shown as a promising indicator for identifying high-risk groups for gastric cancer.1cCPG45EGUX3nZjXKAFgvfVideo

Machine learning aided non-invasive diagnosis of coronary heart disease based on tongue features fusion.

Coronary heart disease (CHD) is the first cause of death globally. Hypertension is considered to be the most important independent risk factor for CHD. Early and accurate diagnosis of CHD in patients with hypertension can plays a significant role in reducing the risk and harm of hypertension combined with CHD. To propose a non-invasive method for early diagnosis of coronary heart disease according to tongue image features with the help of machine learning techniques. We collected standard tongue images and extract features by Diagnosis Analysis System (TDAS) and ResNet-50. On the basis of these tongue features, a common machine learning method is used to customize the non-invasive CHD diagnosis algorithm based on tongue image. Based on feature fusion, our algorithm has good performance. The results showed that the XGBoost model with fused features had the best performance with accuracy of 0.869, the AUC of 0.957, the AUPR of 0.961, the precision of 0.926, the recall of 0.806, and the F1-score of 0.862. We provide a feasible, convenient, and non-invasive method for the diagnosis and large-scale screening of CHD. Tongue image information is a possible effective marker for the diagnosis of CHD.

Feasibility of tongue image detection for coronary artery disease: based on deep learning.

Clarify the potential diagnostic value of tongue images for coronary artery disease (CAD), develop a CAD diagnostic model that enhances performance by incorporating tongue image inputs, and provide more reliable evidence for the clinical diagnosis of CAD, offering new biological characterization evidence. We recruited 684 patients from four hospitals in China for a cross-sectional study, collecting their baseline information and standardized tongue images to train and validate our CAD diagnostic algorithm. We used DeepLabV3 + for segmentation of the tongue body and employed Resnet-18, pretrained on ImageNet, to extract features from the tongue images. We applied DT (Decision Trees), RF (Random Forest), LR (Logistic Regression), SVM (Support Vector Machine), and XGBoost models, developing CAD diagnostic models with inputs of risk factors alone and then with the additional inclusion of tongue image features. We compared the diagnostic performance of different algorithms using accuracy, precision, recall, F1-score, AUPR, and AUC. We classified patients with CAD using tongue images and found that this classification criterion was effective (ACC = 0.670, AUC = 0.690, Recall = 0.666). After comparing algorithms such as Decision Tree (DT), Random Forest (RF), Logistic Regression (LR), Support Vector Machine (SVM), and XGBoost, we ultimately chose XGBoost to develop the CAD diagnosis algorithm. The performance of the CAD diagnosis algorithm developed solely based on risk factors was ACC = 0.730, Precision = 0.811, AUC = 0.763. When tongue features were integrated, the performance of the CAD diagnosis algorithm improved to ACC = 0.760, Precision = 0.773, AUC = 0.786, Recall = 0.850, indicating an enhancement in performance. The use of tongue images in the diagnosis of CAD is feasible, and the inclusion of these features can enhance the performance of existing CAD diagnosis algorithms. We have customized this novel CAD diagnosis algorithm, which offers the advantages of being noninvasive, simple, and cost-effective. It is suitable for large-scale screening of CAD among hypertensive populations. Tongue image features may emerge as potential biomarkers and new risk indicators for CAD.

Using Advanced Convolutional Neural Network Approaches to Reveal Patient Age, Gender, and Weight Based on Tongue Images.

The human tongue has been long believed to be a window to provide important insights into a patient's health in medicine. The present study introduced a novel approach to predict patient age, gender, and weight inferences based on tongue images using pretrained deep convolutional neural networks (CNNs). Our results demonstrated that the deep CNN models (e.g., ResNeXt) trained on dorsal tongue images produced excellent results for age prediction with a Pearson correlation coefficient of 0.71 and a mean absolute error (MAE) of 8.5 years. We also obtained an excellent classification of gender, with a mean accuracy of 80% and an AUC (area under the receiver operating characteristic curve) of 88%. ResNeXt model also obtained a moderate level of accuracy for weight prediction, with a Pearson correlation coefficient of 0.39 and a MAE of 9.06 kg. These findings support our hypothesis that the human tongue contains crucial information about a patient. This study demonstrated the feasibility of using the pretrained deep CNNs along with a large tongue image dataset to develop computational models to predict patient medical conditions for noninvasive, convenient, and inexpensive patient health monitoring and diagnosis.

RTC_TongueNet: An improved tongue image segmentation model based on DeepLabV3.

Tongue segmentation as a basis for automated tongue recognition studies in Chinese medicine, which has defects such as network degradation and inability to obtain global features, which seriously affects the segmentation effect. This article proposes an improved model RTC_TongueNet based on DeepLabV3, which combines the improved residual structure and transformer and integrates the ECA (Efficient Channel Attention Module) attention mechanism of multiscale atrous convolution to improve the effect of tongue image segmentation. In this paper, we improve the backbone network based on DeepLabV3 by incorporating the transformer structure and an improved residual structure. The residual module is divided into two structures and uses different residual structures under different conditions to speed up the frequency of shallow information mapping to deep network, which can more effectively extract the underlying features of tongue image; introduces ECA attention mechanism after concat operation in ASPP (Atrous Spatial Pyramid Pooling) structure to strengthen information interaction and fusion, effectively extract local and global features, and enable the model to focus more on difficult-to-separate areas such as tongue edge, to obtain better segmentation effect. The RTC_TongueNet network model was compared with FCN (Fully Convolutional Networks), UNet, LRASPP (Lite Reduced ASPP), and DeepLabV3 models on two datasets. On the two datasets, the MIOU (Mean Intersection over Union) and MPA (Mean Pixel Accuracy) values of the classic model DeepLabV3 were higher than those of FCN, UNet, and LRASPP models, and the performance was better. Compared with the DeepLabV3 model, the RTC_TongueNet network model increased MIOU value by 0.9% and MPA value by 0.3% on the first dataset; MIOU increased by 1.0% and MPA increased by 1.1% on the second dataset. RTC_TongueNet model performed best on both datasets. In this study, based on DeepLabV3, we apply the improved residual structure and transformer as a backbone to fully extract image features locally and globally. The ECA attention module is combined to enhance channel attention, strengthen useful information, and weaken the interference of useless information. RTC_TongueNet model can effectively segment tongue images. This study has practical application value and reference value for tongue image segmentation.

Tongue color parameters in predicting the degree of coronary stenosis: a retrospective cohort study of 282 patients with coronary angiography.

This retrospective cohort study aimed to analyze the relationship between tongue color and coronary artery stenosis severity in 282 patients after underwent coronary angiography. A retrospective cohort study was conducted to collect data from patients who underwent coronary angiography in the Department of Cardiology, Shanghai Jiading District Central Hospital from October 1, 2023 to January 15, 2024. All patients were divided into four various stenosis groups. The tongue images of each patient was normalized captured, tongue body (TC_) and tongue coating (CC_) data were converted into RGB and HSV model parameters using SMX System 2.0. Four supervised machine learning classifiers were used to establish a coronary artery stenosis grading prediction model, including random forest (RF), logistic regression, and support vector machine (SVM). Accuracy, precision, recall, and F1 score were used as classification indicators to evaluate the training and validation performance of the model. SHAP values were furthermore used to explore the impacts of features. This study finally included 282 patients, including 164 males (58.16%) and 118 females (41.84%). 69 patients without stenosis, 70 patients with mild stenosis, 65 patients with moderate stenosis, and 78 patients with severe stenosis. Significant differences of tongue parameters were observed in the four groups [TC_R (P = 0.000), TC_G (P = 0.003), TC_H (P = 0.001) and TC_S (P = 0.024),CC_R (P = 0.006), CC_B (P = 0.023) and CC_S (P = 0.001)]. The SVM model had the highest predictive ability, with AUC values above 0.9 in different stenosis groups, and was particularly good at identifying mild and severe stenosis (AUC = 0.98). SHAP value showed that high values of TC_RIGHT_R, low values of CC_LEFT_R were the most impact factors to predict no coronary stenosis; high CC_LEFT_R and low TC_ROOT_H for mild coronary stenosis; low TC_ROOT_R and CC_ROOT_B for moderate coronary stenosis; high CC_RIGHT_G and low TC_ROOT_H for severe coronary stenosis. Tongue color parameters can provide a reference for predicting the degree of coronary artery stenosis. The study provides insights into the potential application of tongue color parameters in predicting coronary artery stenosis severity. Future research can expand on tongue features, optimize prediction models, and explore applications in other cardiovascular diseases.

DeepDSP: Deep Convolutional Network with Double Spatial Pyramid for tongue image segmentation

DeepDSP: Deep Convolutional Network with Double Spatial Pyramid for tongue image segmentation

HPA-UNet: A Hybrid Post-Processing Attention U-Net for Tongue Segmentation.

Tongue diagnosis is the kernel method of Traditional Chinese Medicine (TCM), and it has been proved that the condition of the tongue can serve as an indicator of a person's health status. To automatically recognize a person's latent diseases by computer vision technology, getting the tongue segmentation from a picture with high precision has significant importance. However, the precision of tongue segmentation images in most prior methods is not satisfactory, which will inevitably result in misjudging. In this paper, an effective method is proposed for highly precise tongue segmentation, which is combined with an improved U-shaped neural network and an edge refinement post-processing method. The contributions are three-fold. First, a carefully designed data augmentation strategy is imported to prevent the network from over-fitting. Second, an updated U-shaped neural network is designed to segment tongue images with high precision. Third, a post-processing method is imported to refine the edge of the tongue segmentation further. The proposed method achieves competitive performance in almost all experiments on two datasets. Furthermore, the proposed post-processing method can effectively improve all classic neural networks in tongue segmentation, which strongly proves the flexibility and generalization of the proposed method.

Audio-Only Phonetic Segment Classification Using Embeddings Learned From Audio and Ultrasound Tongue Imaging Data

Audio-Only Phonetic Segment Classification Using Embeddings Learned From Audio and Ultrasound Tongue Imaging Data

Deep-DSP: deep convolutional network with double spatial pyramid for tongue image segmentation

Deep-DSP: deep convolutional network with double spatial pyramid for tongue image segmentation

Semi-supervised tongue image segmentation method for traditional chinese medicine based on mutual learning with dual models

Semi-supervised tongue image segmentation method for traditional chinese medicine based on mutual learning with dual models

Observation of Tongue Form at Rest and during Articulation by Using a Tongue Model Constructed from Ultrasonic Tongue Images

Observation of Tongue Form at Rest and during Articulation by Using a Tongue Model Constructed from Ultrasonic Tongue Images

Evaluating deep learning techniques for identifying tongue features in subthreshold depression: a prospective observational study.

This study aims to evaluate the potential of using tongue image features as non-invasive biomarkers for diagnosing subthreshold depression and to assess the correlation between these features and acupuncture treatment outcomes using advanced deep learning models. We employed five advanced deep learning models-DenseNet169, MobileNetV3Small, SEResNet101, SqueezeNet, and VGG19_bn-to analyze tongue image features in individuals with subthreshold depression. These models were assessed based on accuracy, precision, recall, and F1 score. Additionally, we investigated the relationship between the best-performing model's predictions and the success of acupuncture treatment using Pearson's correlation coefficient. Among the models, SEResNet101 emerged as the most effective, achieving an impressive 98.5% accuracy and an F1 score of 0.97. A significant positive correlation was found between its predictions and the alleviation of depressive symptoms following acupuncture (Pearson's correlation coefficient = 0.72, p<0.001). The findings suggest that the SEResNet101 model is highly accurate and reliable for identifying tongue image features in subthreshold depression. It also appears promising for assessing the impact of acupuncture treatment. This study contributes novel insights and approaches to the auxiliary diagnosis and treatment evaluation of subthreshold depression.