Tongue Image Research Articles

Tonguenet: Accurate Localization and Segmentation for Tongue Images Using Deep Neural Networks

Tongue diagnosis is an important way of monitoring human health status in traditional Chinese medicine. As a key step of achieving automatic tongue diagnosis, the major challenges for robust and accurate segmentation and identification of tongue body in tongue images lay in the large variations of tongue appearance, e.g., tongue texture and tongue coating, caused by different diseases for different patients. To cope with these challenges, we propose a novel end-to-end model for multi-task learning of tongue localization and segmentation, named TongueNet, in which pixel-level prior information is utilized for supervised training of deep convolutional neural network. Firstly, we introduce a feature pyramid network based on the designed context-aware residual blocks for the extraction of multi-scale tongue features. Then, the region of interests (ROIs) of tongue candidates are located in advance from the extracted feature maps. Finally, finer localization and segmentation of tongue body are conducted based on the feature maps of ROIs. Quantitative and qualitative comparisons on real-world datasets show that the proposed TongueNet achieves state-of-the-art performance for the segmentation of tongue body in terms of both robustness and accuracy.

Tongue Print Features Extraction By Gabor Filters Family

Nowadays, with the fast evolution of technology, information has been transmitted over public networks and the internet which made the important information need to be secure. Many security applications or techniques are provided secure communication over public networks. One of these techniques is the tongue print. The important task in computer vision and object recognition is Efficient and effective image extraction ROI. The tongue body Extraction from digital images is important for automated tongue identifies. Before the feature extraction process, is needed to extract the tongue from the image, so it is possible to develop an application that can extract the tongue image from an opened mouth image. This research presented a survey on the tongue image segmentation technologies by Double Geo-vector flow (DGF). Also, this paper hasproposed a method of tongue segmentation by extract ROI by using a specific window with fixed size and then the resulted segment region (i.e., ROI) will be used to extract the features in step of extraction features. Gabor filters family is used to extract feature and the results will be obtained from this technique.

Comparing tongue image data with clinical information to find the correlation of clinical indices between Kampo medicine and modern medicine

Comparing tongue image data with clinical information to find the correlation of clinical indices between Kampo medicine and modern medicine

Light and scanning electron microscopic study of lingual papillae in the wolf (Canis lupus).

In this study, it was aimed to perform light microscopy and scanning electron microscopy (SEM) investigation of the tongue and papilla belonging to two wolves. Light microscopy and SEM images of tongues were taken. It was observed that papillae filiformes concentrated in apex of these investigated tongues. In addition to, these papillae were observed in the whole tongues. It was also determined that papillae fungiformes were distributed rarely in between papillae filiformes. Papillae foliatae were placed in the lateral side of tongues. There were two papillae vallatae in the median part of tongues. Papillae vallatae was determined in radix linguae. It was observed that papillae vallatae formed circular extensions in many different dimensions and a hole was present in a circular structure in the center. Papillae conicae were seen on dorsal surface of radix linguae. Papillae foliatae were seen four laminal structure and placed on the each lateral side of tongues. To the best of our knowledge, this is the first study that presents of light microscopy and SEM findings related with the tongues of wolves.

Hybrid Multilevel Thresholding and Improved Harmony Search Algorithm for Segmentation

This paper proposes a new method for image segmentation is hybrid multilevel thresholding and improved harmony search algorithm. Improved harmony search algorithm which is a method for finding vector solutions by increasing its accuracy. The proposed method looks for a random candidate solution, then its quality is evaluated through the Otsu objective function. Furthermore, the operator continues to evolve the solution candidate circuit until the optimal solution is found. The dataset used in this study is the retina dataset, tongue, lenna, baboon, and cameraman. The experimental results show that this method produces the high performance as seen from peak signal-to-noise ratio analysis (PNSR). The PNSR result for retinal image averaged 40.342 dB while for the average tongue image 35.340 dB. For lenna, baboon and cameramen produce an average of 33.781 dB, 33.499 dB, and 34.869 dB. Furthermore, the process of object recognition and identification is expected to use this method to produce a high degree of accuracy.

초음파 혀 영상 분석을 통한 우리말 모음의 조음 특징 조사

초음파 혀 영상 분석을 통한 우리말 모음의 조음 특징 조사

On the development of gestural organization: A cross-sectional study of vowel-to-vowel anticipatory coarticulation.

In the first years of life, children differ greatly from adults in the temporal organization of their speech gestures in fluent language production. However, dissent remains as to the maturational direction of such organization. The present study sheds new light on this process by tracking the development of anticipatory vowel-to-vowel coarticulation in a cross-sectional investigation of 62 German children (from 3.5 to 7 years of age) and 13 adults. It focuses on gestures of the tongue, a complex organ whose spatiotemporal control is indispensable for speech production. The goal of the study was threefold: 1) investigate whether children as well as adults initiate the articulation for a target vowel in advance of its acoustic onset, 2) test if the identity of the intervocalic consonant matters and finally, 3) describe age-related developments of these lingual coarticulatory patterns. To achieve this goal, ultrasound tongue imaging was used to record lingual movements and quantify changes in coarticulation degree as a function of consonantal context and age. Results from linear mixed effects models indicate that like adults, children initiate vowels’ lingual gestures well ahead of their acoustic onset. Second, while the identity of the intervocalic consonant affects the degree of vocalic anticipation in adults, it does not in children at any age. Finally, the degree of vowel-to-vowel coarticulation is significantly higher in all cohorts of children than in adults. However, among children, a developmental decrease of vocalic coarticulation is only found for sequences including the alveolar stop /d/ which requires finer spatiotemporal coordination of the tongue’s subparts compared to labial and velar stops. Altogether, results suggest greater gestural overlap in child than in adult speech and support the view of a non-uniform and protracted maturation of lingual coarticulation calling for thorough considerations of the articulatory intricacies from which subtle developmental differences may originate.

Tongue image segmentation via color decomposition and thresholding

SummaryTongue image segmentation is a crucial step in developing an automatic tongue diagnosis system. After exploring characteristics of image thresholding in different color spaces, we propose a simple and effective tongue image segmentation method. Specifically, the proposed method first extracts an initial tongue body region by performing image thresholding on the transformed hue component in HSI (Hue, Saturation, and Intensity) color space. Then, one of two image thresholding results on the red component of an original tongue image is adaptively selected to find the gap region between the tongue body root and the upper lip. Finally, the initial tongue body region is refined by removing fake object regions such as the upper lip to obtain final tongue image segmentation result. Experimental results on a tongue image dataset containing 200 subjects demonstrate the effectiveness of the proposed method.

Articulatory feature extraction from ultrasound images using pretrained convolutional neural networks

Feature extraction is of great importance to ultrasound tongue image analysis. Inspired by the recent success of deep learning, we explore a novel approach to feature extraction from ultrasound tongue images using pre-trained convolutional neural networks (CNN). The bottleneck features from different pre-trained CNNs, including VGGNet and ResNet, are used as representations of the ultrasound tongue images. Then an image classification task is conducted to assess the effectiveness of CNN-based features. Our dataset consists of 20,000 ultrasound tongue images collected from a female speaker of Mandarin Chinese, which were manually labeled as containing one of the following consonants: /p, t, k, l/. Experiment results show that the Gradient Boost Machines (GBM) classifiers trained on the CNN-based features achieve the best performance, with a classification accuracy of 92.4% for ResNet and 91.6% for VGGNet, outperforming the benchmark GBM classifier trained on the features extracted using Principal Component Analysis (PCA), which only achieves an accuracy of 87.5%. In this preliminary dataset, our method of feature extraction is found to be superior to the PCA-based method. This work demonstrates the potential of applying the pre-trained convolutional neural networks to ultrasound tongue image analysis task.

An ultrasound investigation of the speech motor skills of stuttering Italian children

ABSTRACT This study aims at evaluating speech motor skills in the fluent speech of a cohort of stuttering Italian children. Stuttering is a neurodevelopmental disorder that may arise from an innate limitation of the speech motor control system, which fails to prepare and organize the movements required for fluent speech (Van Lieshout, Hulstijn, & Peters, 2004). Anticipatory coarticulation in CV sequences and stability of speech movements have been used as measures of the maturity of articulatory processes for fluent speech production. This study aims to assess if direct measures of speech dynamics can identify impaired mechanisms in stuttering speech during a phrase-repetition task. The Ultrasound Tongue Imaging data of eight school-aged children, half of whom stutter while the other half don’t, show different articulatory patterns between the two groups, for both motor aspects under investigation. Articulatory data show that the stuttering group presents a higher degree of intra-syllabic coarticulation compared to the control group and decreased stability (i.e. more variability) through multiple repetitions of the same alveolar and velar item. Outcomes of this study suggest that the speech motor control system of children who stutter is less mature in preparing and executing the speech gestures required for fluent speech. This study contributes to shedding light on the impaired articulatory patterns involved in stuttering speech and to identifying the diagnostic markers of the disorder by evaluating the speech of children close to the onset of stuttering.

Using Traditional Chinese Medicine Ideas as a Mechanism to Engage People in Health Awareness

Improving health awareness is essential to health and healthcare sustainability. How to arouse attention to the health of people and encourage them to attend to healthcare progress so that we can reduce the costs of promoting healthcare by achieving more with less effort remains to be explored. In this paper, we provide a simplified health management app, called iTongue, with a basis in traditional Chinese medicine. People use iTongue to take pictures of their tongues to have a general idea of their health. We realize automated tongue image diagnosis using machine learning techniques to establish the relationship between the tongue image features and the cold or hot ZHENG (traditional Chinese medicine syndrome) in traditional Chinese medicine by learning through examples and assisting people to engage in health management. The results show that health management interaction based on traditional Chinese medicine has a positive influence on improving people’s attention to their health, encouraging them to participate in health management activities and develop the habit of caring about their health over the long term. In the future, we could consider using this kind of traditional Chinese medicine idea as a means of publicity to engage people in healthcare and to assist healthcare sustainability development.

Natural tongue physique identification using hybrid deep learning methods

Traditional Chinese Medicine (TCM) illustrates that the physique determines the susceptibility of human to certain diseases and treatment programs for illness. Tongue diagnosis is an important way to identify the physique, but now it is performed by the doctor’s professional experience and the design of a questionnaire. Consequently, accurate physique identification cannot be obtained easily. In this paper, we propose a new method to identify the physique through wild tongue images using hybrid deep learning methods. It begins with constructing a large number of tongue images that are taken in natural conditions, instead of in a controlled environment. Based on the resulting database, a new method of tongue coating detection is put forward that applies a rapid deep learning method to complete the initial tongue coating detection, and then utilizes another deep learning method, a calibration neural network, to further improve the accuracy of tongue detection. Finally, an effective deep learning method is applied to identify the tongue physique. Experiments validate the proposed method, illustrating that physique identification can be performed well using hybrid deep learning methods.

Extraction of Tongue Crack Based on Gray Level and Texture

Tongue diagnosis is one of the most important content of Traditional Chinese Medicine (TCM). While the tongue crack is an important index of tongue diagnosis, there are just a few existing researches of tongue crack extraction and the results of them are unsatisfactory. This paper proposes a new algorithm for tongue crack extraction based on crack’s grayscale and texture in the tongue image. The method analyses and handles the tongue image’s luminance component in HSV color space. Firstly, we use the morphological opening closing reconstruction to eliminate small texture and noise of tongue images. Then we use Bot-hat transform and Otsu adaptive threshold segmentation to extract tongue cracks. Finally, we use correlation and entropy of grayscale symbiotic matrix to screen extracted tongue cracks and realize automatic extraction of tongue cracks. The result has shown that the method can extract tongue cracks with high accuracy and lay a good foundation for subsequent analysis of the crack.

The Emotional Recognition and Sharing-Space Applying in University Library Based on the Analysis of Tongue Image

Knowledge-sharing spaces in university libraries have gradually become a new area for college students to learn, research, and experience. How to apply them in constructing shared knowledge spaces corresponding to different emotions plays an increasing role in promoting library services and development. Taking college students as the main users, this paper gives the relationship between the physical parameters of shared space and the basic emotion of users by systematically analyzing the interaction between user sentiment and situational physical parameters. Besides, the typical emotions are represented and identified by using the tongue diagnosis technique (TDT), for the emotion of library users can be recognized by identifying their tongue color. At last, we put forward an effective method that the building and applying of the library-sharing spaces can be done based on corresponding emotion patterns of the users. The result of the college student questionnaire survey shows that this method has a good validity and feasibility which may contribute a lot to service promotion and library development in universities.

Study on establishment of an animal model of Qi deficiency syndrome base on theory of over exertion leading to Qi consumption in traditional Chinese medicine

Based on the theory of "overexertion leading to Qi consumption", this study aimed to compare the intervention factors of nervous and physical fatigue to establish a rat model of Qi deficiency syndrome, moreover, the systematic evaluationsystem was necessary with the standard of syndrome diagnosis of Qi deficiency. According to the clinical-mimetic principle the Qi deficiency syndrome animal model is established by the factors of sleep deprivation(SD) and exhaustive swimming(ES). Rats were randomly divided into four groups, normal group, SD group, ES group and compound group (exhaustion swimming complex sleep deprivation). The method of ES was force rats to swim until the exhaustion with 5% of weight, and the SD method by using multi-platform sleep deprivation of water environment randomly 14-16 h daily, and the complex set by using of two methods was combined. The movement distance, average speed, activity time and numbers into the central area were detected by open field test, swimming exhaustion time was detected by swimming exhaustion method, the cardiac function (LVEF, CO, FS, LVDd) was detected by echocardiography, tongue imaging analysis and pulsedistention as the index of tongue and pulse diagnosis, the content of ATP and ADP in serum was detected by ELISA, the blood coagulation indexes for blood stasis syndrome as the additional. These results indicated that independent activity, body weight and exercise capacity were decreased significantly, the neurological function injured significantly, heart function was decreased significantly, the tongue surface color was pale white, and the pulse distention was decreased significantly. The content of ATP decreased significantly and the ADP increased significantly. By the method of four-diagnostic evaluation system to compare the factors of SD and ES, the Qi deficiency syndrome animalmodel could be successfully established by sleep deprivation method.

A novel Color Rendition Chart for digital tongue image calibration

AbstractDigital tongue images are usually acquired by a camera under specific illumination environments. In order to guarantee better color representation of the tongue body, we propose a novel tongue Color Rendition Chart acting as a color reference to be used in color calibration algorithms to standardize the captured tongue images. First, based on a large tongue image database captured with our digital tongue image acquisition system, we establish a statistical tongue color gamut. Then, from the first step, different quantities of colors in the Color Rendition Chart are determined via experimentation. Afterwards, results using X‐Rite's ColorChecker® Color Rendition Chart (a standard in the color calibration field) are compared with the proposed tongue Color Rendition Chart by applying the color difference calculation formula of CIELAB and CIEDE2000 as a reference for the mean color calibration error. The results show that the proposed tongue Color Rendition Chart, which has 24 colors, produces a much smaller error (CIELAB —8.0755/CIEDE 2000—6.3482) compared with X‐Rite's ColorChecker® Color Rendition Chart (CIELAB 1976—14.7836/CIEDE 2000—11.7686). This demonstrates the effectiveness of the novel tongue Color Rendition Chart.

Preliminary study on two analytical methods for evaluation of traditional Chinese medicine syndromes model in rats with blood stasis due to sleep deprivation

The comparison on evaluating blood stasis syndrome in sleep deprived rats was carried out by using R, G, B image analysis of Tongue and palm as well as auricle, palm surface laser Doppler flow perfusion. The experiment was performed by means of a small platform on the water environment for sleep deprivation. The rats were weekly weighed at fixed time, and their macroscopic signs were observed; and their tongue and palm images of the control and model group were respectively collected by the SLR camera at the 2nd, 4th and 6th week. Then the color saturation analysis was performed by means of proofreading with the standard colorimetric card. At the same time, the laser dopper flowmetry was used to analyze the perfusion of auricle and foot flow in rats. It turned out that there was no significant difference in the R,G,B value of the tongue and palm in rats between normal group and model group at the first stage(at the 2nd week), so were the perfusion of auricle and foot flow in rats. But at the second stage (at the 4th week), the R value of tongue in model group rats was obviously lower than that in normal group(P<0.01), and the other value (G,B) of tongue in module rats had a decease tendency, but there was no statistical significance. However, the perfusion of left and right auricle flow in model group rats were dramatically decreased as compared with the normal group(P<0.01); there was still no significant difference in the perfusion of the palm between two groups. It was found that R,G,B value in model group had a lower trend as compared with the control group of the tongue and palm images at the third stage (at the 6th week), but no statistically significant difference. The perfusion of left and right auricle flow in model group was constantly decreased as compared with the normal group(P<0.01).Right and left foot blood flow was lower than the normal group, but no statistically significant difference. We can safely conclude that the results of the R, G, B values of the tongue in rats could objectively reflect the characteristics of the rats with blood stasis syndrome, which were consistent with the diagnosis of clinical tongue image. As a method of microcirculation evaluation, the surface laser doppler perfusion of auricle can exhibit the characteristics of blood stasis in model rats, but also was more objective and reproducible. Therefore, the combination of R, G, B value of tongue as well as auricle laser doppler blood flow is more beneficial to the objective evaluation of index in the later study of traditional Chinese medicine blood stasis syndrome model.

0534 Balanced Co-activation of Protrudors and Rectractors is Required to Prevent Pharyngeal Obstruction During Sleep

Obstructive sleep apnea (OSA) is characterized by recurrent upper airway obstruction caused by a loss of upper airway muscle tone during sleep. Reductions in Genioglossus (GG) muscle tone, the main upper airway (UA) dilator, during sleep is thought to play a critical role in the pathogenesis of UA obstruction. Nevertheless, recent evidence suggests that lingual retractors also stabilize UA patency during sleep, leading us to hypothesize that co-activating lingual muscles can augment responses in UA patency during sleep 14 apneic patients implanted with a multi-channel hypoglossal nerve stimulating system (Imthera aura 6000) underwent mid-sagittal ultrasound tongue imaging during wakefulness. Changes in tongue shape and position were characterized by measuring its vertical height and the polar dimensions between the tongue surface and genioglossus origin in the mandible (mental spine). Changes in airway patency were characterized by comparing airflow responses between stimulated and adjacent unstimulated breaths during NREM sleep. Two distinct morphologic responses to stimulation were observed. Anterior tongue base and hyoid bone movement (5.3 ± 0.2 to 4.2 ± 0.5 cm) with concomitant increases in tongue height (4.7 ± 0.4 to 5.3 ± 0.7 cm) resulting in only modest changes in airway patency. In contrast, when stimulation generated forward movement (5.7 ± 0.3 to 4.5 ± 0.7 cm) without any increase in height (4.6 ± 0.5 to 4.6 ± 0.9 cm), marked increases in airway patency were observed. Lingual muscles that protrude the tongue while preserving tongue shape and prevent the tongue from bulging upward were associated with greater increases in pharyngeal patency during sleep. These morphologic differences suggest that co-stimulating lingual muscles stiffen the tongue, and prevent it from prolapsing into the pharynx during sleep. Our findings further suggest that dilating and stiffening actions of lingual protrudors and retractors can maintain UA patency during sleep. Imthera Medical AHA: 16post31000017.

RETRACTED ARTICLE: A novel semi supervised learning algorithm for thyroid and ulcer classification in tongue image

Tongue diagnosis is used to identify various diseases in the body now a day. The tongue image indicates the condition of different parts in body and signs in the tongue shows the misbehavior of internal parts of the body. This paper proposes a semi-supervised learning based approach to identify ulcer and thyroid of a patient by analyzing tongue image. The midpoint of tongue indicates intestine organ; if there is any problem in the intestine, it will be reflected in the midpoint of the tongue. Initially preprocessing steps are implemented to reduce noise and other irrelevant pixels from the image. Then segmentation is done to extract the specific part from the tongue and parameters collected from a healthy patient are considered to be the threshold value. The parameters collected from the extracted tongue image are compared with the threshold value, and the current condition of a patient is identified. A semi-supervised technique is introduced with a show the subtle elements of the patient whether he is in a protected or essential condition. The simulation work does with the MATLAB simulation environment utilizing adaptive semi-supervised learning technique. The simulation output shows the productivity of the semi-supervised learning strategy and its ability to recognize the issues of the tongues. Subsequently the proposed demonstrate delivers better highlights human body tissues. More than 97% efficiency accomplished by utilizing semi adaptive supervised learning technique.

Automatic tongue contour extraction in ultrasound images with convolutional neural networks

Ultrasound imaging of the tongue can provide detailed articulatory information addressing a variety of phonetic questions. However, using this method often requires the time-consuming process of manually labeling tongue contours in noisy images. This study presents a method for the automatic identification and extraction of tongue contours using convolutional neural networks, a machine learning algorithm that has been shown to be highly successful in many biomedical image segmentation tasks. We have adopted the U-net architecture (Ronneberger, Fischer, &amp; Brox 2015, U-Net: Convolutional Networks for Biomedical Image Segmentation, DOI:10.1007/978-3-319-24574-428), which learns from human-annotated splines using multiple, repeated convolution and max-pooling layers in the network for feature extraction, as well as deconvolution layers for generating spatially precise predictions of the tongue contours. Trained using a preliminary dataset of 8881 human-labeled tongue images from three speakers, our model generates discrete tongue splines comparable to those identified by human annotators (Dice Similarity Coefficient = 0.71). Although work is ongoing, this neural network based method shows considerable promise for the post-processing of ultrasound images in phonetic research. [Work supported by NSF grant BCS-1348150 to P.S. Beddor and A.W. Coetzee.]