Mouth Contour Research Articles

Grading facial aging: Comparing the clinical assessments made by three dermatologists with those obtained by an AI-based scoring system that analyses selfie pictures. Afocus on Chinese subjects of both genders.

The objective of this study is to assess the correspondence, in live conditions, between clinical gradings of facial aging signs by three dermatologists and those afforded by an automatic AI-based algorithm that analyses smartphones' selfie images of Chinese subjects. In total, 125 Chinese subjects of both genders, aged 18-62y, took a selfie using their own smartphones and were immediately viewed by three dermatologists. The latter graded the severity of 15 facial signs in women and 9 in men, using the standardized values afforded by a Skin Aging Atlas referential dedicated to Asian skin. The data issued by both methodologies were then statistically compared. The absolute gradings of the automatic system were found highly correlated with clinical assessments, with lower values in most cases. In women, large differences in absolute values were found on the gradings for size of isolated spot, cheek fold, spread macules, and texture of mouth contour women. Analysis of the Mean Absolute Errors (M.A.E) revealed that these rarely exceed 0.6 grading units in women and to a lesser extent in men. The present study confirmed the value of the automatic system towards an extended use towards large human cohorts as a surrogate of clinical evaluations and allowed to detect the points where improvements must be brought to the system.

Analysis of skin aging patterns using a facial imaging system in patients with atopic dermatitis.

There are few studies on skin aging in patients with atopic dermatitis (AD). To clarify the characteristics of facial skin aging in AD patients. Using facial images obtained by a digital imaging system (VISIA evolution), we compared the severity scores for 10 aging signs in 53 women in the AD group and 29 women in the healthy control group, all 35-49 years old. The severity scores for fine lines on the forehead, periorbital wrinkles, nasolabial folds, and texture of the mouth contour were significantly higher in the AD group than in the controls. However, in order to exclude a direct effect of dermatitis at the time of measurement, cases with signs of AD at the evaluation site were excluded from the AD group (defined as the AD [non-lesion] group), revealing no statistical significance between the AD (non-lesion) group and the healthy control group for any of the 10 facial signs. Age subset analysis showed that for individuals in their late 40s, the AD (non-lesion) group exhibited significantly higher scores for crow's feet wrinkle and nasolabial fold compared to the healthy control group. Furthermore, these two scores correlated with one other, suggesting that they may be induced by the same factors. The results of this study show that skin aging associated with AD is prominent in areas prone to transient wrinkling by frequent blinking and speaking or facial expressions. Understanding of the need for appropriate AD treatment from a cosmetic perspective may increase patient adherence.

Estreno absoluto: Concert Version of Felipe Pedrell’s La Celestina (1902) in Madrid’s Teatro de la Zarzuela (2022)

<i>Estreno absoluto</i>: Concert Version of Felipe Pedrell’s <i>La Celestina</i> (1902) in Madrid’s Teatro de la Zarzuela (2022)

Adaptive Semantic-Spatio-Temporal Graph Convolutional Network for Lip Reading

The goal of this work is to recognize words, phrases, and sentences being spoken by a talking face without given the audio. Current deep learning approaches for lip reading focus on exploring the appearance and optical flow information of videos. However, these methods do not fully exploit the characteristics of lip motion. In addition to appearance and optical flow, the mouth contour deformation usually conveys significant information that is complementary to others. However, the modeling of dynamic mouth contour has received little attention than that of appearance and optical flow. In this work, we propose a novel model of dynamic mouth contours called Adaptive Semantic-Spatio-Temporal Graph Convolution Network (ASST-GCN), to go beyond previous methods by automatically learning both the spatial and temporal information from videos. To combine the complementary information from appearance and mouth contour, a two-stream visual front-end network is proposed. Experimental results demonstrate that the proposed method significantly outperforms the state-of-the-art lip reading methods on several large-scale lip reading benchmarks.

Audio-visual speech processing and analysis based on subspace projections

Subject of Research.The paper deals with the problems of the mutual reconstruction (transformation) of acoustic and visual components (modalities) of speech. Audio recording of voice represents the acoustic component whereas the parallel video recording of the speaker’s face comprises the visual component. Because of the different physical nature of these modalities, their mutual analysis is accompanied by numerous difficulties. Reconstruction methods can be used to overcome these difficulties. Method. The proposed approach is based on Principal Component Analysis (PCA), Multiple Linear Regression (MLR), Partial Least Squares regression (PLS regression) and K-means clustering algorithm. Moreover, attention is paid to data preprocessing. Mel-frequency cepstral coefficients (MFCCs) are used as acoustic features, and twenty key points, which represent the mouth contour, comprise visual features. Main Results. The experiments on the reconstruction of the mouth contour from the MFCCs are presented. The experiments were carried out on VidTIMIT dataset of audio-visual phrase recordings in English. Four variants of the proposed approach were tested and evaluated. They are based on PCA and PLS regression with clustering and without it. Quantitative (objective) and qualitative (subjective) assessment confirmed the efficiency of the proposed approach. The implementation based on PLS regression with preliminary clustering led to the best results. Practical Relevance. The proposed approach can be used to develop various bimodal biometric systems, voice-driven virtual “avatars”, mobile access control systems and other useful human-computer interaction solutions. Moreover, it is shown that, given the proper implementation, PCA and PLS reduce significantly the computational complexity of the reconstruction operation. In addition, the clustering step can be omitted to increase additionally the processing speed at the cost of slightly lower reconstruction quality.

Eye and mouth state detection algorithm based on contour feature extraction

Eye and mouth state analysis is an important step in fatigue detection. An algorithm that analyzes the state of the eye and mouth by extracting contour features is proposed. First, the face area is detected in the acquired image database. Then, the eyes are located by an EyeMap algorithm through a clustering method to extract the sclera-fitting eye contour and calculate the contour aspect ratio. In addition, an effective algorithm is proposed to solve the problem of contour fitting when the human eye is affected by strabismus. Meanwhile, the value of chromatism s is defined in the RGB space, and the mouth is accurately located through lip segmentation. Based on the color difference of the lip, skin, and internal mouth, the internal mouth contour can be fitted to analyze the opening state of mouth; at the same time, another unique and effective yawning judgment mechanism is considered to determine whether the driver is tired. This paper is based on the three different databases to evaluate the performance of the proposed algorithm, and it does not need training with high calculation efficiency.

A Novel Semantic Approach for Multi-Ethnic Face Recognition

This paper proposes a semantic concept method to recognize multi-ethnic people based on Axiomatic fuzzy set (AFS) theory with application to image analysis. There are two advantages of the proposed approach: (i) It can convert the facial features to semantic concepts and in such a way we bridge the semantic gap between low level pixel features and interpretable concepts. (ii) It can implement the logical operation of semantic concepts in the AFS framework. Technically, we first construct facial features utilizing the facial landmarks such as eyes, nose, mouth, and face contour. Second, we establish some corresponding semantic concepts to describe facial features. Finally, a set of the semantic concept rules are extracted to form a classifier aimed at identifying facial ethnic attributes. The efficacy of the proposed approach is verified on Chinese Multi-ethnic face database (CMFD), FEI and CK[Formula: see text]. Meanwhile, we first demonstrate that the selected features have two obvious advantages: (1) these features can achieve better performance for ethical recognition than the features based on pixel values directly. (2) The selected features can be obtained via facial landmark detector regardless of the image resolutions. Then, we compare the proposed approach with some existing classifiers using the selected features, such as principal component analysis (PCA), C4.5, Decision table, Cart, Fuzzy Decision Tree (FDT) and Repeated Incremental Pruning to Produce Error Reduction (Ripper), extensive experiments show that our method exhibits a similar performance with these methods, which is demonstrated by Friedman test, however, our proposed approach can provide interpretability and comprehension capability.

Robust lip detection based on histogram of oriented gradient features and convolutional neural network under effects of light and background

Detection of the lip area is essential pre-processing for several applications, such as lip reading and visual information services. In this paper, we propose a lip detection method that finds the lip area in an image using the histogram of oriented gradient (HOG) features and convolutional neural network (CNN). We find the face area from an input image, divide the face image in half, and apply sliding window detection to the bottom half of the image. We obtain the HOG feature vector from the image that corresponds to the window, and use it as the input to a pre-trained support vector machine (SVM). HOG and SVM are used for coarse detection. If SVM determines that the image is not the lip, we reapply sliding window detection. Otherwise, the image is used as input to CNN, which is employed for fine detection and to determine whether the image is the lip. If CNN determines that the image is the lip, we apply canny edge detection to the image to obtain the mouth contour. We use MATLAB to confirm the effectiveness of our method, and can find the mouth area with over 94% accuracy and over 98% precision.

Hypovigilant System: An Approach for Lethargy Detection

There is much attentional impairment while driving that affect driver’s reaction. Among which driving while lethargic is one of the major causes behind road accidents, and exposes the driver to a large extent of crash risk compared to driving while alert. Therefore, the use of an assistive system that monitor a driver’s level of vigilance and alert the driver in case of lethargy can be significant in the prevention of accidents. This paper introduces a new approach towards detection of driver's lethargy based on yawning measurement and head movement. This involves several steps which includes the real time detection and tracking of head movement, the detection of yawning based on measuring the amount of changes in the mouth contour area. Test results express that the proposed system is able to measure the aforesaid parameters and detect driver’s lethargy.

Survey on Driver’s Drowsiness Detection System

There is much attentional impairment while driving that affect driver’s reaction. Among which driving while drowsy is one of the major causes behind road accidents, and exposes the driver to a much higher crash risk compared to driving while alert. Therefore, the use of an assistive system that monitor a driver’s level of vigilance and alert the driver in case of drowsiness can be significant in the prevention of accidents. This paper introduces a new approach towards detection of driver's drowsiness based on yawning measurement and head movement. This involves several steps including the real time detection and tracking of driver’s face, detection and tracking of the mouth contour, the detection of yawning based on measuring both the rate and the amount of changes in the mouth contour area and head movement tracking. Test results demonstrate that the proposed system can efficiently measure the aforementioned parameters and detect driver’s drowsiness.

Second stage surgery for repair of unilateral cleft lip based on the design of nose and mouth contour

Objective To investigate the approach and clinical efficacy of second repair surgery according to the nose and mouth contour design of unilateral cleft lip.Methods Based on the characteristics of anatomy,mechanics and deformity of the cleft lip,we designed a surgical incision of nasolabial contour lines with reference to the horizontal and central vertical lines and the contour.After careful investigation of classic procedures and clinical practice,unilateral cleft lip was repaired by overall stage Ⅱ surgical procedures.Results Application of this surgical method with satisfactory clinical efficacy had been achieved.Conclusions The second stage repair personalized surgical methods of unilateral cleft lip designed according to the nose and mouth contour can achieve maximal correction of malformation and beautiful appearance. Key words: Unilateral cleft lip deformity; Second stage repair; Contour design; Comprehensive repair

FaceWarehouse: A 3D Facial Expression Database for Visual Computing

We present FaceWarehouse, a database of 3D facial expressions for visual computing applications. We use Kinect, an off-the-shelf RGBD camera, to capture 150 individuals aged 7-80 from various ethnic backgrounds. For each person, we captured the RGBD data of her different expressions, including the neutral expression and 19 other expressions such as mouth-opening, smile, kiss, etc. For every RGBD raw data record, a set of facial feature points on the color image such as eye corners, mouth contour, and the nose tip are automatically localized, and manually adjusted if better accuracy is required. We then deform a template facial mesh to fit the depth data as closely as possible while matching the feature points on the color image to their corresponding points on the mesh. Starting from these fitted face meshes, we construct a set of individual-specific expression blendshapes for each person. These meshes with consistent topology are assembled as a rank-3 tensor to build a bilinear face model with two attributes: identity and expression. Compared with previous 3D facial databases, for every person in our database, there is a much richer matching collection of expressions, enabling depiction of most human facial actions. We demonstrate the potential of FaceWarehouse for visual computing with four applications: facial image manipulation, face component transfer, real-time performance-based facial image animation, and facial animation retargeting from video to image.

Detection of Tongue Protrusion Gestures from Video

SUMMARY We propose a system that, using video information, segments the mouth region from a face image and then detects the protrusion of the tongue from inside the oral cavity. Initially, under the assumption that the mouth is closed, we detect both mouth corners. We use a set of specifically oriented Gabor filters for enhancing horizontal features corresponding to the shadow existing between the upper and lower lips. After applying the Hough line detector, the extremes of the line that was found are regarded as the mouth corners. Detection rate for mouth corner localization is 85.33%. These points are then input to a mouth appearance model which fits a mouth contour to the image. By segmenting its bounding box we obtain a mouth template. Next, considering the symmetric nature of the mouth, we divide the template into right and left halves. Thus, our system makes use of three templates. We track the mouth in the following frames using normalized correlation for mouth template matching. Changes happening in the mouth region are directly described by the correlation value, i.e., the appearance of the tongue in the surface of the mouth will cause a decrease in the correlation coefficient through time. These coefficients are used for detecting the tongue protrusion. The right and left tongue protrusion positions will be detected by analyzing similarity changes between the right and left half-mouth templates and the currently tracked ones. Detection rates under the default parameters of our system are 90:20% for the tongue protrusion regardless of the position, and 84:78% for the right and left tongue protrusion positions. Our results demonstrate the feasibility of real-time tongue protrusion detection in vision-based systems and motivates further investigating the usage of this new modality in human

Vowel recognition of patients after total laryngectomy using Mel Frequency Cepstral Coefficients and mouth contour

The paper addresses a problem of isolated vowels recognition in patients following total laryngectomy. The visual and acoustic speech modalities were separately incorporated in the machine learning algorithms. The authors used the Mel Frequency Cepstral Coefficients as acoustic descriptors of a speech signal. A lip contour was extracted from a video signal of the speaking faces using OpenCV software library. In a vowels recognition procedure the three types of classifiers were used for comparison purposes: Artificial Neural Networks, Support Vector Machines and Naive Bayes. The highest recognition rate was evaluated using Support Vector Machines. For a group of the laryngectomees having a different quality of speech the authors achieved 75% for acoustic and 40% for visual recognition performances. The authors obtained higher recognition rate than in a previous research where 10 cross-sectional areas of a vocal tract were estimated. Using presented image processing algorithm the visual features can be extracted automatically from a video signal.

Macrostomia, Preauricular Tags, and External Ophthalmoplegia: A New Autosomal Dominant Syndrome within the Oculoauriculovertebral Spectrum?

First and second branchial arch involvement during early embryonic development results in a wide spectrum of anomalies that encompass diverse, superimposed, and heterogeneous phenotypes within the so-called oculoauriculovertebral spectrum. Nine members of a Brazilian family presenting typical branchial arch involvement in association with external opthalmoplegia are reported. Macrostomia or abnormal mouth contour, preauricular tags, and uni- or bilateral ptosis were present in association in several patients. To our knowledge, this is the first report on this type of autosomal dominant condition. Clinical and genetic aspects are discussed.