Facial Speech Research Articles

Primary surgery in cleft lip and palate part 5

This article discusses primary surgery for patients with cleft lip and palate, which in the UK would conventionally take place in the first year of life. Current protocols in the United Kingdom are described and some of the history in the surgical specialties that led to their development discussed. The basic principles of the surgery are explained and the impact of surgery on future facial aesthetics, dental development and speech are considered. Clinical Relevance: Clefting of the lip produces a significant deformity and, although not functionally debilitating, will lead to a significant aesthetic impact, which in our modern day, aesthetically driven society can produce significant psychosocial consequences, even with repair.

Connecting multimodality in human communication

A successful reciprocal evaluation of social signals serves as a prerequisite for social coherence and empathy. In a previous fMRI study we studied naturalistic communication situations by presenting video clips to our participants and recording their behavioral responses regarding empathy and its components. In two conditions, all three channels transported congruent emotional or neutral information, respectively. Three conditions selectively presented two emotional channels and one neutral channel and were thus bimodally emotional. We reported channel-specific emotional contributions in modality-related areas, elicited by dynamic video clips with varying combinations of emotionality in facial expressions, prosody, and speech content. However, to better understand the underlying mechanisms accompanying a naturalistically displayed human social interaction in some key regions that presumably serve as specific processing hubs for facial expressions, prosody, and speech content, we pursued a reanalysis of the data. Here, we focused on two different descriptions of temporal characteristics within these three modality-related regions [right fusiform gyrus (FFG), left auditory cortex (AC), left angular gyrus (AG) and left dorsomedial prefrontal cortex (dmPFC)]. By means of a finite impulse response (FIR) analysis within each of the three regions we examined the post-stimulus time-courses as a description of the temporal characteristics of the BOLD response during the video clips. Second, effective connectivity between these areas and the left dmPFC was analyzed using dynamic causal modeling (DCM) in order to describe condition-related modulatory influences on the coupling between these regions. The FIR analysis showed initially diminished activation in bimodally emotional conditions but stronger activation than that observed in neutral videos toward the end of the stimuli, possibly by bottom-up processes in order to compensate for a lack of emotional information. The DCM analysis instead showed a pronounced top-down control. Remarkably, all connections from the dmPFC to the three other regions were modulated by the experimental conditions. This observation is in line with the presumed role of the dmPFC in the allocation of attention. In contrary, all incoming connections to the AG were modulated, indicating its key role in integrating multimodal information and supporting comprehension. Notably, the input from the FFG to the AG was enhanced when facial expressions conveyed emotional information. These findings serve as preliminary results in understanding network dynamics in human emotional communication and empathy.

The sequential treatment of extensive mandibular ameloblastoma

Ameloblastoma is the most frequent odontogenic tumor of the jaw. If not treated, ameloblastoma can gain an enormous size and cause severe facial disfigurement and functional impairment. Here we report two patients afflicted with extensive mandibular ameloblastoma (sized in 23 cm × 18 cm × 17 cm and 15 cm × 12 cm × 10 cm, respectively). Both the patients received the same sequential treatment including radical tumor resection, simultaneous reconstruction with fibula free flap graft, vertical distraction osteogenesis on the fibula graft, placement of endosseous dental implants, and final pros-thodontic rehabilitation. It took about 15 months to finish the entire course of treatment. And after the four-year follow-up, neither soft tissue related, nor hard tissue related problems were observed. Satisfactory facial symmetry, chewing and speech functions of the patients were restored. So this sequential treatment for extensive mandibular ameloblastoma can obtain an excellent effect by the shortest time and the lowest economical cost. Furthermore, the series also can be used to reconstruct giant mandibular defects caused by different reasons.

Bimodal Emotion Recognition via Facial Expression and Speech

Bimodal Emotion Recognition via Facial Expression and Speech

Distinct Facial Characteristics Differentiate Communicative Intent of Infant‐Directed Speech

Adults and infants can differentiate communicative messages using the nonlinguistic acoustic properties of infant‐directed (ID) speech. Although the distinct prosodic properties of ID speech have been explored extensively, it is currently unknown whether the visual properties of the face during ID speech similarly convey communicative intent and thus represent an additional source of social information for infants during early interactions. To examine whether the dynamic facial movement associated with ID speech confers affective information independent of the acoustic signal, adults' differentiation of the visual properties of speakers' communicative messages was examined in two experiments in which the adults rated silent videos of approving and comforting ID and neutral adult‐directed speech. In Experiment 1, adults differentiated the facial speech groups on ratings of the intended recipient and the speaker's message. In Experiment 2, an original coding scale identified facial characteristics of the speakers. Discriminant correspondence analysis revealed two factors differentiating the facial speech groups on various characteristics. Implications for perception of ID facial movements in relation to speakers' communicative intent are discussed for both typically and atypically developing infants. Copyright © 2012 John Wiley & Sons, Ltd.

Facial Asymmetry Correction in Facial Palsy Patients with Silhouette Sutures

Introduction: over the last few decades several techniques static and dynamics, have been performed to improve facial asymmetry and functionality alter suffering facial paralysis. Methods: we present a pilot study to show and evaluate the benefits of a new form of facial suspension, with Silhouette sutures. We performed two patients with total and complete facial palsy due to otical tuberculosis in one case and to parotid carcinoma in the other. Results: one year after surgery, both patients have improved facial asymmetry, mastication and speech production which have lead to a higher self-esteem and major social interaction. Conclusions: static facial suspension with Silhouette sutures is a non invasive alternative to dynamic techniques in patients who don’t want or can’t undergo more complex surgeries.

Velopharyngeal Insufficiency and Cleft

Management of orofacial clefts requires a comprehensive approach to address the multifaceted aspects of possible aesthetic and functional deficits. Many children with cleft palate with or without cleft lip continue to have speech abnormalities that can affect socialization, education, and childhood development. An interdisciplinary cleft team ideally manages most orofacial clefts and can include members from the fields of facial plastic surgery, otolaryngology, speech and language pathology, pediatrics, nursing, audiology, genetics, oral surgery, orthodontics, dentistry, and social work. These teams follow a relatively established timeline and algorithm for each of the discrete steps of orofacial cleft treatment. Cleft lip repair is often performed after 3–5 months of age. Palatoplasty is delayed after 8–14 months of age. Speech and language pathology assessment and therapy are targeted as the vocabulary develops (2–5 years old). This special issue deals with the cleft surgeon and speech therapists role in treatment of velopharyngeal dysfunction (VPD), which is often related to velopharyngeal insufficiency (VPI). This lack of coordinated closure of the soft palate to the posterior pharyngeal wall is often a sequela of cleft palate, even after primary repair. One paper from this issue titled “Nonlinear dynamic analysis of vowels in cleft palate children with or without hypernasality” reports on cephalometric differences between the skull base and velopharyngeal anatomy in patients with unilateral cleft lip and palate compared to normal controls. These data contribute to the existing literature of abnormal skull base anatomy and inclination in children with velocardiofacial syndrome, which often associated with VPD. The next paper titled “Noncleft velopharyngeal insufficiency: etiology and need for surgical treatment” analyzes those children with VPD without obvious orofacial clefting, and assesses the etiologies and potential treatment options. This allows for comparison to the traditional orofacial cleft-related VPD management. The velopharyngeal workup is often a coordinated assessment by the cleft surgeon and speech pathologists. It can include subjective assessment, which is the subject of the next two papers. First, the next paper titled “Nonlinear dynamic analysis of vowels in cleft palate children with or without hypernasality” assesses the relationship between two speech assessment tools, Lyapunov exponents, and Nasalance scores in the presence of hypernasal speech. Furthermore, an additional paper titled “Assessment of single-word production for children under three years of age: comparison of children with and without cleft plate” presents a case-control testing of a single word assessment tool in children less than 3 years of age with cleft palate, which indentified speech differences in error patterns, consonants, articulation, and accuracy between children with and with clefts. Objective assessment of VPD also can include either fluoroscopic speech examination or nasopharyngoscopy to characterize the pattern of VPD and target a surgical goal. The next paper titled “Objective assessment of hypernasality in patients with cleft lip and palate with the nasalView system: a clinical validation study” assesses the validity and reliability of a new tool for hypernasality testing called NasalView. Speech and language therapy is instituted to address typical compensatory speech patterns seen in children with cleft palate (e.g., glottal stops). If hypernasal speech is not responsive to speech and language therapy, secondary speech surgery is warranted to address hypernasality in this time period. Surgical options may include a superiorly based pharyngeal flap, dynamic sphincter pharyngoplasty, or posterior pharyngeal wall augmentation. Occasionally, a palate-lengthening procedure (Furlow double-opposing Z-plasty) is performed. These procedures can limit nasal air emissions, but potential obstructive sleep apnea from excessive nasal obstruction is a risk. The risk of sleep apnea after surgically treating the velopharynx may be decreased if the tonsils and adenoids are addressed preemptively. Before pharyngeal flap surgery, some surgeons perform tonsillectomy to limit the potential variability of speech and airway changes that comes with fluctuating tonsillar hypertrophy, tonsillitis, or upper respiratory infections. The use of tonsillectomy in the face of VPI in children with cleft palate is somewhat controversial and is the topic of the last paper “Speech outcomes after tonsillectomy in patients with known velopharyngeal insufficiency”. The guest editors are pleased to present a collection of papers dealing with management of the speech-related difficulties in children with orofacial clefts. In this era of the expanding role of evidenced-based medicine, the effectiveness of treatments can only be monitored and improved by assessment with valid and consistent outcome measures, which are reviewed in this issue. Travis T. Tollefson David White James Brookes Steven Goudy

Emotion-Aware Assistive System for Humanistic Care Based on the Orange Computing Concept

Mental care has become crucial with the rapid growth of economy and technology. However, recent movements, such as green technologies, place more emphasis on environmental issues than on mental care. Therefore, this study presents an emerging technology called orange computing for mental care applications. Orange computing refers to health, happiness, and physiopsychological care computing, which focuses on designing algorithms and systems for enhancing body and mind balance. The representative color of orange computing originates from a harmonic fusion of passion, love, happiness, and warmth. A case study on a human-machine interactive and assistive system for emotion care was conducted in this study to demonstrate the concept of orange computing. The system can detect emotional states of users by analyzing their facial expressions, emotional speech, and laughter in a ubiquitous environment. In addition, the system can provide corresponding feedback to users according to the results. Experimental results show that the system can achieve an accurate audiovisual recognition rate of 81.8% on average, thereby demonstrating the feasibility of the system. Compared with traditional questionnaire-based approaches, the proposed system can offer real-time analysis of emotional status more efficiently.

Is theory of mind related to social dysfunction and emotional problems in 22q11.2 deletion syndrome (velo-cardio-facial syndrome)?

Social dysfunction is intrinsically involved in severe psychiatric disorders such as depression and psychosis and linked with poor theory of mind. Children with 22q11.2 deletion syndrome (22q11DS, or velo-cardio-facial syndrome) have poor social competence and are also at a particularly high risk of developing mood (40%) and psychotic (up to 30%) disorders in adolescence and young adulthood. However, it is unknown if these problems are associated with theory of mind skills, including underlying social-cognitive and social-perceptual mechanisms. The present cross-sectional study included classic social-cognitive false-belief and mentalising tasks and social-perceptual face processing tasks. The performance of 50 children with 22q11DS was compared with 31 age-matched typically developing sibling controls. Key findings indicated that, while younger children with 22q11DS showed impaired acquisition of social-cognitive skills, older children with 22q11DS were not significantly impaired compared with sibling controls. However, children with 22q11DS were found to have social-perceptual deficits, as demonstrated by difficulties in matching faces on the basis of identity, emotion, facial speech and gaze compared with sibling controls. Furthermore, performance on the tasks was associated with age, language ability and parentally rated social competence and emotional problems. These results are discussed in relation to the importance of a better delineation of social competence in this population.

Clinical Management of Microstomia Due to the Static Treatment of Facial Paralysis and Oral Rehabilitation With Dental Implants

Facial-nerve paralysis is seldom seen and may occur because of a broad spectrum of causes. The most commonly seen cause of facial paralysis is the Bell palsy; iatrogenic causes and tumors are relatively rare. Facial asymmetry, drooling, garbled speech, and difficulty in feeding: all adversely affect the psychosocial conditions of the patients. Fascial and tendon sling procedures may be performed for the static treatment of the unilateral permanent facial paralysis. These techniques are used both for the correction of the asymmetry of the face, especially by providing static support for the corner of the mouth, and to prevent drooling. Microstomia after a sling procedure is not a previously observed complication in the literature. A patient is presented with the surgical management of the complication of microstomia that had risen because of a static treatment of his unilateral facial paralysis via a tendon that passes circularly through his orbicularis oris muscle. Oral rehabilitation thereafter was maintained with the support of dental implants and fixed prosthodontics. The most efficient treatment protocol was decided with an interdisciplinary consultation of the oral and maxillofacial surgeon, the plastic surgeon, and the prosthodontist.

The relationships between processing facial identity, emotional expression, facial speech, and gaze direction during development

Four experiments were conducted with 5- to 11-year-olds and adults to investigate whether facial identity, facial speech, emotional expression, and gaze direction are processed independently of or in interaction with one another. In a computer-based, speeded sorting task, participants sorted faces according to facial identity while disregarding facial speech, emotional expression, and gaze direction or, alternatively, according to facial speech, emotional expression, and gaze direction while disregarding facial identity. Reaction times showed that children and adults were able to direct their attention selectively to facial identity despite variations of other kinds of face information, but when sorting according to facial speech and emotional expression, they were unable to ignore facial identity. In contrast, gaze direction could be processed independently of facial identity in all age groups. Apart from shorter reaction times and fewer classification errors, no substantial change in processing facial information was found to be correlated with age. We conclude that adult-like face processing routes are employed from 5 years of age onward.

Reading faces for information about words and emotions in adolescents with autism

Studies of explicit processing of facial expressions by individuals with autism spectrum disorder (ASD) have found a variety of deficits and preserved abilities compared to their typically developing (TD) peers. However, little attention has been paid to their implicit processing abilities for emotional facial expressions. The question has also been raised whether preferential attention to the mouth region of a speaker's face by ASD individuals has resulted in a relative lipreading expertise. We present data on implicit processing of pseudo-dynamic facial emotions and visual speech in adolescents with autism. We compared 25 ASD and 25 TD participants on their ability to recreate the sequences of four dynamic emotional facial expressions (happy, sad, disgust, and fear) as well as four spoken words (with, bath, thumb, and watch) using six still images taken from a video sequence. Typical adolescents were significantly better at recreating the dynamic properties of emotional expressions than those of facial speech, while the autism group showed the reverse accuracy pattern. For Experiment 2 we obscured the eye region of the stimuli and found no significant difference between the 22 adolescents with ASD and 22 TD controls. Fearful faces achieved the highest accuracy results among the emotions in both groups.

Stylized synthesis of facial speech motions

AbstractStylized synthesis of facial speech motions is central to facial animation. Most synthesis algorithms put emphasis on the reasonable concatenation of captured motion segments. The dynamic modeling of speech units, e.g. visemes and visyllables (the visual appearance of a syllable), has not drawn much attention. In this paper, we address the fundamental issues regarding the stylized dynamic modeling of visyllables. The decomposable generalized model is learnt for the stylized motion synthesis. The visyllable modeling includes two parts: (1) A dynamic model for each kind of visyllable that is learnt based on a Gaussian Process Dynamical Model; (2) A multilinear model based unified mapping between the high dimensional observation space and low dimensional latent space. The dynamic visyllable model embeds the high dimensional motion data, and constructs the dynamic mapping in the latent space simultaneously. To generalize the visyllable model from several instances, the mapping coefficient matrices are assembled to a tensor, which is decomposed into independent modes, e.g. identity and uttering styles. Therefore, with the linear combination of components in each mode, the novel stylized motions can be synthesized. Copyright © 2007 John Wiley & Sons, Ltd.

Ritualisierte Konfliktbewältigung bei Eilhart und Gottfried

This article focuses on the terms ‹ritual‹ and ‹ritualization‹ as well as literary traditions of describing ritualized emotions. Ritualized expression authenticates emotions through an ostentatious styling of one’s physical presence: through facial expression, gesture, movement, voice, and speech. Contrasting approaches to the use of these techniques are compared in the Tristan-Romances of Eilhart von Oberg and Gottfried von Strasburg. While Eilhart’s version is usually considered inferior to the famous text of Gottfried, Eilhart in fact motivates the episode of the attempted murder more coherently and convincingly. The clarity of Eilhart’s version is due to his more consistent use of epic narrative style, supported by his descriptions of traditional, ritualized conflict management techniques. Although Gottfried follows a similar strategy in The Ordeal, he is less successful in the episode of the attempted murder of Brangane. As Eilhart’s and Gottfried’s versions of the attempted murder are compared, it becomes clear that Gottfried’s version suffers from the ambivalence of employing both the tradition of epic storytelling as well as indications of psychological motivation. In the end, he does not succeed in incorporating the incident into his narrative.

Effect of face familiarity on age decision

The present experiment was planned to check whether the extraction of apparent age is affected by face identity (familiarity) or not. According to the traditional view, age estimation should be carried out independently of face identity, because it is one of the visually derived semantic codes (like gender and ethnicity). However, little is known about its underlying mechanisms. Moreover, some recent studies have cast doubt on the parallel thesis regarding facial expression, facial speech, ethnicity, and gender. Given the promising results of a pilot experiment ( n = 24), 16 Caucasian participants were enrolled in an “age decision” task on morphed faces derived from one old and one young source-face, in the proportion 70:30. The respondents had previously been familiarised with half the source faces by a learning procedure (associating the face, surname, occupation and city of residence of the person displayed), while the remaining half were unfamiliar. The results showed that age decision was affected by face familiarity, at least when the task was perceptually difficult enough. This adds support to the thesis that the identification of identity and the extraction of visually derived semantic codes are not made independently from each other. The status of age, within the visually derived semantic codes, is also discussed.

Range- and domain-specific exaggeration of facial speech

Is it possible to exaggerate the different ways in which people talk, just as we can caricature their faces? In this paper, we exaggerate animated facial movement to investigate how the emotional manner of speech is conveyed. Range-specific exaggerations selectively emphasized emotional manner whereas domain-specific exaggerations of differences in duration did not. Range-specific exaggeration relative to a time-locked average was more effective than absolute exaggeration of differences from the static, neutral face, despite smaller absolute differences in movement. Thus, exaggeration is most effective when the average used captures shared properties, allowing task-relevant differences to be selectively amplified. Playing the stimuli backwards showed that the effects of exaggeration were temporally reversible, although emotion-consistent ratings for stimuli played forwards were higher overall. Comparison with silent video showed that these stimuli also conveyed the intended emotional manner, that the relative rating of animations depends on the emotion, and that exaggerated animations were always rated at least as highly as video. Explanations in terms of key frame encoding and muscle-based models of facial movement are considered, as are possible methods for capturing timing-based cues.

Asymmetric interference between sex and emotion in face perception

Previous research with speeded-response interference tasks modeled on the Garner paradigm has demonstrated that task-irrelevant variations in either emotional expression or facial speech do not interfere with identity judgments, but irrelevant variations in identity do interfere with expression and facial speech judgments. Sex, like identity, is a relatively invariant aspect of faces. Drawing on a recent model of face processing according to which invariant and changeable aspects of faces are represented in separate neurological systems, we predicted asymmetric interference between sex and emotion classification. The results of Experiment 1, in which the Garner paradigm was employed, confirmed this prediction: Emotion classifications were influenced by the sex of the faces, but sex classifications remained relatively unaffected by facial expression. A second experiment, in which the difficulty of the tasks was equated, corroborated these findings, indicating that differences in processing speed cannot account for the asymmetric relationship between facial emotion and sex processing. A third experiment revealed the same pattern of asymmetric interference through the use of a variant of the Simon paradigm. To the extent that Garner interference and Simon interference indicate interactions at perceptual and response-selection stages of processing, respectively, a challenge for face processing models is to show how the same asymmetric pattern of interference could occur at these different stages. The implications of these findings for the functional independence of the different components of face processing are discussed.

Speaker Variations Influence Speechreading Speed for Dynamic Faces

We investigated the influence of task-irrelevant speaker variations on speechreading performance. In three experiments with video digitised faces presented either in dynamic, static-sequential, or static mode, participants performed speeded classifications on vowel utterances (German vowels /u/ and /i/). A Garner interference paradigm was used, in which speaker identity was task-irrelevant but could be either correlated, constant, or orthogonal to the vowel uttered. Reaction times for facial speech classifications were slowed by task-irrelevant speaker variations for dynamic stimuli. The results are discussed with reference to distributed models of face perception (Haxby et al, 2000 Trends in Cognitive Sciences 4 223-233) and the relevance of both dynamic information and speaker characteristics for speechreading.

Facial expression decoding in early Parkinson's disease

The ability to derive emotional and non-emotional information from unfamiliar, static faces was evaluated in 21 adults with idiopathic Parkinson's disease (PD) and 21 healthy control subjects. Participants' sensitivity to emotional expressions was comprehensively assessed in tasks of discrimination, identification, and rating of five basic emotions: happiness, (pleasant) surprise, anger, disgust, and sadness. Subjects also discriminated and identified faces according to underlying phonemic (“facial speech”) cues and completed a neuropsychological test battery. Results uncovered limited evidence that the processing of emotional faces differed between the two groups in our various conditions, adding to recent arguments that these skills are frequently intact in non-demented adults with PD [R. Adolphs, R. Schul, D. Tranel, Intact recognition of facial emotion in Parkinson's disease, Neuropsychology 12 (1998) 253–258]. Patients could also accurately interpret facial speech cues and discriminate the identity of unfamiliar faces in a normal manner. There were some indications that basal ganglia pathology in PD contributed to selective difficulties recognizing facial expressions of disgust, consistent with a growing literature on this topic. Collectively, findings argue that abnormalities for face processing are not a consistent or generalized feature of medicated adults with mild-moderate PD, prompting discussion of issues that may be contributing to heterogeneity within this literature. Our results imply a more limited role for the basal ganglia in the processing of emotion from static faces relative to speech prosody, for which the same PD patients exhibited pronounced deficits in a parallel set of tasks [M.D. Pell, C. Leonard, Processing emotional tone from speech in Parkinson's disease: a role for the basal ganglia, Cogn. Affect. Behav. Neurosci. 3 (2003) 275–288]. These diverging patterns allow for the possibility that basal ganglia mechanisms are more engaged by temporally-encoded social information derived from cue sequences over time.

Ethnic categorisation of faces is not independent of face identity.

Is the extraction of a visually derived semantic code from faces (ethnicity) affected by face identity (familiarity) or not? The traditional view considers that this operation is made independently of face identity, and in parallel with the recognition of identity. However, some recent studies cast doubt on this parallel thesis regarding other visually derived semantic codes, namely: facial expression, facial speech, apparent age, and gender. Twenty-eight Caucasian participants were enrolled in an 'ethnic-decision' task on morphed faces made of an Asiatic source face and a Caucasian source face, in the proportion of 70%-30%. Half of the original faces were previously made familiar by a learning procedure (associating the face, surname, occupation, and city of residence of the person displayed), while the remaining half were unfamiliar. The results showed clearly that ethnic decision was affected by face familiarity. This adds support to the thesis according to which the identification of identity and the extraction of visually derived semantic codes are not made independently from each other and that the 'parallel-route' hypothesis becomes weakly supported.