Eyebrow Movements Research Articles

Cross-modal facilitation in speech prosody

Speech prosody has traditionally been considered solely in terms of its auditory features, yet correlated visual features exist, such as head and eyebrow movements. This study investigated the extent to which visual prosodic features are able to affect the perception of the auditory features. Participants were presented with videos of a speaker pronouncing two words, with visual features of emphasis on one of these words. For each trial, participants saw one video where the two words were identical in both pitch and amplitude, and another video where there was a difference in either pitch or amplitude that was congruent or incongruent with the visual changes. Participants were asked to decide which video contained the sound difference. Thresholds were obtained for the congruent and incongruent videos, and for an auditory-alone condition. It was found that the congruent thresholds were better than the incongruent thresholds for both pitch and amplitude changes. Interestingly, the congruent thresholds for amplitude were better than for the auditory-alone condition, which implies that the visual features improve sensitivity to loudness changes. These results demonstrate that visual stimuli can affect auditory thresholds for changes in pitch and amplitude, and furthermore support the view that visual prosodic features enhance speech processing.

Are Eyebrow Movements Linked to Voice Variations and Turn-taking in Dialogue? An Experimental Investigation

Following our work on the relationship between eyebrow movements and the fundamental frequency of the voice, this article presents the results of a study on this phenomenon, and also on the temporal location of rapid eyebrow movements with respect to speaking turns during dialogue. We used an automatic movement-acquisition system coupled with the simultaneous, synchronized recording of the vocal production. This procedure permits an objective analysis of eyebrow movements in relation to the vocal production. The data obtained showed that the speakers' rapid eyebrow movements were associated both with turn-taking (occurring right before or right after speaking turn onset) and with changes in the fundamental frequency. These findings suggest that rapid eyebrow movements are mainly linked to motivation and the intention to communicate, and may also act as a focus marker in speech.

Mixed Signals: Combining Linguistic and Affective Functions of Eyebrows in Questions in Sign Language of the Netherlands

The eyebrows are used as conversational signals in face-to-face spoken interaction (Ekman, 1979). In Sign Language of the Netherlands (NGT), the eyebrows are typically furrowed in content questions, and raised in polar questions (Coerts, 1992). On the other hand, these eyebrow positions are also associated with anger and surprise, respectively, in general human communication (Ekman, 1993). This overlap in the functional load of the eyebrow positions results in a potential conflict for NGT signers when combining these functions simultaneously. In order to investigate the effect of the simultaneous realization of both functions on the eyebrow position we elicited instances of both question types with neutral affect and with various affective states. The data were coded using the Facial Action Coding System (FACS: Ekman, Friesen, & Hager, 2002) for type of brow movement as well as for intensity. FACS allows for the coding of muscle groups, which are termed Action Units (AUs) and which produce facial appearance changes. The results show that linguistic and affective functions of eyebrows may influence each other in NGT. That is, in surprised polar questions and angry content question a phonetic enhancement takes place of raising and furrowing, respectively. In the items with contrasting eyebrow movements, the grammatical and affective AUs are either blended (occur simultaneously) or they are realized sequentially. Interestingly, the absence of eyebrow raising (marked by AU 1+2) in angry polar questions, and the presence of eyebrow furrowing (realized by AU 4) in surprised content questions suggests that in general AU 4 may be phonetically stronger than AU 1 and AU 2, independent of its linguistic or affective function.

Facial expressions in various emotional states in congenitally blind and sighted subjects

Most of the studies comparing facial expressions of born blind and sighted persons that have been carried out up until now have involved children as subjects. Various studies claimed that the quantity and quality of facial expressions of congenitally blind persons deteriorate with increasing age. Here, we compared facial expressions of born blind and sighted individuals using adults predominantly. Facial expressions were documented in an individual interview inducing such emotions as think-concentrate, sadness, anger, disgust, joy, and surprise.Common characteristics found amongst studied individuals were: similar repertoires*of movements over the entire interview, high-frequency and high-repertoire proportion of facial movements in concentration, sadness, and anger relative to those in disgust, joy, and surprise, similar distributions of a cumulative repertoire proportion of facial movements, and also common behavioral profiles of frequencies of facial movements in the emotional states discussed. Similar displays of eyebrow movements were found as well in concentration, sadness, and anger.Our study indicates that most tested characteristics of facial movements are common to born blind and sighted subjects, except for different cumulative mean frequencies in different emotional states, which is possibly related to the lack of visual feedback in born blind persons.Our study substantiates the hypothesis that facial expressions are innate and have important cues in the evolution of social communication.

Comparison of Deliberate and Spontaneous Facial Movement in Smiles and Eyebrow Raises.

We investigated movement differences between deliberately posed and spontaneously occurring smiles and eyebrow raises during a videotaped interview that included a facial movement assessment. Using automated facial image analysis, we quantified lip corner and eyebrow movement during periods of visible smiles and eyebrow raises and compared facial movement within participants. As in an earlier study, maximum speed of movement onset was greater in deliberate smiles. Maximum speed and amplitude were greater and duration shorter in deliberate compared to spontaneous eyebrow raises. Asymmetry of movement did not differ within participants. Similar patterns contrasting deliberate and spontaneous movement in both smiles and eyebrow raises suggest a common pattern of signaling for spontaneous facial displays.

Analysis and synthesis of nonverbal facial motion

Until recently, most efforts in audio-visual speech synthesis have been concerned with verbal content. However, in human-human communication it is obvious that nonverbal signals plays an important role, such as when expressing emotions and attitudes. Interaction is also often regulated using facial cues, for example gaze, head and eyebrow movements. Some of these cues have a direct coupling to the speech signal, while other occur during both while speaking and listening. When applying interactive talking agents in man-machine systems, nonverbal signals may be very important in easing the flow of communication. In a series of experiments we have been exploring the function of nonverbal facial motion. These studies include an experiment on the interaction between expressive speech and prominence, as well as an attempt to synthesize emotions and attitudes in a talking head, using 3D motion capture data. Further we will report on a real-time experiment with human-human avatar-mediated conversation, where the subjects' turn-taking behavior is affected by facial motion in the avatars.

Suture Fixation Technique for Endoscopic Brow Lift

Endoscopic brow lift has become widely accepted as a procedure for restoring a youthful brow, as only three, hardly noticeable incisions of the scalp are needed for this subperiosteal dissection and final repositioning of the brow. It has become an acceptable technique, an alternative to the conventional technique or transcoronal browpexy. Endoscopic brow lift allows separation and repositioning of the periosteum of the orbital rims and zygomaxilla. In a 7-year period beginning September 1999, 350 patients underwent endoscopic brow lift. In these patients, we used our fixation approach, which was accomplished with an absorbable suture subperiosteally. Satisfactory forehead rejuvenation was obtained in all patients with correct eyebrow movement. Long-term results over 7 years confirm the strength and durability of our fixation approach. We consider this approach to be a simple, fast, and reliable fixation technique that establishes precise, secure, and direct brow fixation that allows satisfactory and long-lasting cosmetic results.

Corpus-based generation of head and eyebrow motion for an embodied conversational agent

Humans are known to use a wide range of non-verbal behaviour while speaking. Generating naturalistic embodied speech for an artificial agent is therefore an application where techniques that draw directly on recorded human motions can be helpful. We present a system that uses corpus-based selection strategies to specify the head and eyebrow motion of an animated talking head. We first describe how a domain-specific corpus of facial displays was recorded and annotated, and outline the regularities that were found in the data. We then present two different methods of selecting motions for the talking head based on the corpus data: one that chooses the majority option in all cases, and one that makes a weighted choice among all of the options. We compare these methods to each other in two ways: through cross-validation against the corpus, and by asking human judges to rate the output. The results of the two evaluation studies differ: the cross-validation study favoured the majority strategy, while the human judges preferred schedules generated using weighted choice. The judges in the second study also showed a preference for the original corpus data over the output of either of the generation strategies.

Interrelation Between Speech and Facial Gestures in Emotional Utterances: A Single Subject Study

The verbal and nonverbal channels of human communication are internally and intricately connected. As a result, gestures and speech present high levels of correlation and coordination. This relationship is greatly affected by the linguistic and emotional content of the message. The present paper investigates the influence of articulation and emotions on the interrelation between facial gestures and speech. The analyses are based on an audio-visual database recorded from an actress with markers attached to her face, who was asked to read semantically neutral sentences, expressing four emotion states (neutral, sadness, happiness, and anger). A multilinear regression framework is used to estimate facial features from acoustic speech parameters. The levels of coupling between the communication channels are quantified by using Pearson's correlation between the recorded and estimated facial features. The results show that facial and acoustic features are strongly interrelated, showing levels of correlation higher than r = 0.8 when the mapping is computed at sentence-level using spectral envelope speech features. The results reveal that the lower face region provides the highest activeness and correlation levels. Furthermore, the correlation levels present significant interemo- tional differences, which suggest that emotional content affect the relationship between facial gestures and speech. Principal component analysis (PCA) shows that the audiovisual mapping parameters are grouped in a smaller subspace, which suggests that there is an emotion-dependent structure that is preserved from across sentences. The results suggest that this internal structure seems to be easy to model when prosodic-features are used to estimate the audiovisual mapping. The results also reveal that the correlation levels within a sentence vary according to broad phonetic properties presented in the sentence. Consonants, especially unvoiced and fricative sounds, present the lowest correlation levels. Likewise, the results show that facial gestures are linked at different resolutions. While the orofacial area is locally connected with the speech, other facial gestures such as eyebrow motion are linked only at the sentence-level. The results presented here have important implications for applications such as facial animation and multimodal emotion recognition.

The effects of visual beats on prosodic prominence: Acoustic analyses, auditory perception and visual perception

Speakers employ acoustic cues (pitch accents) to indicate that a word is important, but may also use visual cues (beat gestures, head nods, eyebrow movements) for this purpose. Even though these acoustic and visual cues are related, the exact nature of this relationship is far from well understood. We investigate whether producing a visual beat leads to changes in how acoustic prominence is realized in speech, and whether it leads to changes in how prominence is perceived by observers. For Experiment I (“making beats”) we use an original experimental paradigm in which speakers are instructed to realize a target sentence with different distributions of acoustic and visual cues for prominence. Acoustic analyses reveal that the production of a visual beat indeed has an effect on the acoustic realization of the co-occurring speech, in particular on duration and the higher formants ( F 2 and F 3), independent of the kind of visual beat and of the presence and position of pitch accents. In Experiment II (“hearing beats”), it is found that visual beats have a significant effect on the perceived prominence of the target words. When a speaker produces a beat gesture, an eyebrow movement or a head nod, the accompanying word is produced with relatively more spoken emphasis. In Experiment III (“seeing beats”), finally, it is found that when participants see a speaker realize a visual beat on a word, they perceive it as more prominent than when they do not see the beat gesture.

A Microswitch-based Program to Enable Students with Multiple Disabilities to Choose among Environmental Stimuli

Students with multiple disabilities, such as severe to profound mental retardation combined with motor and visual impairment, are usually unable to engage in constructive activity or play a positive role in their daily context. Microswitches are technical tools that may help them improve their status by allowing them to control environmental events with small and simple responses suitable to their condition. Recently, a pilot study was conducted to assess such a program with a boy who had multiple disabilities, including total blindness and minimal motor behavior. The results were encouraging, in that the boy became quite active and chose stimuli he was expected to prefer and bypassed those he was not, thus exhibiting purposeful choice behavior. The present study reported in this paper served as a replication and extension of the pilot study, with two new students who had multiple disabilities. These students relied on a vocalization response to activate the microswitch as opposed to the eyebrow movements used in the pilot study. A computer system was used to handle sets of expected preferred and nonpreferred stimuli: it presented brief samples of the stimuli and eventually turned them on for longer periods of time if students selected them. The results of this study found encouraging evidence for the possibility of using specifically developed microswitch programs to enable individuals with minimal residual vision, limited motor behavior, and circumscribed intellectual ability to choose among environmental stimuli. Further research may help determine the generalizability of these results and assess ways of integrating different microswitch programs into the daily educational schedule of such individuals.

A novel mixture of propofol, alfentanil, and lidocaine for regional block with monitored anesthesia care in ophthalmic surgery

Study Objective The purpose of this study is to determine the efficacy and safety of sedation/analgesia using a mixture of propofol, alfentanil, and lidocaine. Design A retrospective case review was undertaken. Setting This study took place at a university medical center. Patients Eighty-nine American Society of Anesthesiologists physical status 1, 2, and 3 adult patients undergoing ophthalmic surgery with regional block and monitored anesthesia care were studied. Intervention Six milliliters of propofol, 2 mL of alfentanil, and 2 mL of 2% lidocaine (6-2-2 mixture) were freshly mixed. The bolus dose was determined based on the patients' age: 5 μg/kg of alfentanil (and 0.3 mg/kg of propofol) for patients older than 75 years; the dose increased 1 μg/kg per 10-year decrease in age; and up to 9 μg/kg of alfentanil (0.54 mg/kg of propofol) for patients younger than 45 years. Regional block was performed at 1 minute after bolus completion. Blood pressure (BP), Sa o 2 , electrocardiogram, capnography, clinical signs of sedation, responses to block, need for airway support, nausea and vomiting (N/V), pain due to propofol infusion, recall, and patient and surgeon satisfaction were recorded. Measurements and Main results Seventy-eight percent of patients achieved analgesia and sedation without adverse response to the block. Twelve percent achieved good analgesia and sedation with only eyebrow movement upon needle insertion. Twenty-seven percent had respiratory depression but were able to follow commands and maintain adequate ventilation. Two percent had brief apnea alleviated by chin lift or jaw thrust. None had pain because of propofol infusion or N/V. Before sedation, average systolic BP was significantly increased ( P < 0.0001) compared with baseline. After sedation and block, systolic BP decreased 6% from baseline ( P < 0.005). Conclusion Adjusted for age and weight, the dose of the 6-2-2 mixture met the sedation requirements for most patients. With a low incidence of need for airway support, no pain during infusion, and no N/V, this novel mixture of propofol, alfentanil, and lidocaine provided adequate analgesia and sedation as well as hemodynamic stability for ophthalmic surgery under regional block.

A microswitch-based programme to enable a boy with multiple disabilities and minimal motor behaviour to choose among environmental stimuli

Purpose. The present study assessed a microswitch-based programme designed to enable a nine-year-old boy with multiple disabilities and minimal motor behaviour to choose among environmental stimuli.Method. The programme relied on the boy's eyebrow movements, a microswitch to detect such movements, and a computer system to handle sets of allegedly preferred and non-preferred stimuli. For each of the 30 stimuli available in every session, the computer system provided a sample/preview of 2 – 3 seconds. During intervention, the boy's responding to a stimulus sample (i.e., moving the eyebrow and activating the microswitch) led the computer system to turn on that stimulus for 25 seconds. The boy's lack of responding led the computer system to present the next stimulus sample.Results. Intervention data showed that the boy's choice making (i.e., responding to a stimulus sample or avoiding to do so) was largely consistent with the alleged preference value of the stimuli, that is, he tended to choose the stimuli that were considered preferred and to bypass those considered non-preferred for him. The same performance was maintained during post-intervention checks, occurring 6 and 10 weeks after the end of the intervention, and over sets of stimuli not used during the intervention.Conclusions. The present microswitch-based programme may be considered useful to allow students with multiple disabilities an active choice (control) role in a stimulation-enriched environment.

How Children and Adults Produce and Perceive Uncertainty in Audiovisual Speech

We describe two experiments on signaling and detecting uncertainty in audiovisual speech by adults and children. In the first study, utterances from adult speakers and child speakers (aged 7-8) were elicited and annotated with a set of six audiovisual features. It was found that when adult speakers were uncertain they were more likely to produce fillers, delays, high intonation, eyebrow movements, and "funny faces." The basic picture for the child speakers was somewhat similar, in that the presence of certain audiovisual cues correlated with uncertainty, but the differences were relatively small and less often significant. In the second study both adult and child judges watched responses from adult and child speakers selected from the first study to find out whether they were able to correctly estimate a speakers' level of uncertainty. It was found that both child and adult judges gave more accurate scores for answers from adult speakers than from child speakers and that child judges overall provided less accurate scores than adult judges.

Detecting deception in pain expressions: the structure of genuine and deceptive facial displays

Clinicians tend to assign greater weight to non-verbal expression than to patients' self-report when judging the location and severity of pain. Judgments can misrepresent the actual experience because patients can successfully alter their pain expressions. The present research provides a basis for discriminating genuine and deceptive pain expressions by expanding detailed accounts of facial expressions to include previously unexamined variables, including study of temporal patterns and contiguity of facial actions as well as the occurrence of specific deception cues. Low back patients' facial expressions ( n=40) were videotaped at rest and while undergoing a painful straight leg raise with instructions to: (1) genuinely express their pain, or (2) pretend that it did not hurt. As well, they were asked to fake pain without moving. The Facial Action Coding System was used to describe and quantify facial activity. The different types of expression were compared on the frequency, type, intensity, temporal pattern and contiguity of facial actions, as well as on the frequency of specific deception cues. Findings confirmed the difficulty of discriminating the facial expressions, but indicated that faked pain expressions show a greater number of pain-related and non-pain-related actions, have a longer peak intensity and overall duration, and the facial actions observed tend to be less temporally contiguous than are those in genuine pain expressions. The differences between masked pain and neutral expressions were subtle, with a greater frequency of mouth opening and residual eyebrow movement in masked pain expressions. Thus, there is an empirical basis for discriminating genuine and deceptive facial displays.

Recognition of facial expressions in a Japanese monkey (Macaca fuscata) and humans (Homo sapiens)

Recognition of facial expressions by a Japanese monkey and two humans was studied. The monkey subject matched 20 photographs of monkey facial expressions and 20 photographs of human facial expressions. Humans sorted the same pictures. Matching accuracy by the monkey was about 80% correct for both human and monkey facial expressions. The confusion matrices of those facial expressions were analyzed by a multi-dimensional scaling procedure (MDSCAL). The resulting MDS plots suggested that the important cues in recognizing facial expressions of monkeys were “thrusting the mouth” and ‘raising the eyebrows.” Comparison of the MDS plots by the monkey subject with those by human subjects suggested that the monkey categorized the human “happiness” faces. This may suggest that the monkey has an ability to recognize human smile face even though it is learned. However, the monkey did not differentiate the human “anger/disgust” faces from the human “sad” faces, while human subjects clearly did. This may correlate with the lack of eyebrow movement in monkeys.

Long-term complications of the galeal-frontalis myofascial flap in craniofacial surgery

Frontal contour irregularities, sensory deficit of the forehead and loss of frontalis muscle function may occur following elevation and use of the galeal-frontalis flap. These three problems were evaluated retrospectively using a questionnaire and clinical examination in 48 cases. Frontal contour deformities were seen in 9 cases. Hypoesthesia of the forehead and weakness of eyebrow movement were seen in 30 and 35 patients, respectively. It was found that a number of patients had not noticed their deformities before the examination. The postoperative complications of the galeal-frontalis flap seemed to be of no concern to the patients.

Are the accessory facial movements of the stutterer learned behaviours?

The purpose of the present study was to explore the accessory nonverbal behaviours emitted by stutterers when their speech was fluent, normally disfluent, or stuttered. Subjects were 25 stutterers who were required to speak spontaneously for a 2-min. period. Seven types of nonverbal behavior were observed. Significant differences among the three speech categories were obtained for jaw movements, mouth movements, forehead movements, eyebrow movements, and head movements. Eyelid movements and eye blinks were nonsignificant. The results are discussed with respect to the various functions that can be attributed to nonverbal behaviour in stuttering.

On the function of facial muscles in different behavioral situations. A study based on muscle morphology and electromyography.

AbstractThe anatomy and function of the facial muscles of Macaca cyclopis, M. fuscata, M. irus, M. mulatta, and M. nemestrina have been correlated and analyzed by means of electromyograms made by a 12‐channel electromyograph. We made records only of those areas where there is one layer of facial muscles. Morphologically the muscular differences among the species are insignificant for the recording of EMGs. Functionally, the facial muscles moving the eyebrows and ears differ under certain conditions. There is no muscle activity during sleep: during dozing, only a slight lowering of the eyebrows was noted and sometimes an extremely low level of watching behavior. The ear and eyebrow movements during the watching responses differ among the species. The basis for threatening behavior is watching: but most species raise the ears and eyebrows more extremely in threat, and other muscle activities differ among the species. In all species, the facial muscles are contracted in fear to a greater or lesser degree. Some muscular responses, too brief to be detected by observers but recorded by the EMG, can thus be analyzed and compared with the expressions of observed behavior.