Facial Signals Research Articles

Emotionally Assisted Human–Robot Interaction Using a Wearable Device for Reading Facial Expressions

In this paper, we introduce a novel paradigm of emotionally assisted interaction between humans and robots. We present a personal wearable device that can be worn on the side of the face to unobtrusively and continuously detect physiological signals that are a mixture of facial electromyographic signals. Through real-time pattern classification, facial expressions can be identified from them and interpreted as positive and negative responses from a human. We report on successful facial expression identification using Independent Component Analysis and an Artificial Neural Network, and show the design of the interface device that can be used for coaching a real robot.

Intensity of Smiling and Attractiveness as Facial Signals of Trustworthiness in Women

Facial attractiveness is associated with a variety of positive social characteristics including trustworthiness. Variations in smiling, such as the appearance of the Duchenne marker and increased intensity of expression, have likewise been linked with positive judgments of trustworthiness. The study investigated the interaction of the effects of models' attractiveness and their smiling intensity on impressions of perceived trustworthiness. Participants rated the attractiveness and expressivity of neutral, low intensity, and high intensity smiling images of 45 women models. These images were also presented to a second group of participants who rated trustworthiness. Repeated measures analysis of covariance of the effects of attractiveness and manipulated smile intensity on trustworthiness indicated a main effect for smile intensity: increased smile intensity was associated with greater trustworthiness. Attractiveness also contributed to rated trustworthiness independently of smiling intensity. Results suggested there is an additional contribution of facial expression in creating social impressions of trustworthiness.

얼굴 영상 및 음성신호 측정을 통한 신장 수지침 효과 분석 기법의 제안

In this paper, we would like to propose techniques to analyze effect according to stimulation kidney associated hand acupuncture by applying technique to measure changes of facial image and voice signal. To this end, we measured color change of JIGAK(jaw) area associated kidney in facial image and voice signal stimulation before and after of kidney associated hand acupuncture. In addition, we measured changes of the first formant frequency bandwidth and Shimmer to element of voice signal analysis in connection with kidney in experiment. We can be measured reduction of the first formant frequency bandwidth and Shimmer, black of JIGAK area according to stimulation of kidney associated hand acupuncture. Finally, we would like to demonstrate objective effect of kidney associated hand acupuncture through the analysis of statistical significance by measurement techniques of facial image and voice signal.

Facilitating Play Through Communication: Significance of Teeth Exposure in the Gorilla Play Face

Primate facial expressions (FEs) likely play an important role in primate society: through facial signals, individuals can potentially send and receive information and may benefit from coordinating their behavior accordingly. Many primates use a relaxed open mouth (ROM) facial display or “play face” (PF) during play behavior, where the mouth is open but teeth are covered. In addition to this conventional PF, however, Western Lowland gorillas (Gorilla gorilla gorilla) also use a full PF where the upper teeth are exposed. As the teeth are similarly exposed in the bared-teeth expression (which is a signal of appeasement, submission and/or affiliation), the full PF may be a blend of the PF and bared-teeth face, and have a different signal function to the PF alone. Focal animal sampling of captive Western Lowland gorillas (N=10) showed that the full PF was more often observed in intense rather than gentle play, and intense play bouts that featured the full PF were longer than those that featured only the PF. Both expressions were associated with an increase in affinitive behavior between sender and receiver postplay, but only the full PF was associated with an increase higher than that of play alone. Overall, the findings suggest that the full PF has an additional role in coordinating and maintaining play, possibly though reducing uncertainty in the receiver and confirming that play is only play.

Emotion Detection System Based on Speech and Facial Signals

This paper introduces the present status of speech emotion detection. In order to improve the emotion recognition rate of single mode, the bimodal fusion method based on speech and facial expression is proposed. First, we establishes emotional database include speech and facial expression. For different emotions, calm, happy, surprise, anger, sad, we extract ten speech parameters and use the PCA method to detect the speech emotion. Then we analyze the bimodal emotion detection of fusing facial expression information. The experiment results show that the emotion recognition rate with bimodal fusion is about 6 percent points higher than the recognition rate with only speech prosodic features

Emotion Detection System Based on Speech and Facial Signals

Emotion Detection System Based on Speech and Facial Signals

Fear and happiness in the eyes: An intra-cerebral event-related potential study from the human amygdala

We present the response pattern of intracranial event-related potentials (ERPs) recorded from depth-electrodes in the human amygdala (four patients) to faces or face parts encoding fearful, happy or neutral expressions. The amygdala showed increased amplitude ERPs (from 200 to 400 ms post-stimulus) in response to the eye region of the face compared to whole faces and to the mouth region. In particular, a strong emotional valence effect was observed, both at group and at single-subject level, with a preferential response to fearful eyes respect to every other stimulus category from 200 to 400 ms after stimulus presentation. A preferential response to smiling eyes compared to happy faces and smiling mouths was also observed at group level from 300 to 400 ms post-stimulus presentation. A complementary time–frequency analysis was performed showing that an increase in the theta frequency band (4–7 Hz) accounted for the main event-related band power (ERBP) change during the 200–500 ms post stimulus interval. The analysis of the ERBPs changes according to their emotional valence showed a strong increase in theta ERBP to fearful eyes, which was higher respect to any other facial stimulus. Moreover, theta ERBP increase to “smiling eyes” was larger respect with that evoked by smiling mouths and whole happy faces. Minimal post-stimulus ERBPs changes were evoked by neutral stimuli. These data are consistent with a special role of the amygdala in processing facial signals, both with negative and positive valence, conveyed by the eye region of the face.

Cosmetics as a Feature of the Extended Human Phenotype: Modulation of the Perception of Biologically Important Facial Signals

Research on the perception of faces has focused on the size, shape, and configuration of inherited features or the biological phenotype, and largely ignored the effects of adornment, or the extended phenotype. Research on the evolution of signaling has shown that animals frequently alter visual features, including color cues, to attract, intimidate or protect themselves from conspecifics. Humans engage in conscious manipulation of visual signals using cultural tools in real time rather than genetic changes over evolutionary time. Here, we investigate one tool, the use of color cosmetics. In two studies, we asked viewers to rate the same female faces with or without color cosmetics, and we varied the style of makeup from minimal (natural), to moderate (professional), to dramatic (glamorous). Each look provided increasing luminance contrast between the facial features and surrounding skin. Faces were shown for 250 ms or for unlimited inspection time, and subjects rated them for attractiveness, competence, likeability and trustworthiness. At 250 ms, cosmetics had significant positive effects on all outcomes. Length of inspection time did not change the effect for competence or attractiveness. However, with longer inspection time, the effect of cosmetics on likability and trust varied by specific makeup looks, indicating that cosmetics could impact automatic and deliberative judgments differently. The results suggest that cosmetics can create supernormal facial stimuli, and that one way they may do so is by exaggerating cues to sexual dimorphism. Our results provide evidence that judgments of facial trustworthiness and attractiveness are at least partially separable, that beauty has a significant positive effect on judgment of competence, a universal dimension of social cognition, but has a more nuanced effect on the other universal dimension of social warmth, and that the extended phenotype significantly influences perception of biologically important signals at first glance and at longer inspection.

Effects of Acute Tryptophan Depletion on Prefrontal-Amygdala Connectivity While Viewing Facial Signals of Aggression

BackgroundReduced levels of serotonin (5-HT) within prefrontal cortex (PFC)–amygdala circuits have long been implicated in impulsive aggression. However, whether lowering 5-HT alters the dynamic interplay between the PFC and the amygdala has not been directly tested in humans. It is known that manipulating 5-HT via acute tryptophan depletion (ATD) causes variable effects on brain responses to a variety of emotional stimuli, but it remains unclear whether ATD affects functional connectivity in neural networks involved in processing social signals of aggression (e.g., angry faces).MethodsThirty healthy individuals were enrolled in a randomized, double-blind, placebo-controlled ATD study. On each treatment, brain responses to angry, sad, and neutral faces were measured with functional magnetic resonance imaging. Two methods (psycho-physiological-interaction in a general linear model and dynamic causal modeling) were used to assess the impact of ATD on the functional connectivity between PFC and amygdala.ResultsData from 19 subjects were available for the final analyses. A whole-brain psycho-physiological-interaction in a general linear model showed that ATD significantly modulated the connectivity between the amygdala and two PFC regions (ventral anterior cingulate cortex and ventrolateral PFC) when processing angry vs. neutral and angry vs. sad but not sad vs. neutral faces. Dynamic causal modeling corroborated and extended these findings by showing that 5-HT depletion reduced the influence of processing angry vs. neutral faces on circuits within PFC and on PFC–amygdala pathways.ConclusionsWe provide strong support for neurobiological accounts positing that 5-HT significantly influences PFC–amygdala circuits implicated in aggression and other affective behaviors.

Expression of smoothened in mouse embryonic maxillofacial development

Hedgehog (Hh) signaling plays many key roles in the development of Drosophila and vertebrate embryos including regulation of craniofacial development. The seven-transmembrane protein, smoothened (Smo) transduces the Hh signal across the plasma membrane as an essential receptor of PTCHED1/2. There are few studies that evaluate the detailed expression of Smo in mouse embryonic craniofacial development. We investigated the expression patterns of Smo during murine embryonic craniofacial development using in situ hybridization (ISH), studies of whole-mounts and sections, immunohistochemistry, quantitative real time PCR, and Western blot analysis. We found that Smo mRNA was expressed in the face of mouse embryos at 11 and 12.5 days post coitum (dpc). After 13.5 dpc, the expression decreased to a low level and was faintly detected after birth. Smo protein could be detected also in embryos at 11, 12.5, and 14.5 dpc. After 15.5 dpc, the expression was very faint and paralleled the gene expression studies. No expression was detected in whisker follicle during facial development and faint signal was detected in Meckel's cartilage. These findings concerning Smo expression should guide further investigation of sonic Hh signaling pathway gene function during maxillofacial development.

Facial Wrinkles May Signal Bone Mineral Loss

Facial Wrinkles May Signal Bone Mineral Loss

Personality influences the neural responses to viewing facial expressions of emotion

Cognitive research has long been aware of the relationship between individual differences in personality and performance on behavioural tasks. However, within the field of cognitive neuroscience, the way in which such differences manifest at a neural level has received relatively little attention. We review recent research addressing the relationship between personality traits and the neural response to viewing facial signals of emotion. In one section, we discuss work demonstrating the relationship between anxiety and the amygdala response to facial signals of threat. A second section considers research showing that individual differences in reward drive (behavioural activation system), a trait linked to aggression, influence the neural responsivity and connectivity between brain regions implicated in aggression when viewing facial signals of anger. Finally, we address recent criticisms of the correlational approach to fMRI analyses and conclude that when used appropriately, analyses examining the relationship between personality and brain activity provide a useful tool for understanding the neural basis of facial expression processing and emotion processing in general.

Familiarity affects the assessment of female facial signals of fertility by free-ranging male rhesus macaques

Animals signal their reproductive status in a range of sensory modalities. Highly social animals, such as primates, have access not only to such signals, but also to prior experience of other group members. Whether this experience affects how animals interpret reproductive signals is unknown. Here, we explore whether familiarity with a specific female affects a male's ability to assess that female's reproductive signals. We used a preferential looking procedure to assess signal discrimination in free-ranging rhesus macaques, a species in which female facial luminance covaries with reproductive status. We collected images of female faces throughout the reproductive cycle, and using faecal hormone analysis to determine ovulation, categorized images as coming from a female's pre-fertile, ovulating, or post-fertile period. We printed colour-calibrated stimuli of these faces, reproducing stimuli perceptually the same in colour and luminance to the original appearance of females. These images were presented to males who were either unfamiliar or familiar with stimuli females. Overall, males distinguished ovulatory from pre-ovulatory faces. However, a significant proportion of males did so only among males familiar with stimuli females. These experiments demonstrate that familiarity may increase a receiver's ability to use a social partner's signals to discern their reproductive status.

The language void: the need for multimodality in primate communication research

Theories of language evolution often draw heavily on comparative evidence of the communicative abilities of extant nonhuman primates (primates). Many theories have argued exclusively for a unimodal origin of language, usually gestural or vocal. Theories are often strengthened by research on primates that indicates the absence of certain linguistic precursors in the opposing communicative modality. However, a systematic review of the primate communication literature reveals that vocal, gestural and facial signals have attracted differing theoretical and methodological approaches, rendering cross-modal comparisons problematic. The validity of the theories based on such comparisons can therefore be questioned. We propose that these a priori biases, inherent in unimodal research, highlight the need for integrated multimodal research. By examining communicative signals in concert we can both avoid methodological discontinuities as well as better understand the phylogenetic precursors to human language as part of a multimodal system.

The Importance of Facial Features and Their Spatial Organization for Attractiveness is Modulated by Gender

Many studies suggest that facial attractiveness signals mate quality. Fewer studies argue that the preference criteria emerge as a by-product of cortical processes. One way or the other, preference criteria should not be necessarily identical between female and male observers because either their preferences may have different evolutionary roles or they may even be due to known differences in visiospatial skills and brain function lateralization (ie, advantages favoring males' inability to determine spatial relations despite distracting information). The goal of this study was to assess sex differences in face attractiveness judgments by estimating the importance of facial features and their spatial organization. To this end, semipartial correlations were measured between intact-face preferences and preferences based on specific facial parts (eyes, nose, mouth, and hairstyle) or preferences based more on configuration (as reflected by low spatial frequency images). The results show strategy modulations by both observers' and faces' genders. In general, the association between intact-face preferences and parts-based preferences was significantly higher for female compared with male participants. For female faces, males' preferences were more strongly associated with their low spatial frequency preferences than were those of females. The two genders' strategies were more similar when judging male faces, and males performed more criteria modifications across face gender. The similarities between sexes regarding male faces are in line with previous studies that showed higher assignment of importance among men to attractiveness. Moreover, the results may suggest that men adjust their strategy to assess the danger of other males as potential rivals for mates.

Playing with the face: Playful facial “chattering” and signal modulation in a monkey species (Theropithecus gelada).

Darwin (1872), in The expression of emotions in man and animals, underlined that human facial expressions represent a shared heritage of our species with nonhuman primates. Play is a fertile field to examine the role of facial expressions that we share with our common ancestors because the primate play face is homologous to human laughter. Here, we focus on the use of two playful expression variants (PF: play face, mouth opened with only the lower teeth exposed; FPF: full play face, lower/upper teeth and gums exposed via the actively retraction of the upper lip) in Theropithecus gelada. During ontogeny PF was replaced by FPF; in older subjects PF was virtually absent. The ontogenetic transition appears to reflect the phylogenetic sequence of the two playful displays with FPF considered a derived form of PF. This age-trend bias of facial displays is probably due to their different roles in communication. The correspondence between facial signals emitted and elicited is a valuable criterion to evaluate playmates' attentional state. Adults were more sensitive than immatures in responding to the play faces of others. Probably, previous playful experience, social competence, and neural circuit maturation are at the basis of adult sensitiveness. Similar to humans, where unconscious laughing is deserved for close friends and/or relatives, FPF was extremely frequent during gelada mother-offspring play. Probably, under some intimate circumstances, facial displays should be primarily linked to the spontaneous expression of emotional states of the sender more than to the strategic transfer of actual information to the receiver.

Interaction between trait anxiety and trait anger predict amygdala reactivity to angry facial expressions in men but not women

The amygdala is critically involved in mediating physiological and behavioral responses to threat. In particular, neuroimaging research indicates that the amygdala is highly responsive to facial signals of threat such as fearful and angry expressions. However, individuals differ substantially in both their relative sensitivity to threat and the magnitude of amygdala reactivity to facial signals of threat. Here, we report the novel finding that individual differences in trait anger are positively correlated with bilateral dorsal amygdala reactivity to angry facial expressions, but only among men with elevated trait anxiety scores. These findings add to the growing body of evidence indicating that variability in personality traits contribute to individual differences in threat-related amygdala reactivity and further suggest that heightened amygdala reactivity to angry faces may be uniquely involved in the expression of reactive aggression in men.

Are positive vocalizations perceived as communicating happiness across cultural boundaries?

Laughter communicates a feeling of enjoyment across cultures, while non-verbal vocalizations of several other positive emotions, such as achievement or sensual pleasure, are recognizable only within, but not across, cultural boundaries. Are these positive vocalizations nevertheless interpreted cross-culturally as signaling positive affect? In a match-to-sample task, positive emotional vocal stimuli were paired with positive and negative facial expressions, by English participants and members of the Himba, a semi-nomadic, culturally isolated Namibian group. The results showed that laughter was associated with a smiling facial expression across both groups, consistent with previous work showing that human laughter is a positive, social signal with deep evolutionary roots. However, non-verbal vocalizations of achievement, sensual pleasure, and relief were not cross-culturally associated with smiling facial expressions, perhaps indicating that these types of vocalizations are not cross-culturally interpreted as communicating a positive emotional state, or alternatively that these emotions are associated with positive facial expression other than smiling. These results are discussed in the context of positive emotional communication in vocal and facial signals. Research on the perception of non-verbal vocalizations of emotions across cultures demonstrates that some affective signals, including laughter, are associated with particular facial configurations and emotional states, supporting theories of emotions as a set of evolved functions that are shared by all humans regardless of cultural boundaries.

Flow of affective information between communicating brains

When people interact, affective information is transmitted between their brains. Modern imaging techniques permit to investigate the dynamics of this brain-to-brain transfer of information. Here, we used information-based functional magnetic resonance imaging (fMRI) to investigate the flow of affective information between the brains of senders and perceivers engaged in ongoing facial communication of affect. We found that the level of neural activity within a distributed network of the perceiver's brain can be successfully predicted from the neural activity in the same network in the sender's brain, depending on the affect that is currently being communicated. Furthermore, there was a temporal succession in the flow of affective information from the sender's brain to the perceiver's brain, with information in the perceiver's brain being significantly delayed relative to information in the sender's brain. This delay decreased over time, possibly reflecting some ‘tuning in’ of the perceiver with the sender. Our data support current theories of intersubjectivity by providing direct evidence that during ongoing facial communication a ‘shared space’ of affect is successively built up between senders and perceivers of affective facial signals.

Involvement of IGF-1/IGFBP-3 signaling on the conspicuousness of facial pores

Conspicuous facial pores are one type of serious esthetic defects for many women. We previously reported that the severity of impairment of skin architecture around facial pores correlates well with the appearance of facial pores in several ethnic groups. In our last report, we showed that serum levels of insulin-like growth factor-1 (IGF-1) correlate well with facial pore size and with the severity of impairment of epidermal architecture around facial pores. However, our results could not fully explain the implication between facial pores and IGF signaling. In this study, we conducted a histological analysis of facial skin to determine whether potential changes in IGF-1 availability occur in the skin with or without conspicuous pores. Immunohistochemical observations showed that expression of insulin-like growth factor binding protein-3 (IGFBP-3) is limited to the suprapapillary epidermis around facial pores and to basal cells of rete pegs without tips in epidermis with conspicuous pores. In contrast, in basal cells of skin without conspicuous pores, IGFBP-3 expression is very low. Ki-67 and IGF-1 receptor-positive cells are abundant in basal cells in the tips of the rete pegs in skin with typical epidermal architecture around facial pores. No obvious differences were observed in the expression of filaggrin, involucrin, K1, K6 or K17 in skin with or without conspicuous pores. However, increased expression of K16 was observed in skin with conspicuous pores suggesting hyperproliferation. These results suggest that the IGF-1/IGFBP-3 signaling pathway is involved in the formation of conspicuous facial pores due to the epidermal architecture around facial pores.