Avatar Faces Research Articles

Video-Driven Creation of Virtual Avatars by Component-based Transferring of Facial Expressions *

This paper proposes an efficient and economic video-driven technique that enables the instant creation of a large diversity of virtual avatars and the automatic syntheses of vivid facial animations. The proposed technique addresses the expression transferring problem which transfers a given facial expression of a source human character to the corresponding one of a synthesized avatar. In tackling the expression transferring problem, we propose a component-based approach which is more appealing than the existing approaches which treat the whole face as a single unit for expression transferring. Our approach acquires a much higher diversity in synthesizing virtual avatars and facial expressions by composing the synthesized target face from the facial components of different avatars. The proposed method achieves a good way to transfer the synthesizing parameters acquired from the source human face to those of the target avatar face which complies well with the person-specific characteristics of the target avatar. Additionally, the removal of color inconsistencies among the facial components from different avatars is also well handled. Some experimental results are demonstrated to show that the proposed method can achieve interesting and colorful transfers of facial expressions and synthesize a large diversity of virtual avatars instantly.

Children and adults use attractiveness as a social cue in real people and avatars

Observing social interactions between children and adults is a major method in the toolkit of psychologists who examine social development and social relationships. Although this method has revealed many interesting phenomena, it cannot determine the effect of behavior independent of other traits. Research on the role of attractiveness in social development provides an example of this conundrum: Are attractive and unattractive children/adults treated differently because of their attractiveness (independent of their behavior), do they behave differently and thus elicit differential treatment, or both? Virtual world and avatar-based technologies allow researchers to control the social behaviors of targets; however, whether children and adults use the facial attractiveness of avatars as a social cue in the same way as they do with real peers is currently unknown. Using Mii avatars from the popular Nintendo Wii video game console, Study 1 found that the facial attractiveness ratings of real people strongly predicted the attractiveness ratings of avatar faces based on the former group. Study 2 revealed that adults (n=46) and children (n=42) prefer attractive avatars as social partners. The results of this set of methodological studies may help to clarify future research on the relationship between attractiveness and behavior throughout the lifespan. Furthermore, the use of avatars may allow studies to experimentally examine the effects of attractiveness in situations where such research is not ethical (e.g., peer victimization).

Cultural background modulates how we look at other persons’ gaze

The current study investigated the role of cultural norms on the development of face-scanning. British and Japanese adults’ eye movements were recorded while they observed avatar faces moving their mouth, and then their eyes toward or away from the participants. British participants fixated more on the mouth, which contrasts with Japanese participants fixating mainly on the eyes. Moreover, eye fixations of British participants were less affected by the gaze shift of the avatar than Japanese participants, who shifted their fixation to the corresponding direction of the avatar’s gaze. Results are consistent with the Western cultural norms that value the maintenance of eye contact, and the Eastern cultural norms that require flexible use of eye contact and gaze aversion.

Category Processing and the human likeness dimension of the Uncanny Valley Hypothesis: Eye-Tracking Data.

The Uncanny Valley Hypothesis (Mori, 1970) predicts that perceptual difficulty distinguishing between a humanlike object (e.g., lifelike prosthetic hand, mannequin) and its human counterpart evokes negative affect. Research has focused on affect, with inconsistent results, but little is known about how objects along the hypothesis’ dimension of human likeness (DHL) are actually perceived. This study used morph continua based on human and highly realistic computer-generated (avatar) faces to represent the DHL. Total number and dwell time of fixations to facial features were recorded while participants (N = 60) judged avatar versus human category membership of the faces in a forced choice categorization task. Fixation and dwell data confirmed the face feature hierarchy (eyes, nose, and mouth in this order of importance) across the DHL. There were no further findings for fixation. A change in the relative importance of these features was found for dwell time, with greater preferential processing of eyes and mouth of categorically ambiguous faces compared with unambiguous avatar faces. There were no significant differences between ambiguous and human faces. These findings applied for men and women, though women generally dwelled more on the eyes to the disadvantage of the nose. The mouth was unaffected by gender. In summary, the relative importance of facial features changed on the DHL’s non-human side as a function of categorization ambiguity. This change was indicated by dwell time only, suggesting greater depth of perceptual processing of the eyes and mouth of ambiguous faces compared with these features in unambiguous avatar faces.

Wavelet-Based Multiscale Adaptive LBP with Directional Statistical Features for Recognizing Artificial Faces

Recognizing avatar faces is a very important issue for the security of virtual worlds. In this paper, a novel face recognition technique based on the wavelet transform and the multiscale representation of the adaptive local binary pattern (ALBP) with directional statistical features is proposed to increase the accuracy rate of recognizing avatars in different virtual worlds. The proposed technique consists of three stages: preprocessing, feature extraction, and recognition. In the preprocessing and feature extraction stages, wavelet decomposition is used to enhance the common features of the same subject of images and the multiscale ALBP (MALBP) is used to extract representative features from each facial image. Then, in the recognition stage the wavelet MALBP (WMALBP) histogram dissimilarity with statistical features of each test image and each class model is used within the nearest neighbor classifier to improve the classification accuracy of the WMALBP. Experiments conducted on two virtual world avatar face image datasets show that our technique performs better than LBP, PCA, multiscale local binary pattern, ALBP, and ALBP with directional statistical features (ALBPF) in terms of the accuracy and the time required to classify each facial image to its subject.

The development of the uncanny valley in infants

When adults view very realistic humanoid robots or computer avatars they often exhibit an aversion to them. This phenomenon, known as the "uncanny valley," is assumed to be evolutionary in origin, perhaps tapping into modules for disgust or attractiveness that detect violations of our normal expectations regarding social signals. Here, we test an alternative hypothesis that the uncanny valley is developmental in origin and, thus, that specific early experience with real human faces leads to its eventual emergence. To test this idea, we measured visual preferences in response to all possible pairs of a human face, realistic avatar face, and an unrealistic avatar face in groups of 6-, 8-, 10-, and 12-month-old infants. Consistent with the developmental hypothesis, we found that the uncanny valley effect emerges at 12 months of age suggesting that perceptual experience with real human faces is critical to its emergence.

Gender effects in pain detection: Speed and accuracy in decoding female and male pain expressions

The ability to detect facial expressions of pain is crucial in eliciting prosocial behaviors towards the individual experiencing pain. Previous studies have shown that the sufferers’ gender can affect the observers’ explicit judgment of the pain face, thus suggesting its possible influence on pain decoding. The present study investigates whether the sufferer’s gender affects the observer’s reflexive or implicit detection of facial expression of pain. More specifically, we used implicit measures to test whether observers detect pained expression more quickly or accurately on male or female faces. In three experimental studies, we devised a set of stimuli using computer-generated faces. In Experiment 1, prototypical female and male avatars with different facial expressions (pain, anger, disgust, and neutral) were displayed, while subjects’ ( N = 34) accuracy and speed at identifying the expressions were recorded. In Experiment 2, participants ( N = 56) watched videos of the avatars displaying dynamic expressions and had to quickly and accurately identify each expression. In Experiment 3, participants ( N = 38) were shown an androgynous avatar face showing different expressions and were asked to identify the face as either female or male. Overall, we found that the target’s gender affected the observer’s reflexive decoding of the facial expression of pain. Specifically, the results showed that participants, regardless of their gender, were slower and less accurate in recognizing pain expressions (but not other expressions) on female faces. Furthermore, androgynous faces displaying pained expressions were more likely to be categorized as male than female. Several potential explanations are discussed.

Virtual faces as a tool to study emotion recognition deficits in schizophrenia

Studies investigating emotion recognition in patients with schizophrenia predominantly presented photographs of facial expressions. Better control and higher flexibility of emotion displays could be afforded by virtual reality (VR). VR allows the manipulation of facial expression and can simulate social interactions in a controlled and yet more naturalistic environment. However, to our knowledge, there is no study that systematically investigated whether patients with schizophrenia show the same emotion recognition deficits when emotions are expressed by virtual as compared to natural faces. Twenty schizophrenia patients and 20 controls rated pictures of natural and virtual faces with respect to the basic emotion expressed (happiness, sadness, anger, fear, disgust, and neutrality). Consistent with our hypothesis, the results revealed that emotion recognition impairments also emerged for emotions expressed by virtual characters. As virtual in contrast to natural expressions only contain major emotional features, schizophrenia patients already seem to be impaired in the recognition of basic emotional features. This finding has practical implication as it supports the use of virtual emotional expressions for psychiatric research: the ease of changing facial features, animating avatar faces, and creating therapeutic simulations makes validated artificial expressions perfectly suited to study and treat emotion recognition deficits in schizophrenia.

Effects of damping head movement and facial expression in dyadic conversation using real–time facial expression tracking and synthesized avatars

When people speak with one another, they tend to adapt their head movements and facial expressions in response to each others' head movements and facial expressions. We present an experiment in which confederates' head movements and facial expressions were motion tracked during videoconference conversations, an avatar face was reconstructed in real time, and naive participants spoke with the avatar face. No naive participant guessed that the computer generated face was not video. Confederates' facial expressions, vocal inflections and head movements were attenuated at 1 min intervals in a fully crossed experimental design. Attenuated head movements led to increased head nods and lateral head turns, and attenuated facial expressions led to increased head nodding in both naive participants and confederates. Together, these results are consistent with a hypothesis that the dynamics of head movements in dyadicconversation include a shared equilibrium. Although both conversational partners were blind to the manipulation, when apparent head movement of one conversant was attenuated, both partners responded by increasing the velocity of their head movements.

Modulation of facial reactions to avatar emotional faces by nonconscious competition priming

To investigate whether subliminally priming for competition influences facial reactions to facial emotional displays, 49 participants were either subliminally competition primed or neutrally primed. Thereafter, they viewed computer generated avatar faces with happy, neutral, and sad expressions while Corrugator supercilii and Zygomaticus major reactions were recorded. Results revealed facial mimicry to happy and sad faces in the neutrally primed group but not the competition primed group. Furthermore, subliminal competition priming enhanced Corrugator supercilii activity after an initial relaxation while viewing happy faces. An impression formation task revealed counter empathic effects confirming successful competition priming. Overall, results indicate that nonconscious processes influence a presumably nonconscious behavior.

Humanoid Audio–Visual Avatar With Emotive Text-to-Speech Synthesis

Emotive audio-visual avatars are virtual computer agents which have the potential of improving the quality of human-machine interaction and human-human communication significantly. However, the understanding of human communication has not yet advanced to the point where it is possible to make realistic avatars that demonstrate interactions with natural-sounding emotive speech and realistic-looking emotional facial expressions. In this paper, We propose the various technical approaches of a novel multimodal framework leading to a text-driven emotive audio-visual avatar. Our primary work is focused on emotive speech synthesis, realistic emotional facial expression animation, and the co-articulation between speech gestures (i.e., lip movements) and facial expressions. A general framework of emotive text-to-speech (TTS) synthesis using a diphone synthesizer is designed and integrated into a generic 3-D avatar face model. Under the guidance of this framework, we therefore developed a realistic 3-D avatar prototype. A rule-based emotive TTS synthesis system module based on the Festival-MBROLA architecture has been designed to demonstrate the effectiveness of the framework design. Subjective listening experiments were carried out to evaluate the expressiveness of the synthetic talking avatar.

Amygdala activation at 3T in response to human and avatar facial expressions of emotions

Facial expressions of emotions are important in nonverbal communication. Although numerous neural structures have been identified to be involved in emotional face processing, the amygdala is thought to be a core moderator. While previous studies have relied on facial images of humans, the present study is concerned with the effect of computer-generated (avatar) emotional faces on amygdala activation. Moreover, elicited activation patterns in response to viewing avatar faces are compared with the neuronal responses to human facial expressions of emotions. Twelve healthy subjects (five females) performed facial emotion recognition tasks with optimized 3T event-related fMRI. Robust amygdala activation was apparent in response to both human and avatar emotional faces, but the response was significantly stronger to human faces in face-sensitive structures, i.e. fusiform gyri. We suggest that avatars could be a useful tool in neuroimaging studies of facial expression processing because they elicit amygdala activation similarly to human faces, yet have the advantage of being highly manipulable and fully controllable. However, the finding of differences between human and avatar faces in face-sensitive regions indicates the presence of mechanisms by which human brains can differentiate between them. This mechanism merits further investigation.

Experimental Results of Affective Valence and Arousal to Avatar's Facial Expressions

The objectives of this study were to propose a method of presenting dynamic facial expressions to experimental subjects, in order to investigate human perception of avatar's facial expressions of different levels of emotional intensity. The investigation concerned how perception varies according to the strength of facial expression, as well as according to an avatar's gender. To accomplish these goals, we generated a male and a female virtual avatar with five levels of intensity of happiness and anger using a morphing technique. We then recruited 16 normal healthy subjects and measured each subject's emotional reaction by scoring affective arousal and valence after showing them the avatar's face. Through this study, we were able to investigate human perceptual characteristics evoked by male and female avatars' graduated facial expressions of happiness and anger. In addition, we were able to identify that a virtual avatar's facial expression could affect human emotion in different ways according to the avatar's gender and the intensity of its facial expressions. However, we could also see that virtual faces have some limitations because they are not real, so subjects recognized the expressions well, but were not influenced to the same extent. Although a virtual avatar has some limitations in conveying its emotion using facial expressions, this study is significant in that it shows that a new potential exists to use or manipulate emotional intensity by controlling a virtual avatar's facial expression linearly using a morphing technique. Therefore, it is predicted that this technique may be used for assessing emotional characteristics of humans, and may be of particular benefit for work with people with emotional disorders through a presentation of dynamic expression of various emotional intensities.

Converting Faces by Preserving the Expression from the Actor's Viewpoint Based on Examples Obtained from User Interactions

This article discusses how to realize the process of converting the user's face into an avatar's face by preserving the expression. The objective of this work is to make this process reflect the user's subjective view about correspondence between the facial images of the user and the avatar. In this article, it is proposed to adapt the process to the user's view through interactions with the user. This is realized by an example-based approach, in which the correspondence between the facial images of the user and the avatar is estimated from its examples obtained from the interactions with the user. The process increasingly reflects the user's view with the number of examples.