Avatar Animation Research Articles

A Dynamic Platform for Developing 3D Facial Avatars in a Networked Virtual Environment

Avatar facial expression and animation in 3D collaborative virtual environment (CVE) systems are reconstructed through a complex manipulation of muscles, bones, and wrinkles in 3D space. The need for a fast and easy reconstruction approach has emerged in the recent years due to its application in various domains: 3D disaster management, virtual shopping, and military training. In this work we proposed a new script language based on atomic parametric action to easily produce real-time facial animation. To minimize use of the game engine, we introduced script-based component where the user introduces simple short script fragments to feed the engine with a new animation on the fly. During runtime, when an embedded animation is required, an xml file is created and injected into the game engine without stopping or restarting the engine. The resulting animation method preserves the real-time performance because the modification occurs not through the modification of the 3D code that describes the CVE and its objects but rather through modification of the action scenario that rules when an animation happens or might happen in that specific situation.

Brain activity correlates with emotional perception induced by dynamic avatars

An accurate judgment of the emotional state of others is a prerequisite for successful social interaction and hence survival. Thus, it is not surprising that we are highly skilled at recognizing the emotions of others. Here we aimed to examine the neuronal correlates of emotion recognition from gait. To this end we created highly controlled dynamic body-movement stimuli based on real human motion-capture data (Roether et al., 2009). These animated avatars displayed gait in four emotional (happy, angry, fearful, and sad) and speed-matched neutral styles. For each emotional gait and its equivalent neutral gait, avatars were displayed at five morphing levels between the two. Subjects underwent fMRI scanning while classifying the emotions and the emotional intensity levels expressed by the avatars.Our results revealed robust brain selectivity to emotional compared to neutral gait stimuli in brain regions which are involved in emotion and biological motion processing, such as the extrastriate body area (EBA), fusiform body area (FBA), superior temporal sulcus (STS), and the amygdala (AMG). Brain activity in the amygdala reflected emotional awareness: for visually identical stimuli it showed amplified stronger response when the stimulus was perceived as emotional. Notably, in avatars gradually morphed along an emotional expression axis there was a parametric correlation between amygdala activity and emotional intensity.This study extends the mapping of emotional decoding in the human brain to the domain of highly controlled dynamic biological motion. Our results highlight an extensive level of brain processing of emotional information related to body language, which relies mostly on body kinematics.

Engaging adolescents in a computer-based weight management program: avatars and virtual coaches could help.

This research focuses on the potential ability of animated avatars (a digital representation of the user) and virtual agents (a digital representation of a coach, buddy, or teacher) to deliver computer-based interventions for adolescents' chronic weight management. An exploration of the acceptance and desire of teens to interact with avatars and virtual agents for self-management and behavioral modification was undertaken. The utilized approach was inspired by community-based participatory research. Data was collected from 2 phases: Phase 1) focus groups with teens, provider interviews, parent interviews; and Phase 2) mid-range prototype assessment by teens and providers. Data from all stakeholder groups expressed great interest in avatars and virtual agents assisting self-management efforts. Adolescents felt the avatars and virtual agents could: 1) reinforce guidance and support, 2) fit within their lifestyle, and 3) help set future goals, particularly after witnessing the effect of their current behavior(s) on the projected physical appearance (external and internal organs) of avatars. Teens wanted 2 virtual characters: a virtual agent to act as a coach or teacher and an avatar (extension of themselves) to serve as a "buddy" for empathic support and guidance and as a surrogate for rewards. Preferred modalities for use include both mobile devices to accommodate access and desktop to accommodate preferences for maximum screen real estate to support virtualization of functions that are more contemplative and complex (e.g., goal setting). Adolescents expressed a desire for limited co-user access, which they could regulate. Data revealed certain barriers and facilitators that could affect adoption and use. The current study extends the support of teens, parents, and providers for adding avatars or virtual agents to traditional computer-based interactions. Data supports the desire for a personal relationship with a virtual character in support of previous studies. The study provides a foundation for further work in the area of avatar-driven motivational interviewing. This study provides evidence supporting the use of avatars and virtual agents, designed using participatory approaches, to be included in the continuum of care. Increased probability of engagement and long-term retention of overweight, obese adolescent users and suggests expanding current chronic care models toward more comprehensive, socio-technical representations.

A gesture recogintion architecture for Arabic sign language communication system

Sign language is the most natural and expressive way for the hearing impaired to communicate. With technological advances in multimedia systems and applications, technology-mediated sign language communication systems have long attracted researchers to enhance the communication capabilities for the speech and hearing impaired, promising improved social opportunities and integration. This paper introduces a framework for Arabic sign language communication using Microsoft Kinect device. The merit of the proposed framework is twofold: first, the framework supports an affordable and easily deployable real-time communication system using Arabic sign language, and secondly, it provides a real-time feedback about the signer performance via real-time avatar animation. A prototype application is developed to demonstrate the merits of the proposed framework. Experimental results show that the proposed Arabic sign language method enjoys a sign detection rate of 96 %. Furthermore, the average task completion time to complete an Arabic sign was about 2.2 s. This implies that the proposed method can be used to create a real-time Arabic sign language communication system. Finally, participants of the study highlighted that the proposed system is user-friendly and easy to use, and can be used at low cost to recognize and display Arabic signs.

Interactive editing in French Sign Language dedicated to virtual signers: requirements and challenges

Signing avatars are increasingly used as an interface for communication to the deaf community. In recent years, an emerging approach uses captured data to edit and generate sign language (SL) gestures. Thanks to motion editing operations (e.g., concatenation, mixing), this method offers the possibility to compose new utterances, thus facilitating the enrichment of the original corpus, enhancing the natural look of the animation, and promoting the avatar's acceptability. However, designing such an editing system raises many questions. In particular, manipulating existing movements does not guarantee the semantic consistency of the reconstructed actions. A solution is to insert the human operator in a loop for constructing new utterances and to incorporate within the utterance's structure constraints that are derived from linguistic patterns. This article discusses the main requirements for the whole pipeline design of interactive virtual signers, including: (1) the creation of corpora, (2) the needed resources for motion recording, (3) the annotation process as the heart of the SL editing process, (4) the building, indexing, and querying of a motion database, (5) the virtual avatar animation by editing and composing motion segments, and (6) the conception of a dedicated user interface according to user' knowledge and abilities. Each step is illustrated by the authors' recent work and results from the project Sign3D, i.e., an editing system of French Sign Language (LSF) content (http://sign3d.websourd.org/sltat).

An exploratory study of using 3D avatars as online salespeople

Purpose– The purpose of this paper is to examine consumers’ perceptions of using different types of 3D animated avatars as salespeople based on credibility, homophily, attractiveness and intention to interact.Design/methodology/approach– Using a survey method, participants (n=120) evaluated four types of 3D animated avatars (i.e. human, fantasy, animal and humanoid) based on the supposition the avatars would represent a salesperson for an online retailer.Findings– The results show that avatar type has a significant influence on the perception of credibility, homophily, and attractiveness. Furthermore, credibility, homophily, and attractiveness significantly influence the intention to interact with the avatar. Overall, the human avatar was found to be perceived as the most credible, homophilous, attractive and was regarded as the highest in relation to intention to interact.Originality/value– The practical and theoretical implications are discussed to offer guidance to online retailers, graphic designers, and researchers in the benefits and pitfalls of utilizing 3D animated avatars as salespeople.

Countering "What God Thought and the Neighbors Said": Alternative Gender Possibility and Becoming-Animal in Harriette Arnow’s Hunter’s Horn

Abstract This article reads Harriette Arnow’s 1949 novel of the Kentucky hills through a lens of twenty-first-century queer theory and post-structuralism. It briefly surveys past criticism, but instead of suggesting, as many past critics have, that the novel merely offers, and fails to resolve, the tensions between its central hunting narrative and its gender critique, this paper instead argues for another way of viewing the novel’s structure, troubling denouement (when a loving and unconventional father requires his brilliant daughter to succumb to an unwanted marriage), and unique central symbol, an almost supernaturally clever red fox who plagues and challenges the community. This reading of Arnow’s mid-twentieth-century Appalachian hunting tale offers a new look at the ways in which a third-sex possibility offered through an animal avatar can undermine patriarchal authority, cross gender boundaries, and establish a uniquely Appalachian icon of alterity.

Perception and Automatic Recognition of Laughter from Whole-Body Motion: Continuous and Categorical Perspectives

Despite its importance in social interactions, laughter remains little studied in affective computing. Intelligent virtual agents are often blind to users’ laughter and unable to produce convincing laughter themselves. Respiratory, auditory, and facial laughter signals have been investigated but laughter-related body movements have received less attention. The aim of this study is threefold. First, to probe human laughter perception by analyzing patterns of categorisations of natural laughter animated on a minimal avatar. Results reveal that a low dimensional space can describe perception of laughter “types”. Second, to investigate observers’ perception of laughter (hilarious, social, awkward, fake, and non-laughter) based on animated avatars generated from natural and acted motion-capture data. Significant differences in torso and limb movements are found between animations perceived as laughter and those perceived as non-laughter. Hilarious laughter also differs from social laughter. Different body movement features were indicative of laughter in sitting and standing avatar postures. Third, to investigate automatic recognition of laughter to the same level of certainty as observers’ perceptions. Results show recognition rates of the Random Forest model approach human rating levels. Classification comparisons and feature importance analyses indicate an improvement in recognition of social laughter when localized features and nonlinear models are used.

Neural signals evoked by stimuli of increasing social scene complexity are detectable at the single-trial level and right lateralized.

Classification of neural signals at the single-trial level and the study of their relevance in affective and cognitive neuroscience are still in their infancy. Here we investigated the neurophysiological correlates of conditions of increasing social scene complexity using 3D human models as targets of attention, which may also be important in autism research. Challenging single-trial statistical classification of EEG neural signals was attempted for detection of oddball stimuli with increasing social scene complexity. Stimuli had an oddball structure and were as follows: 1) flashed schematic eyes, 2) simple 3D faces flashed between averted and non-averted gaze (only eye position changing), 3) simple 3D faces flashed between averted and non-averted gaze (head and eye position changing), 4) animated avatar alternated its gaze direction to the left and to the right (head and eye position), 5) environment with 4 animated avatars all of which change gaze and one of which is the target of attention. We found a late (> 300 ms) neurophysiological oddball correlate for all conditions irrespective of their complexity as assessed by repeated measures ANOVA. We attempted single-trial detection of this signal with automatic classifiers and obtained a significant balanced accuracy classification of around 79%, which is noteworthy given the amount of scene complexity. Lateralization analysis showed a specific right lateralization only for more complex realistic social scenes. In sum, complex ecological animations with social content elicit neurophysiological events which can be characterized even at the single-trial level. These signals are right lateralized. These finding paves the way for neuroscientific studies in affective neuroscience based on complex social scenes, and given the detectability at the single trial level this suggests the feasibility of brain computer interfaces that can be applied to social cognition disorders such as autism.

Digital Solutions for Informed Decision Making: An Academic-Community Partnership for the Development of a Prostate Cancer Decision Aid for African American Men.

African American (AA) men are significantly more likely to die of prostate cancer (PrCA) than other racial groups, and there is a critical need to identify strategies for providing information about PrCA screening and the importance of informed decision making (IDM). To assess whether a computer-based IDM intervention for PrCA screening would be appropriate for AA men, this formative evaluation study examined their (1) PrCA risk and screening knowledge; (2) decision-making processes for PrCA screening; (3) usage of, attitudes toward, and access to interactive communication technologies (ICTs); and (4) perceptions regarding a future, novel, computer-based PrCA education intervention. A purposive convenience sample of 39 AA men aged 37 to 66 years in the Southeastern United States was recruited through faith-based organizations to participate in one of six 90-minute focus groups and complete a 45-item descriptive survey. Participants were generally knowledgeable about PrCA. However, few engaged in IDM with their doctor and few were informed about the associated risks and uncertainties of PrCA screening. Most participants used ICTs on a daily basis for various purposes including health information seeking. Most participants were open to a novel, computer-based intervention if the system was easy to use and its animated avatars were culturally appropriate. Because study participants had low exposure to IDM for PrCA, but frequently used ICTs, IDM interventions using ICTs (e.g., computers) hold promise for AA men and should be explored for feasibility and effectiveness. These interventions should aim to increase PrCA screening knowledge and stress the importance of participating in IDM with doctors.

The use of virtual reality for language investigation and learning.

The use of virtual reality for language investigation and learning.

A bio-inspired limb controller for avatar animation

"A bio-inspired limb controller for avatar animation." Computer Methods in Biomechanics and Biomedical Engineering, 17(sup1), pp. 174–175Keywords:: motor controlmusculoskeletal modellinghuman motion synthesislinearising feedback AcknowledgementsThis work was supported by the French ANR project ENTRACTE (Grant agreement: ANR 13-CORD-002-01).

Breathing life into shape

Modeling how the human body deforms during breathing is important for the realistic animation of lifelike 3D avatars. We learn a model of body shape deformations due to breathing for different breathing types and provide simple animation controls to render lifelike breathing regardless of body shape. We capture and align high-resolution 3D scans of 58 human subjects. We compute deviations from each subject's mean shape during breathing, and study the statistics of such shape changes for different genders, body shapes, and breathing types. We use the volume of the registered scans as a proxy for lung volume and learn a novel non-linear model relating volume and breathing type to 3D shape deformations and pose changes. We then augment a SCAPE body model so that body shape is determined by identity, pose, and the parameters of the breathing model. These parameters provide an intuitive interface with which animators can synthesize 3D human avatars with realistic breathing motions. We also develop a novel interface for animating breathing using a spirometer , which measures the changes in breathing volume of a "breath actor".

Head motion synthesis from speech using deep neural networks

This paper presents a deep neural network (DNN) approach for head motion synthesis, which can automatically predict head movement of a speaker from his/her speech. Specifically, we realize speech-to-head-motion mapping by learning a DNN from audio-visual broadcast news data. We first show that a generatively pre-trained neural network significantly outperforms a conventional randomly initialized network. We then demonstrate that filter bank (FBank) features outperform mel frequency cepstral coefficients (MFCC) and linear prediction coefficients (LPC) in head motion prediction. Finally, we discover that extra training data from other speakers used in the pre-training stage can improve the head motion prediction performance of a target speaker. Our promising results in speech-to-head-motion prediction can be used in talking avatar animation.

Translating bus information into sign language for deaf people

This paper describes the application of language translation technologies for generating bus information in Spanish Sign Language (LSE: Lengua de Signos Española). In this work, two main systems have been developed: the first for translating text messages from information panels and the second for translating spoken Spanish into natural conversations at the information point of the bus company. Both systems are made up of a natural language translator (for converting a word sentence into a sequence of LSE signs), and a 3D avatar animation module (for playing back the signs). For the natural language translator, two technological approaches have been analyzed and integrated: an example-based strategy and a statistical translator. When translating spoken utterances, it is also necessary to incorporate a speech recognizer for decoding the spoken utterance into a word sequence, prior to the language translation module. This paper includes a detailed description of the field evaluation carried out in this domain. This evaluation has been carried out at the customer information office in Madrid involving both real bus company employees and deaf people. The evaluation includes objective measurements from the system and information from questionnaires. In the field evaluation, the whole translation presents an SER (Sign Error Rate) of less than 10% and a BLEU greater than 90%.

Emotion Expression for Affective Social Communication

Human interaction with social networking services (SNS) is currently a very active research area. In recent years, microblogs and social networking sites have become increasingly popular among online communities. Examples of such sites are Facebook, Twitter, Google+, My Space, Hi5, and WAYN. Micro log posts, such as tweets, allow users to broadcast their ideas in short form of text, voice, or images, using mobile devices and computers. Modern mobile devices- such as feature phones, smart phones, tablets, and net books- are internet-capable and feature various convenient hardware capabilities for building and managing social relationships between people. Text and speech enriched with emotions is one of the major ways of exchanging ideas, especially via telephony and SNS. By analyzing a voice stream using a Hidden Markov Model (HMM) and Log Frequency Cepstral Coefficients (LFPC) based system, different emotions can be recognized. Using a simple Java client, recognized emotions can be delivered to a sever as an index. A mobile client can then retrieve the emotion and display it through colored icons. Each emotion is mapped to a particular color, as it is natural to use colors to represent various expressions. We believe that with the help of this application one could conceivably change one's way of talking or avoid chatting with somebody whose emotional state is negative! By analysing a text snippet using C4.5 decision tree, Support Vector Machine (SVM), different emotions can be recognized. Not only for voice, these methods can be used when a user doing text message. NTT Do Como provides a service called iappli as an integrated platform for Java-based application programs. Because of its compatibility with Java, maturity, and wide usage, we chose this platform to display emotions of human voice through colored icons. In the mobile interface there are displayed icons representing different users and different emotions are represented using colors. Each emotion is mapped to a particular color. This paper proposes an emotion expression systems. Human emotion can be expressed through many kinds of media such as speech, image, facial expression, animating and so forth. This paper focuses on emotion expression using color and as avatar animations. We discuss here emotion extraction from speech.

An Exploratory Examination of Student Usage and Perceptions of Technology-based Learning Tools in a Blended Learning Class

The purpose of this study was to examine the usage of technology-based learning tools (TLTs) in a blended learning/hybrid cost accounting class. The tools used were screen capture videos, an animated avatar video, electronic flashcards with related games, and an electronic sample quiz with detailed solutions. Data for nine TLTs and one set of PowerPoint slides was captured within a learning management system. It was found that student usage was highest for the PowerPoint slides, the vocabulary-related flash cards with related games and the electronic sample quiz. Usage was lowest for the animated avatar video. When anonymously surveyed, students felt that vocabulary-related flashcards and electronic sample quiz added more value to their learning experience as opposed to the other two technology-based learning tools.

Using postural synergies to animate a low-dimensional hand avatar in haptic simulation.

A technique to animate a realistic hand avatar with 20 DoFs based on the biomechanics of the human hand is presented. The animation does not use any sensor glove or advanced tracker with markers. The proposed approach is based on the knowledge of a set of kinematic constraints on the model of the hand, referred to as postural synergies, which allows to represent the hand posture using a number of variables lower than the number of joints of the hand model. This low-dimensional set of parameters is estimated from direct measurement of the motion of thumb and index finger tracked using two haptic devices. A kinematic inversion algorithm has been developed, which takes synergies into account and estimates the kinematic configuration of the whole hand, i.e., also of the fingers whose end tips are not directly tracked by the two haptic devices. The hand skin is deformable and its deformation is computed using a linear vertex blending technique. The proposed synergy-based animation of the hand avatar involves only algebraic computations and is suitable for real-time implementation as required in haptics.

Increasing adaptability of a speech into sign language translation system

This paper describes a new version of a speech into sign language translation system with new tools and characteristics for increasing its adaptability to a new task or a new semantic domain. This system is made up of a speech recognizer (for decoding the spoken utterance into a word sequence), a natural language translator (for converting a word sequence into a sequence of signs belonging to the sign language), and a 3D avatar animation module (for playing back the signs). In order to increase the system adaptability, this paper presents new improvements in all the three main modules for generating automatically the task dependent information from a parallel corpus: automatic generation of Spanish variants when generating the vocabulary and language model for the speech recogniser, an acoustic adaptation module for the speech recogniser, data-oriented language and translation models for the machine translator and a list of signs to design. The avatar animation module includes a new editor for rapidly design of the required signs. These developments have been necessary to reduce the effort when adapting a Spanish into Spanish sign language (LSE: Lengua de Signos Española) translation system to a new domain. The whole translation presents a SER (Sign Error Rate) lower than 10% and a BLEU higher than 90% while the effort for adapting the system to a new domain has been reduced more than 50%.

A non-expert-user interface for posing signing avatars

We describe a graphical user interface designed to allow non-expert users to pose 3D characters to create American Sign Language (ASL) computer animation. The interface is an important component of a software system that allows educators of the Deaf to add sign language translation, in the form of 3D character animations, to digital learning materials, thus making them accessible to deaf learners. A study indicates that users with no computer animation expertize can create animated ASL signs quickly and accurately.Implications for RehabilitationDeaf education, especially in science, technology, engineering and math (STEM), is a pressing national problem in the U.S. Deaf individuals are significantly underrepresented in STEM fields.An important underlying cause of the educational lag is that young deaf students have limited access to grade-level curriculum materials (because of their low English literacy level).Computer animation of American Sign Language (ASL) has the potential to improve learning outcomes by making educational content deaf accessible, thus providing deaf children with the same learning opportunities as hearing students.The work reported in the paper shows that it is possible for deaf educators, who are not animators, to annotate digital lessons with ASL translation in the form of animated avatars, thus making them accessible to deaf children.