Abstract
Abstract Hala is a bi-lingual (Arabic and English) robot receptionist located at Carnegie Mellon in Qatar. Hala is presented to users as a 3-D animated face on a screen. Users type to her, and she replies in speech synced with realistic facial expressions. However there are two existing problems with the robot. First, Hala's animation engine does not fully adhere to existing research on face dynamics, which makes it difficult to create natural and interesting facial expressions. Natural expressions help towards an engaging user experience by articulating non-verbal aspects (ex: confusion, glee, horror, etc). Second, while speaking in Arabic lip-movements are not realistic because they were adopted from English utterances. In this work we address these two limitations. Similar to the movie and video-game industry, we leverage Paul Ekman's seminal work on Facial Action Coding System (FACS) to demarcate Hala's 3D face model into muscle-primitives. These primitives are used to compose complex, yet natural, facial expressions. We have also authored an in-house tool, which allows non-programmers (for ex: artists) to manipulate the face in real-time to create expressions. The sounds humans make while talking are symbolically captured as “phonemes”. The corresponding shapes of the lips, for these sounds (i.e. phonemes), are called “visemes”. We used existing research and observed each other (and a mirror), to develop visemes that accurately capture Arabic pronunciations. Hala can thus utilize English and Arabic visemes for accurate lip-movement and syncing. We empirically tested and evaluated our work by comparing it with previous lip movements for common Arabic utterances. Nonetheless, certain pronunciations can fire less-than-ideal visemes if they are preceded by silence. Upon identifying and addressing these limitations, Hala has 11 new facial expressions for a more natural looking and behaving robot. This work also pioneered the first implemented subset of Arabic visemes on a robot.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Qatar Foundation Annual Research Forum Proceedings
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
