Combining trunk movement and facial expression enhances the perceived intensity and believability of an avatar's pain expression

Abstract

The use of computer-generated characters (avatars) is increasingly being used to study emotion and social cognition in humans, as it offers a highly controllable, yet potentially interactive experimental set-up. However, avatars often fall short at conveying credible emotions. This study explored the interaction between body motion and facial expression on the perceived intensity and believability of an avatar's pain expression. Adults were shown videos of an agematched avatar displaying facial expressions of pain while the body was static or while the trunk was oscillating at varied amplitudes, which represented human dynamic equilibrium (idle motion) or trunk rocking expressing a sustained pain (pain behavior). Pupil size was recorded during the task as an objective marker of emotional reaction. Results showed that the avatar's pain was perceived to be more intense and more believable in the presence of both idle motion and trunk rocking than in the static condition. Pupil dilated more when facial pain expression was combined with trunk rocking than in the static and idle conditions. This work demonstrated the critical role of idle motion when creating dynamical pain-expressing avatars, as well as the potentiating effect of body motion when combined with facial expression on the perception of avatar's pain.