Automatic temporal ranking of children’s engagement levels using multi-modal cues

Abstract

As children of ages 5–8 often play with each other in small groups, their differences in social development and personality traits usually cause various levels of engagement among others. For example, one child may just observe without engaging at all with others while another child may be interested in both the other children as well as the activity. To develop child-friendly interaction technology such as social robots that can adapt robot behaviours to the social situation of a group of children and facilitate harmonious engagement, we aim to study how we can automatically detect these children’s engagement levels. In this paper, we present a novel automatic method that ranks children in a group according to their engagement level in a temporal way based on non-verbal cues that are robust in naturalistic group settings. Our method combines the omission probability of each rank transformed from discriminative outputs from an SVM ranking method and the transition probability between ranks in time. In comparing our proposed method to other existing methods (such as rule-based ranking, basic SVM, SVM ordinal regression, SVM ranking, and SVMHMM), we found that our novel method yields promising results.