Introduction to the special issue on multimodal corpora for modeling human multimodal behavior

Abstract

There is an increasing interest in multimodal communication as suggested by several national and international projects (ISLE, HUMAINE, SIMILAR, CHIL, AMI, CALO, VACE, CALLAS), the attention devoted to the topic by well-known institutions and organizations (the National Institute of Standards and Technology, the Linguistic Data Consortium), and the success of conferences related to multimodal communication (ICMI, IVA, Gesture, Measuring Behavior, Nordic Symposium on Multimodal Communication, LREC Workshops on Multimodal Corpora). As Dutoit et al. (2006) lament, however, « there is a lack of multimodal corpora suitable for the evaluation of recognition/synthesis approaches and interaction strategies . . . one must admit that most corpora available today target the study of a limited number of modalities, if not one ». Corpora are not only relevant to evaluation purposes, their importance extending to all the stages of design and development of multimodal systems. Moreover, established practices and guidelines