Cross-cultural Training Analysis Via Social Science and Computer Vision Methods

Abstract

Computer Vision technology is an invaluable addition to cross-cultural communication training for military personnel. It allows trainers to assess trainees in real time and provide feedback grounded in social science research. The present study reports on a joint analysis of military cross-cultural training data by Computer Vision specialists from GE Global Research as well as analyses from Georgetown University's Social Interaction Research Group (SIRG). Data for this study were collected over 10 days at the Army Infantry Basic Officer Leaders Course (IBOLC). 80 lieutenants participated in classroom role-play scenarios designed to assess their ability to communicate cross-culturally. GE and SIRG researchers video-recorded interactions among the role players and Soldiers and correlations were observed between these automatic interpretations and those gleaned by the SIRG analysis team in order to augment understanding of the efficacy of the cross-cultural training. For the Computer Vision methods, Each person was represented as a stream of visual cues which include: position, articulated motion, facial expressions and gaze directions. The social science researchers conducted multimodal (including embodied elements such as eye gaze, hand gestures, and body positioning), mixed method (qualitative and quantitative) discourse analyses of the data. SIRG researchers developed a coding scheme, marking specific human behavioral features within each interaction. From such coding, SIRG identified key skills in cross-cultural interaction, including observation and adaptation to unfamiliar communicative norms, rapport building, and trouble recovery (for details see Logan-Terry & Damari, forthcoming). Various correlations between raw computer vision measurements and the social science coding scheme was observed. Such results represent a significant step towards establishing the efficacy of the joint analysis of automated Computer Vision and established social science methods with regards to complex social interaction analysis.