Video-based tracking approach for nonverbal synchrony: A comparison of Motion Energy Analysis and OpenPose.

Abstract

The present study investigated the comparability of two video-based tracking techniques, namely, Motion Energy Analysis and OpenPose, in the context of nonverbal synchrony research. Participants aged 23-69 years held a 15-min unstructured conversation with a same-gender partner of the same generation. Each participant's movements were quantified with two algorithms, which were separately submitted to a wavelet transform to assess synchrony between the partners. Wavelet coherence and information in the relative phase were used to measure the intensities and patterns of synchrony, respectively. As a benchmark of the test results, gender effects and correlation with the Big Five personality traits were examined. Irrespective of the tracking technique used, the results consistently illustrated gender effects: female dyads showed more synchrony than male dyads. Among the Big Five personality traits, only Extraversion was significantly associated with synchrony in either tracking technique. The gender effect remained significant in both the tracking techniques even when controlling for personality traits, indicating that both individual (i.e., personality) and social (i.e., gender) factors contribute to synchrony. The use of video-tracking techniques in the research on synchrony for future studies was also discussed.