Internet of emotional people: Towards continual affective computing cross cultures via audiovisual signals

Abstract

Despite considerable advances achieved in affective computing over the past decade, the related learning paradigms still remain in an isolated fashion, i.e., models are often designed and developed task-dependently. Nevertheless, with the inherently heterogeneous and dynamic property of multiple tasks, we inevitably face several challenges, such as the implementation feasibility, when dealing with the growing number of new tasks of interest. For this reason, in this study, we endeavour to shed some fresh light on shifting the conventional isolated affective computing into a lifelong learning paradigm, namely continual affective computing. As the first tentative work in audio and video domains, we explore the lifelong learning algorithm of elastic weight consolidation for this benchmark work, in an application of well-established audiovisual emotion recognition in a cross-culture scenario, i.e., French and German emotion recognition. To evaluate the feasibility and effectiveness of the introduced lifelong learning, we perform extensive experiments across the RECOLA and SEWA databases. The empirical results show that the implemented lifelong learning approach remarkably outperforms other baselines in most cases, and is even competitive to the joint training process in some cases, indicating its capability when handling the sequential learning process with multiple tasks.