Public Life in Public Space (PLPS): A multi-task, multi-group video dataset for public life research

Abstract

Human-computer interaction (HCI) is a multidisciplinary field of study focusing on the design of computer technology and, in particular, the interactions between humans and computers. Public space, between the urban buildings, is an open and accessible area to people. Public life, happening in public spaces, is about human activity, human interaction, expression of human feeling in the wild. Affective behavior analysis in the public space is the basic topic of the public life research, which is the key to achieve HCI applications through comprehensively understanding people’s feelings, emotions, social behaviors and their correlations in a ‘human-centered’ and engaging manner. However, it is a challenging task to design a robust HCI system due to the lack of multi-task datasets (including emotion, behavior, social relations, etc), collected under the uncontrolled conditions in real public spaces. In spite that existing separate datasets in computer vision can somehow meet the requirement of public life research, they are neither captured from real public spaces nor for multiple tasks, which cannot comprehensively support the joint research of public life. To tackle this issue, this paper presents a multi-task, multi-group human-oriented video dataset, namely public life in public space (PLPS). Specifically, multi-tasks in terms of activity recognition, emotion recognition and social relation recognition are integrated for each video data. Multi-group and multi-level labels in terms of individuals, groups, video clips are included in the dataset. With PLPS, more sophisticated computer vision model for comprehensive public life research can be facilitated.