A two-branch deep learning with spatial and pose constraints for social group detection

Abstract

Group detection is a crucial yet challenging task in various application domains, including video surveillance analysis and service robot interactions. Previous studies have struggled with complicated group structures and diverse human behaviors, resulting in low performance in multiple activity group detections. In this article, we propose a two-branch deep learning framework with spatial and pose constraints that combines physical proximity and social behavior to improve group detection performance. Specifically, we design an original Variable Group Detection Transform Network (VGDTN) that integrates dyad representations and contextual information to recognize complex group arrangements. Additionally, we present a novel pose-based refined clustering module that efficiently explores human behavior within multiple activity group detections by utilizing ternary postures classification. To address the computational intensity of DL-based methods in extensive surveillance video analysis, a designed lightweight VGDTN which consists of NIN blocks maintains good performance while reducing the computational burden. The experimental results show that our method outperforms the state-of-the-art methods by 7%, 5%, 2%, and 4% in F1_score (T =1) on four public datasets.