Joint Pedestrian and Body Part Detection via Semantic Relationship Learning

Junhua Gu,Wenbai Chen,Chuanxin Lan,Hu Han

doi:10.3390/app9040752

Abstract

While remarkable progress has been made to pedestrian detection in recent years, robust pedestrian detection in the wild e.g., under surveillance scenarios with occlusions, remains a challenging problem. In this paper, we present a novel approach for joint pedestrian and body part detection via semantic relationship learning under unconstrained scenarios. Specifically, we propose a Body Part Indexed Feature (BPIF) representation to encode the semantic relationship between individual body parts (i.e., head, head-shoulder, upper body, and whole body) and highlight per body part features, providing robustness against partial occlusions to the whole body. We also propose an Adaptive Joint Non-Maximum Suppression (AJ-NMS) to replace the original NMS algorithm widely used in object detection, leading to higher precision and recall for detecting overlapped pedestrians. Experimental results on the public-domain CUHK-SYSU Person Search Dataset show that the proposed approach outperforms the state-of-the-art methods for joint pedestrian and body part detection in the wild.

Highlights

Pedestrian and body part detection has wide potential applications, in person re-identification [1], intelligent surveillance, action recognition from body parts [2], etc
We propose a Body Part Indexed Feature (BPIF) representation to encode the semantic relationship between individual body parts, providing robustness against partial even full occlusions of the body part
A brief analysis why the proposed Adaptive Joint Non-Maximum Suppression (AJ-non-maximum suppression (NMS)) should work for the joint pedestrian and body part detection task is that as shown in Figure 1, the two pedestrians’ bodies are seriously occluded, their joint IOU considering both the pedestrian and the body parts according to our AJ-NMS remains lower than the threshold to be suppressed because the overlap between the two pedestrians’ heads can be very small

Summary

Introduction

Pedestrian and body part (e.g., upper body, head-shoulder and head) detection has wide potential applications, in person re-identification [1], intelligent surveillance, action recognition from body parts [2], etc. The performance of joint pedestrian and body part detection in the case of severe overlap/occlusion, remains unsatisfactory To this end, we propose a Body Part. Indexed Feature (BPIF) to encode the semantic relationship between individual body parts (i.e., head, head-shoulder, upper body, and whole body), leading to improved robustness against partial even full occlusions of the body part. We propose a BPIF representation to encode the semantic relationship between individual body parts (i.e., head, head-shoulder, upper body, and whole body), providing robustness against partial even full occlusions of the body part. The proposed AJ-NMS treat one person’s head, head-shoulder, upper-body, and body as a whole unit, leading to higher recall for detecting overlapped pedestrians, and small part such as pedestrian head. The proposed approach advances the state-of-the-art in joint pedestrian and body part detection on the widely used CUHK-SYSU Person Search Dataset [25]

Pedestrian Detection

Non-Maximum Suppression

Object Relation Learning

Overview

Adaptive Joint Non-Maximum Suppression

Network Training

Datasets and Settings

Comparisons with the State-of-Art

Ablation Study

Findings

Conclusions