Learning Visual Clue for UWB-based multi-person pose estimation

Abstract

Compared to camera image-based methods, radio frequency (RF) based pose estimation has great potential for use in situations where the field of view is obstructed. In this paper, we present a novel RF-based Pose Estimation framework with Transformer (RPET) that operates in a fully end-to-end fashion and uses an easy-to-install portable radar. RPET eliminates the need for complex preprocessing and hand-crafted post-processing modules, such as region-of-interest (RoI) cropping, non-maximum suppression (NMS), and keypoint grouping. We also introduce a novel concept called Visual Clue (VC), which mimics a pose feature represented in image-based methods and improves the learning performance of multi-person pose estimation from RF signals. Our experimental results demonstrate the effectiveness of VC and the generalizability of our model to different environmental conditions, including changes in location and obstructed views.