All Skeletons are Created Equal! A Domain Adaptation Transformer to Handle Multiple Topologies

Abstract

Translate Article

Designing an effective Human Pose Estimation (HPE) pipeline necessitates handling diverse datasets, a task often deemed necessary but burdensome by researchers. Existing datasets are annotated using different conventions, with the human body represented as a parametric 3D model or as a collection of 2D/3D joints and bones, known as skeleton-based annotation. Despite its widespread use in training both 2D and 3D HPE networks, the lack of standardization in the topologies of joint-based pose annotations requires considerable difficulties when evaluating the algorithms performances across different datasets. To solve this issue, we introduce a novel self-supervised human pose domain adaptation approach to map a given skeletal model into a target model of choice. We design a transformer-based architecture trained to reconstruct missing joints within a given topology, aligning them with the target model. During testing, the network seamlessly reconstructs the missing joints, treating them as if masked, based on the common joints shared by the original topology and the target one. Unlike previous approaches, our method works with arbitrary body poses, joints number, and skeleton topologies, across multiple datasets.<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>The code is available at https://github.com/mmlab-cv/DAT.git