Recurrent bidirectional visual human pose retrieval

Abstract

Content‐based image retrieval technique is an essential component under various application scenarios. In this paper, instead of visual similarity defined by colors, shapes, or textures, we aim to retrieve images with respect to the visual similarity defined by the human pose. In our framework, all the poses are derived from images, inspired by the recent development of three‐dimension (3D) human pose reconstruction. Furthermore, to make the retrieval more robust against reconstruction error, we propose a recurrent bidirectional similarity measure called recurrent best‐buddies similarity (RBBS). Specifically, we treat the similarity measure between two visual poses as a distance measure between two point vectors, with each point representing one of the reconstructed 3D human pose candidates. We then recur the similarity measure by the displacement of query. As a justification, we verify the validity of RBBS in a one dimension (1D) Gaussian situation. In experiments, we build an original dataset for the retrieval task. Both the qualitative and quantitative results show the usefulness of our framework; the quantitative results evaluated by mean average precision (MAP or mAP) especially demonstrate that RBBS is improved by 14.13% compared to the most competitive alternative methods. © 2019 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.