Abstract

Transformer-based and interaction point-based methods have demonstrated promising performance and potential in human-object interaction detection. However, due to differences in structure and properties, direct integration of these two types of models is not feasible. Recent Transformer-based methods divide the decoder into two branches: an instance decoder for human-object pair detection and a classification decoder for interaction recognition. While the attention mechanism within the Transformer enhances the connection between localization and classification, this paper focuses on further improving HOI detection performance by increasing the intrinsic correlation between instance and action features. To address these challenges, this paper proposes a novel Transformer-based HOI Detection framework. In the proposed method, the decoder contains three parts: learnable query generator, instance decoder, and interaction classifier. The learnable query generator aims to build an effective query to guide the instance decoder and interaction classifier to learn more accurate instance and interaction features. These features are then applied to update the query generator for the next layer. Especially, inspired by the interaction point-based HOI and object detection methods, this paper introduces the prior bounding boxes, keypoints detection and spatial relation feature to build the novel learnable query generator. Finally, the proposed method is verified on HICO-DET and V-COCO datasets. The experimental results show that the proposed method has the better performance compared with the state-of-the-art methods.

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.