Human–Object Interaction Detection Based on Star Graph

Abstract

Graph attention networks (GATs)-based method performs well in Human–Object Interaction (HOI) detection due to its ability to aggregate contextual information. However, the traditional GATs-based methods are computationally intensive, and the graph’s structure cannot effectively represent the HOI in an image. Meanwhile, the graph models are unable to predict the interactions that involve less contextual information correctly. In this paper, we design a method based on graph models called V-SGATs, which stands for Visual Branch and Star Graph Attention Networks. The human-centric star graph and object-centric star graph are adopted to reduce computational complexity, and the logical structure of the star graph can represent HOI more reasonably. Meanwhile, the visual branch that recognizes the interaction without utilizing the contextual information is designed to aid the graph model in predicting the interactions that involve less contextual information. Experiments are carried out on two large-scale HOI public benchmarks V-COCO and HICO-DET, and the results show that the proposed method performs better than most of the existing methods based on GATs.