Paired relation feature network for spatial relation recognition

Abstract

Recognizing relations between objects in an image is challenging for neural networks because some relations may not have obvious dedicated visual features. This paper proposes a Paired Relation Feature Network (PRFN), where all spatial and semantic features are extracted from the subject–object pair jointly, without using any hand-crafted features. PRFN includes a paired 2D spatial feature module that can learn the representative features from a pair of bounding boxes. By focusing on the paired depth feature between the subject and object, the problem of depth feature extraction is simplified to the recognition of a ternary relation {−1, 0, 1}, which is much easier to learn from training data. Experimental results demonstrate the effectiveness of PRFN for both the cases of RGB-D images and RGB images with estimated depth.