Abstract
As humans, we regularly interpret scenes based on how objects are related , rather than based on the objects themselves. For example, we see a person riding an object X or a plank bridging two objects. Current methods provide limited support to search for content based on such relations. We present raid , a relation-augmented image descriptor that supports queries based on inter-region relations. The key idea of our descriptor is to encode region-to-region relations as the spatial distribution of point-to-region relationships between two image regions. raid allows sketch-based retrieval and requires minimal training data, thus making it suited even for querying uncommon relations. We evaluate the proposed descriptor by querying into large image databases and successfully extract non-trivial images demonstrating complex inter-region relations, which are easily missed or erroneously classified by existing methods. We assess the robustness of raid on multiple datasets even when the region segmentation is computed automatically or very noisy.
Highlights
Detecting, encoding, and synthesizing relationships between objects is critical for many shape analysis and scene synthesis tasks
We describe a novel descriptor for complex relationship between two image regions
While most images that we consider are two-dimensional projections of three-dimensional scenes, our goal is to describe the twodimensional composition of image regions rather than inferring a three-dimensional layout of the scene and analyzing relationships in three dimensions
Summary
Detecting, encoding, and synthesizing relationships between objects is critical for many shape analysis and scene synthesis tasks. Handling even simple relations like ‘on top of,’ ‘is next to,’ or ‘is touching’ has been shown to be very useful for scene understanding [Liu et al 2014], structuring raw RGBD images [Shao et al 2014], realistic scene synthesis [Fisher et al 2012; Chen et al 2014], object retrieval [Fisher et al 2011], etc. More advanced relationship descriptors like IBS [Zhao et al 2014] and ICON [Hu et al 2015] have demonstrated the value of capturing. Copyrights for thirdparty components of this work must be honored. C 2016 Copyright held by the owner/author(s).
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
