Abstract
We propose a novel problem revolving around two tasks: (i) given a scene, recommend objects to insert, and (ii) given an object category, retrieve suitable background scenes. A bounding box for the inserted object is predicted in both tasks, which helps downstream applications such as semiautomated advertising and video composition. The major challenge lies in the fact that the target object is neither present nor localized in the input, and furthermore, available datasets only provide scenes with existing objects. To tackle this problem, we build an unsupervised algorithm based on object-level contexts, which explicitly models the joint probability distribution of object categories and bounding boxes using a Gaussian mixture model. Experiments on our own annotated test set demonstrate that our system outperforms existing baselines on all sub-tasks, and does so using a unified framework. Future extensions and applications are suggested.
Highlights
The problem of context-based recommendation for object insertion in visual scenes involves decidingManuscript received: 2019-12-24; accepted: 2020-01-29 whether an object category is compatible with the content of a given image, and where such an object might be feasibly placed
The scene retrieval task provides a specialized search engine that is capable of retrieving images given an object, that goes beyond previous content-based image retrieval systems [3,4,5]
Our work aims to provide inspiration for object insertion, which is closely related to the topic of image composition
Summary
Manuscript received: 2019-12-24; accepted: 2020-01-29 whether an object category is compatible with the content of a given image, and where such an object might be feasibly placed. This new problem has many applications, such as advertising, augmented reality, and interior design. Our goal is to build a bidirectional recommendation system [1, 2] that performs two tasks using a unified framework: Object Recommendation: Given a scene, recommend a sorted list of categories of objects suitable for insertion, with associated bounding boxes. Scene Retrieval: Given a category of objects to be inserted, retrieve a sorted list of suitable background scenes and corresponding bounding boxes for object placement. As we show in our experiments, this enables applications such as automatic previewing for a gallery of target
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
