An interactive approach for functional prototype recovery from a single RGBD image

Yuliang Rong,Youyi Zheng,Tianjia Shao,Kun Zhou,Yin Yang

doi:10.1007/s41095-016-0032-x

Yuliang Rong, Youyi Zheng + Show 3 more

Open Access

https://doi.org/10.1007/s41095-016-0032-x

Copy DOI

Abstract

Inferring the functionality of an object from a single RGBD image is difficult for two reasons: lack of semantic information about the object, and missing data due to occlusion. In this paper, we present an interactive framework to recover a 3D functional prototype from a single RGBD image. Instead of precisely reconstructing the object geometry for the prototype, we mainly focus on recovering the object’s functionality along with its geometry. Our system allows users to scribble on the image to create initial rough proxies for the parts. After user annotation of high-level relations between parts, our system automatically jointly optimizes detailed joint parameters (axis and position) and part geometry parameters (size, orientation, and position). Such prototype recovery enables a better understanding of the underlying image geometry and allows for further physically plausible manipulation. We demonstrate our framework on various indoor objects with simple or hybrid functions.

Highlights

We develop an interactive system to recover functional prototypes from a single RGBD image
We focus on prototypes with simple proxies representing their parts as a means to alleviate the difficulties of precise 3D reconstruction which is a harder problem
We have presented a novel approach to recovering a functional prototype from a user’s highlevel annotations of relationships

Summary

Introduction

Much research effort has been devoted to recovering high-quality 3D information from RGBD images [1, 2] Most of these approaches, starting either from a single image or multiple images, are dedicated to faithfully recovering the 3D geometry of image objects, while their semantic relations, underlying physical settings, or even functionality are overlooked. Shao et al [3] leverage physical stability to suggest possible interactions between image objects and obtain a physically plausible reconstruction of objects in RGBD images. Such high-level semantic information plays an important role in constraining the underlying geometric structure

Objectives

Results

Conclusion