An Image Degradation Model for Depth-augmented Image Editing

Abstract

AbstractImages remain the most popular medium to capture our surroundings. Although significant advances have been made in developing image editing tools, the key challenge is to intelligently account for missing depth information. The growing popularity of depth images offers a new avenue to revisit image editing tasks. In this work, we investigate how even coarse depth information can be exploited to address some of the fundamental challenges in image editing namely producing correct perspective, handling occlusion, and obtaining segmentation. To this end, we propose a novel image degradation model that predicts how well an image edit can be performed in presence of coarse depth information. Technically, we create proxy geometry to summarize available depth information, and use it to predict occlusions and ordering between image patches, complete occluded regions, and anticipate image‐level changes under camera movement. We evaluate the proposed image degradation model in the context of parallax photography from single depth images.