Image Outpainting with Depth Assistance

Abstract

In some scenarios such as autonomous diriving, we can get a sparse point cloud with a large field of view, but an RGB image with a limited FoV. This paper studies the problem of image expansion using depth information converted from sparse point cloud projection. General image expansion tasks only use images as input for expansion. Such expansion is only carried out by RGB information, and the expanded content has limitations and does not conform to reality. Therefore, we propose introducing depth information into the image expansion task, and offering a reasonable image expansion model using the depth information of the sparse point cloud. Our model can generate more realistic and more reliable image expansion content than general image expansion. The results generated by our work must be authentic, which further enhances the practical significance of the image expansion problem. Furthermore, we also designed a variety of experimental research schemes on how to perform interactive matching between depth and RGB information further to enhance the help of depth for image expansion.