Interactive Video Editing for Occluded Object Using Synthetic Aperture Imaging

Abstract

This paper proposes a new method to enable a user to interactively edit an existing object displayed in a virtual scene on a free-viewpoint video in real time simultaneously while capturing its original real-world scene by a camera array. The proposed method consists of the pure extraction method and the editing method. The pure extraction method is based on synthetic aperture imaging and purely extracts the whole shape of a target object that a user wants to edit on a free-viewpoint video even if the object is partially occluded by other objects. The editing method enables the user to interactively edit the extracted target object by 2D geometric transformations directly on the video in which the object is displayed in front of its background. The background is always displayed as it is in the real-world scene with no influence by the editing in the virtual scene; for example, any empty regions are not caused in the background in the virtual scene by the editing even if the background contains moving objects. Some experimental results show a good ability of the proposed method.