Multiple-Layer Visibility Propagation-Based Synthetic Aperture Imaging through Occlusion.

Tao Yang,Xiaomin Tong,Lingyan Ran,Wenguang Ma,Yanning Zhang,Jingyi Yu,Jing Li,Rui Yu

doi:10.3390/s150818965

Abstract

Heavy occlusions in cluttered scenes impose significant challenges to many computer vision applications. Recent light field imaging systems provide new see-through capabilities through synthetic aperture imaging (SAI) to overcome the occlusion problem. Existing synthetic aperture imaging methods, however, emulate focusing at a specific depth layer, but are incapable of producing an all-in-focus see-through image. Alternative in-painting algorithms can generate visually-plausible results, but cannot guarantee the correctness of the results. In this paper, we present a novel depth-free all-in-focus SAI technique based on light field visibility analysis. Specifically, we partition the scene into multiple visibility layers to directly deal with layer-wise occlusion and apply an optimization framework to propagate the visibility information between multiple layers. On each layer, visibility and optimal focus depth estimation is formulated as a multiple-label energy minimization problem. The layer-wise energy integrates all of the visibility masks from its previous layers, multi-view intensity consistency and depth smoothness constraint together. We compare our method with state-of-the-art solutions, and extensive experimental results demonstrate the effectiveness and superiority of our approach.

Highlights

The capability of seeing through occlusions in heavily-cluttered scenes is beneficial to many practical computer vision application fields, ranging from hidden object imaging to detection, tracking and recognition in surveillance
We have compared the performance of our method with the synthetic aperture imaging methods of Vaish et al [11] and Pei et al [6] on four datasets, including the CD case behind plants from
We have presented a novel synthetic aperture imaging approach for creating all-in-focus images through occlusion

Summary

Introduction

The capability of seeing through occlusions in heavily-cluttered scenes is beneficial to many practical computer vision application fields, ranging from hidden object imaging to detection, tracking and recognition in surveillance. Many different camera arrays have been built over the past few years (as shown in Figure 1), and the camera array synthetic aperture imaging or SAI [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17] provides a unique capability of seeing through occlusions. SAI warps and integrates the multiple view images to simulate a virtual camera with an ultra-large convex lens, and it can focus on different frontal-parallel [1] or oblique [2] planes with a narrow depth of field. Objects laying on the virtual focus plane, even if being occluded in reality, would be clearly imaged

Objectives

Results

Conclusion