Accuracy in Depth Recovery and 3D Image Synthesis From Single Image Using Multi-Color Filter Aperture and Shallow Depth of Field

Rashmi R Deshpande,Mahabaleswara Ram Bhatt,C H Renu Madhavi

doi:10.1109/access.2021.3109865

Rashmi R Deshpande, Mahabaleswara Ram Bhatt + Show 1 more

Open Access

https://doi.org/10.1109/access.2021.3109865

Copy DOI

Abstract

A computational 3D image generation using a single view with multi-color filter aperture (MCA) and multi-plane representation is a cost-effective approach and most useful when there is no option to acquire either stereo or multi-views with orientation at all. Although this approach generates 3D perception image that includes multiple objects with both similar and dissimilar colors having occluded by each other, it may be insufficient for virtual/augmented reality applications due to inaccurate depth. In this article, we obtain a more accurate geometric depth estimation by formulating a suitable relationship between inter-objects depth of the 3D scene in the depth-of-field (DoF) zone and its corresponding inter-image plane depths of a 3D perception image in depth-of-focus (DoFo) zone of a given camera under shallow DoF zone constraint. But, this shallow depth zone is configured to be dependent only on the focal distance between the lens and object while the remaining parameters such as aperture diameter, focal length, and sensor sensitivity are held at constant values. All-in-focus 3D perception image is synthesized from multi-plane images (MPIs) by utilizing the inter-image plane depths computed from the disparities caused across the boundaries and its smooth surface from image textures inside the respective boundaries of the 2D MCA image. The 2.1D sketch is used as a semantic segmentation technique to determine the number of objects in the 3D scene as one in-focus region and the rest as out-of-focus regions due to the circle of confusions (CoCs) on the fixed image sensor plane. The same enables both ordering of the image regions and identifying occlusion wherever applicable. An accurate depth 3D image is synthesized, replacing accurate inter-depths in place of inter-depth between MPIs used for 3D perception image. In the end, the paper summarizes few experimental validations for the proposed approach with some salient examples having depth gaps between 0.5cm to 10.5cm.

Highlights

A Ll in-focus 3D image generation having accurate depths is an essential need in many computer vision-based applications for getting crucial details regarding the 3D scene
We develop a quantitative model for the computational imaging system suggested in [14] that employs both depth cues from multi-color filter aperture (MCA) [8] and composition of a 3D image using multi-plane images (MPIs) by using multi-region image (MRI) decomposition of the acquired image using a 2.1D sketch [15] as semantic segmentation
ON EXPERIMENTAL EVALUATIONS FOR 3D IMAGING SYSTEM we discuss some simulated experimental results to validate the quantified 3D image generation formulated in previous sections

Summary

Introduction

A Ll in-focus 3D image generation having accurate depths is an essential need in many computer vision-based applications for getting crucial details regarding the 3D scene. Most of the approaches suggested are based on either single view, stereo-view or multi-views [1] [2] for generating 3D perception qualitatively for the real 3D world scene from the irradiant optical stimulations (mathematically it would be of one to many mapping), [3], which we term as "3D perception image". Among those approaches that produce 3D perception image, single view image-based 3D image generation is a most challenging one though it is advantageous because of two reasons, namely, (i) it involves less optics with an extensive computational complexity and (ii) it is helpful in some scenarios where stereo or multi-view imaging is not possible.

Objectives

Discussion

Conclusion