Low complexity depth map extraction and all-in-focus rendering for close-to-the-pixel embedded platforms

Abstract

Providing depth information along with images is a recurrent topic and many methods have been proposed in the literature, but most of them at the expense of heavy processing. With the development of new autonomous systems, smart sensors should be developed and algorithms simplified in order to embed processing closer to the signal acquisition. Among the reported depth extraction methods, depth from focus methods do not require heavy processing, they represent the most promising depth extraction process for close-to-the-pixel implementations. We developed low-processing-requirements depth-extraction methods based on bloc contrast analysis and compared them to a manually annotated reference ground truth and present obtained results along with results from similar works. Proposed methods are built on a sharpness criterion computed from horizontal and vertical gradients selected for their low complexity. Depth and confidence maps are then processed along horizontal and vertical directions and combined into unified depth and confidence maps. Two flavors of this process are presented: a high-resolution one for pixel-boundary precision and a low-resolution one that provides low-noise output and a drastic 4 to 1 memory requirement reduction. An error-removal method and smoothing method is also proposed to improve the final depth map. Finally, all-in-focus images are computed based on the previously processed depth maps. The algorithm output is a fully-focused RGB-D-C image.