Relative multiscale deep depth from focus

Abstract

The problem of reconstructing a depth map from a sequence of differently focused images (focal stack) is called Depth from focus. The core idea of this method is to analyze the sharpness of each pixel and compare it along the z axis of the focal stack to estimate the true depth value. This approach has two main drawbacks: it depends on the optics of the camera and on the focus measure operator. Recent advances in deep learning techniques show promising results in this way, however, still have problems generalizing to different scenes, cameras, optics and focal stack focus positions. In this paper we propose a novel deep learning based method to approach this problem. Firstly, we propose to estimate distances relative to the focal stack focus position instead of estimating true or invariant depths, allowing us to generalize different scenes and optical setups without losing the possibility to extract real distances. Secondly, we present our novel architecture: a 2D siamese encoder–3D decoder with a differentiable argmax regression that is able to compute depth from stacks of variable sizes. Finally, we compare our method with 2 other depth from focus algorithms and with a monocular depth estimation method.