Solving Shape-From-Shading problem through shape and depth joint optimization

Abstract

Objective:Shape-From-Shading (SFS) is one unique and long-standing problem aiming to reconstruct shape/depth from only one image. The SFS problem can be regarded as an inverse process of image rendering, i.e., solving the Bi-directional Reflection Distribution Function (BRDF) model with the given image. Among various BRDF models, physically based perspective Cook-Torrance BRDF model is generally used because of its universality. However, the solution of Cook-Torrance BRDF model based SFS problem has not been properly addressed because the model is very complicated and non-convex. Method:In this paper, we present a shape and depth joint optimization method under the neural network framework to solve this kind of SFS problem. Our method has three aspects of contributions. Firstly, we remove the unrealistic surface smoothness constraint imposed on the solution of SFS. Secondly, the design of shape and depth joint optimization provides the method with a higher probability of bypassing local optimum and finding the global optimum. Thirdly, the neural network framework enables our method with the ability of performing point-wise optimization, that is, the optimization does not subject to the approximation on the discrete domain. Result:Through experiments, we prove that our method can solve Cook-Torrance BRDF model based SFS problem with good accuracy. The maximum Mean Absolute Error, Mean Relative Error and Root-mean-square error of depth results are 0.8 mm, 1.0% and 1.23 mm respectively, which outperforms related methods.