Non-Lambertian photometric stereo vision algorithm based on a multi-scale convolution deep learning architecture

Abstract

As a form of fine 3D measurement technology, photometric stereo vision is widely used in 3D reconstruction, defect detection, biomedicine, and other fields, but the traditional reflection model has limited ability to reflect the real physical properties of materials and is not suitable for non-Lambertian surfaces with nonlinear light reflection characteristics such as bright metal, greatly limiting broader application of the technology. This paper proposes a photometric stereo vision algorithm based on a multi-scale convolution deep learning architecture that implements high precision reconstruction of a nontransparent material surface under arbitrary illumination conditions. The algorithm implements a multi-scale convolution structure in the deep network. The small-scale convolution kernel strengthens the expression of photometric physics that gives the model superior detail prediction performance. The large-scale convolution kernel encourages deep networks to take advantage of neighborhood features and enhances the ability of models to overcome shadows and distinguish multiple materials. To further address the problem of arbitrary illumination conditions, the algorithm employs a multi-resolution polar coordinate division method for the illumination vector space that integrates the input image information and fully utilizes the multi-scale convolution. It is experimentally demonstrated that the multi-scale convolution deep learning architecture effectively integrates the advantages of both photometric stereo and deep learning and greatly improves the adaptability to non-Lambertian surfaces while retaining the ability to perform fine reconstruction. It provides powerful support for a wide range of applications in photometric technology.