Photometric Stereo Approach Research Articles

In-situ 3D reconstruction of worn surface topography via optimized photometric stereo

Since worn surfaces contain rich information of the wear mechanisms, in-situ measurements of surface topography can characterize ongoing wear degradation in machines. With the help of photometric stereo vision, three-dimensional (3D) topography of worn surfaces is obtained with a monocular microscope. However, the accuracy of the reconstructed surfaces remains low due to the non-Lambertian reflections of worn surfaces and noise in the image acquisition equipment. To address this issue, an optimized photometric stereo approach is proposed for the improvement of worn surface reconstruction. To accommodate the non-Lambertian reflections, a multi-branch network is constructed to estimate normal vectors from both the photometric images and the incident illumination directions. The estimated normal vectors are adopted to reconstruct worn surface topography by embedding prior knowledge. With this design, the overall distortion caused by image noise is effectively suppressed. The proposed method is verified by comparing with the Laser Scanning Confocal Microscopy (LSCM). As the main result, over 88% similarity on the worn surface roughness can be obtained.

Deep Photometric Stereo Network with Multi-Scale Feature Aggregation.

We present photometric stereo algorithms robust to non-Lambertian reflection, which are based on a convolutional neural network in which surface normals of objects with complex geometry and surface reflectance are estimated from a given set of an arbitrary number of images. These images are taken from the same viewpoint under different directional illumination conditions. The proposed method focuses on surface normal estimation, where multi-scale feature aggregation is proposed to obtain a more accurate surface normal, and max pooling is adopted to obtain an intermediate order-agnostic representation in the photometric stereo scenario. The proposed multi-scale feature aggregation scheme using feature concatenation is easily incorporated into existing photometric stereo network architectures. Our experiments were performed with a DiLiGent photometric stereo benchmark dataset consisting of ten real objects, and they demonstrated that the accuracies of our calibrated and uncalibrated photometric stereo approaches were improved over those of baseline methods. In particular, our experiments also demonstrated that our uncalibrated photometric stereo outperformed the state-of-the-art method. Our work is the first to consider the multi-scale feature aggregation in photometric stereo, and we showed that our proposed multi-scale fusion scheme estimated the surface normal accurately and was beneficial to improving performance.

Three-dimensional imaging through scattering media using a single pixel detector.

Imaging objects hidden by turbid media, such as smoke, fog, or biological tissues, is a challenge for scientists. Compressive ghost imaging and a photometric stereo approach are employed to estimate the 3D shape of an object behind a scattering medium. A sequence of speckle patterns is projected onto the object. Four images with different shadings are accurately reconstructed from the object behind a diffuser. The 3D shape is obtained by applying the photometric stereo to the reconstructed 2D images. This technique is robust against scattering, even for 3D imaging in opaque media with multiple scattering, and it is not sensitive to changes in the scattering media or displacement of the medium. Also, it has the benefits of compressive ghost imaging strategy, such as hyperspectral or polarimetric imaging with sub-Nyquist sampling. As a proof for this concept, the 3D shape of a target behind a diffuser was experimentally retrieved, and the results were compared with the original 3D shape and the 3D shape reconstructed in the absence of the diffuser. The accuracy of the reconstructed 3D shapes was maintained in the presence of the diffuser plate in various thicknesses and orientations.

Contactless Biometric Identification Using 3D Finger Knuckle Patterns.

Study on finger knuckle patterns has attracted increasing attention for the automated biometric identification. However, finger knuckle pattern is essentially a 3D biometric identifier and the usage or availability of only 2D finger knuckle databases in the literature is the key limitation to avail full potential from this biometric identifier. This paper therefore introduces (first) contactless 3D finger knuckle database in public domain, which is acquired from 130 different subjects in two-session imaging using photometric stereo approach. This paper investigates on the 3D information from the finger knuckle patterns and introduces a new feature descriptor to extract discriminative 3D features for more accurate 3D finger knuckle matching. An individuality model for the proposed feature descriptor is also presented. Comparative experimental results using the state-of-the-art feature extraction methods on this challenging 3D finger knuckle database validate the effectiveness of our approach. Although our feature descriptor is designed for 3D finger knuckle patterns, it is also attractive for other hand-based biometric identifiers with similar patterns such as the palmprint and fingerprint. This observation is validated from the outperforming results, using the state-of-the-art pixel-wise 3D palmprint and 3D fingerprint feature descriptors, on other publicly available datasets.

A Differential Approach to Shape from Polarisation: A Level-Set Characterisation

Despite the longtime research aimed at retrieving geometrical information of an object from polarimetric imaging, physical limitations in the polarisation phenomena constrain current approaches to provide ambiguous depth estimation. As an additional constraint, polarimetric imaging formulation differs when light is reflected off the object specularly or diffusively. This introduces another source of ambiguity that current formulations cannot overcome. With the aim of deriving a formulation capable of dealing with as many heterogeneous effects as possible, we propose a differential formulation of the Shape from Polarisation problem that depends only on polarimetric images. This allows the direct geometrical characterisation of the level-set of the object keeping consistent mathematical formulation for diffuse and specular reflection. We show via synthetic and real-world experiments that diffuse and specular reflection can be easily distinguished in order to extract meaningful geometrical features from just polarimetric imaging. The inherent ambiguity of the Shape from Polarization problem becomes evident through the impossibility of reconstructing the whole surface with this differential approach. To overcome this limitation, we consider shading information elegantly embedding this new formulation into a two-light calibrated photometric stereo approach..

High-Precision 3D Sensing with Hybrid Light Field & Photometric Stereo Approach in Multi-Line Scan Framework

High-Precision 3D Sensing with Hybrid Light Field & Photometric Stereo Approach in Multi-Line Scan Framework

A photometric stereo approach for chronic wound measurement

Purpose – This paper aims to review state of the art of techniques for dimensioning chronic wounds, and validate the possibilities of employing a new proposed optical imaging approach for general task of wound assessment. Design/methodology/approach – Current techniques used for quantifying wound surface are reviewed and evaluated from various perspectives to exam their usability in wound care clinical settings. A photometric stereo (PS) approach will be identified and verified to work as an alternative solution to better satisfy practical requirements on quantifying the dimension of real and mocked wounds. Findings – Both contact and contactless approaches provide some useful functions for wound management; however, new imaging modalities are still required for achieving good portability, affordability and applicability in assisting decision-making in clinical settings. The PS approach can work as a potential solution to provide these functionalities as well as dense geometrical and color texture information of measured areas. The experiments demonstrate that the new approach is able to conveniently produce comparable results to those from latest stereo vision-based techniques. Research limitations/implications – This work proposed and initially verified the potential of PS technique for the task of wound measurement. Substantial improvements on the prototype and more clinical trials are still required to validate the new technique before it is accepted as a tool for practical wound measurement. Originality/value – This new PS approach has good potential to reliably measure the dimension of wounds as well as recover their color texture which could contain additional valuable information for predicting a healing procedure for those wound occurring deeper underneath the skin surface.

Near Field Photometric Stereo with Point Light Sources

Shape recovery of an object based on shading variations resulting from different light sources has recently been reconsidered. Improvements have been made that allow for the photometric stereo approach to serve as a competitive alternative to other shape reconstruction methods. However, most photometric stereo methods tend to ignore factors that are relevant in practical applications. The setup considered in this paper tackles photometric stereo reconstruction in the case of a specific near-field imaging. This means that both the camera and the light sources are close to the imaged object, where close can be loosely considered as a setup having similar distances between lights, camera, and object. The known challenges that characterize the problem involve perspective viewing geometry, point light sources, and images that may include shadowed regions. Here, we pay special attention to the question of how to faithfully model these aspects and at the same time design an efficient and robust numerical solver. We present a mathematical formulation that integrates the above assumptions into a single coherent model based on quasi-linear PDEs. The well-posedness is proved showing uniqueness of a weak (i.e., Lipschitz continuous) solution. The surface reconstruction in our near-field scenario can then be executed efficiently in linear time. The merging strategy of the irradiance equations provided for each light source allows us to consider a characteristic expansion model which enables the direct computation of the surface. We evaluate several types of light attenuation models with a nonuniform albedo and noise on synthetic data. We also demonstrate the proposed method on surface reconstruction of real data using three images, each one taken with a different light source by a working prototype. We demonstrate the accuracy of the proposed method compared to other methods that ignore the near-field setup and assume distant, parallel beam light sources.

A 3-D Surface Reconstruction Approach Based on Postnonlinear ICA Model

Photometric stereo technique deals with the reconstruction of three-dimensional (3-D) shape of an object by using several images of the same surface taken from the same viewpoint but under illuminations from different directions. In this paper, we propose a new photometric stereo scheme based on a new reflectance model and the postnonlinear (PNL) independent components analysis (ICA) method. The proposed nonlinear reflectance model consists of diffuse components and specular components for modeling the surface reflectance of a stereo object in an image. Unlike the previous approaches, these two components are not separated and processed individually in the proposed model. An unsupervised learning adaptation algorithm is developed to estimate the reflectance model based on image intensities. In this algorithm, the PNL ICA method is used to obtain the surface normal on each point of an image. Then, the 3-D surface model is reconstructed based on the estimated surface normal on each point of image by using the enforcing integrability method. Two experiments are performed to assess the performance of the proposed approach. We test our algorithm on synthetically generated images for the reconstruction of surface of objects and on a number of real images captured from the Yale Face Database B. These testing images contain variability due to illumination and varying albedo in each point of surface of human faces. All the experimental results are compared to those of the existing photometric stereo approaches tested on the same images. The results clearly indicate the superiority of the proposed nonlinear reflectance model over the conventional Lambertian model and the other linear hybrid reflectance model.

Integration of vision and laser displacement sensor for efficient and precise digitizing

This paper presents a new digitizing technique which integrates computer vision methods, such as the photometric stereo approach, with the laser displacement sensors (LDS). The curvature at each point of the surface can be measured cost-effectively by the photometric stereo approach. Once the information of curvature of the surface is provided, the measuring speed of LDS can be improved by digitizing at the selected points of the surface and meanwhile the accuracy of digitization is still maintained. By means of the integration, the merits of both approaches can be achieved simultaneously and the efficiency of the digitization can be improved. Experimental results are provided to verify the proposed method.

A neural network approach to photometric stereo inversion of real-world reflectance maps for extracting 3-D shapes of objects

Presents a neural network approach to the problem of photometric stereo inversion of the reflectance maps of real-world objects for the purpose of estimating the 3-D attitudes of the surface patches of objects. As in the photometric stereo approach, here also the observation that there is a one-to-one mapping between the n-tuples of the photometric stereo image intensities and the orientations of the surface normals is valid. A multilayered feedforward neural network with backpropagation training algorithm is used as dimensionality reducer to effectively encode this mapping by associating the two components of surface normals to the observed intensities from three photometric stereo images of the underlying surface patches. The training patterns are sampled from the images of a Gaussian sphere of average reflectance containing both diffuse and specular components. The neural network thus trained has been tested on images of real-world objects with different shapes and reflectance properties. Using the surface normals estimated by the neural network, 3-D shapes of the objects have been reconstructed to a good approximation.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>