Photometric Stereo Research Articles

Photometric stereo light calibration optimization and high-reflective distance fitness compensation method to improve reconstruction

This paper proposes an interpolated light calibration optimization and high-reflective area compensation method to solve the accuracy loss caused by idealization of the photometric stereo (PS) model and high-reflective area. The spatial distribution model of light intensity is defined as a cubic interpolation function, which is used to obtain an intensity coefficient matrix to optimize the PS model. A light source adaptation model is adopted to select the appropriate position of the light source based on the characteristics of the object. Finally, a distance fitness compensation model is established to repair the high-reflective area, which selects the optimal pixel values of the non-high-reflective area. The experiment result shows that the proposed method can obtain the light source spatial characteristics, and high-reflective areas could be repaired.

Shape-from-Shading Derived DEMs from ShadowCam Images in Permanently Shadowed Regions at the Lunar South Pole: A First Look

Abstract. The lunar south pole, particularly its permanently shadowed regions (PSRs), has been the focus of future lunar exploration. The solar altitude angle at the lunar south pole is less than ∼ 2°, resulting in the PSRs not being directly illuminated and the possible existence of water ice in the PSRs due to the low temperatures. High-resolution digital elevation models (DEMs) in PSRs are crucial for future exploration missions and scientific research. However, existing image datasets (e.g., LROC NAC images) cannot penetrate the PSRs. Laser altimetry (e.g., LOLA) can measure the PSRs, but the DEMs derived from LOLA suffer from noises and interpolation defects. ShadowCam is a newly deployed camera onboard the Korea Pathfinder Lunar Orbiter, with a high imaging sensitivity. ShadowCam can capture the faint secondary illumination within PSRs, revealing previously indistinguishable topographic features in persistent darkness. Based on our existing Shape-from-Shading (SfS) pipeline for generating high-resolution DEMs from monocular images, we introduce a novel SfS approach that utilizes secondary illumination and reflectance to reconstruct pixel-wise high-resolution DEMs from ShadowCam images in PSRs. The approach consists of three steps: (1) modeling the secondary illumination on the target surfaces in PSRs based on the view factor; (2) optimizing a low-resolution LOLA DEM using the secondary illumination model; (3) refining the DEM to pixel-wise resolution based on SfS using the ShadowCam images. A test area (approximately 1 km × 1 km) within the PSR in the Shackleton crater near the lunar south pole is used for experimental evaluation. The results show that the DEM generated by the proposed approach is highly consistent with the direct LOLA measurements, particularly in small-scale topographic features such as crater floors and rims that are absent from the LOLA DEM. A first look at the results indicates that the proposed approach has great potential for generating high-resolution DEMs from ShadowCam images in PSRs, which can aid future exploration missions targeting the PSRs at the lunar south pole and support related scientific research on the water ice in the PSRs.

Artificial Skin Based on Visuo‐Tactile Sensing for 3D Shape Reconstruction: Material, Method, and Evaluation

AbstractArtificial skin has shown great potential in robot perception and human healthcare. It provides multifunctional tactile sensing, including 3D shape reconstruction, contact feedback, and temperature perception, where the 3D reconstruction function is indispensable for dexterous hands in tactile cognition and interaction. Vision‐based tactile sensor (VTS) is an innovative bionic tactile sensor and supports high‐resolution, high‐precision, and high‐density tactile reconstruction compared with electronic tactile sensors. Considering the unique contribution of visuo‐tactile sensing to artificial skin, this review focuses on the 3D reconstruction techniques of the VTS. 3D reconstruction methods are classified into five categories based on sensing modalities, hardware categories, and modeling approaches: 1) photometric stereo, 2) binocular depth calibration, 3) optical flow, 4) deep learning, and 5) ToF (time of flight). In addition, the association and difference of reconstruction methods are analyzed from the hardware perspective, and the development and technological details of 3D reconstruction are summarized. On this basis, the challenges and development direction are discussed. This review can be viewed as a technology guide to provide references for interested researchers. Furthermore, it is expected to promote the extensive application of the VTS in artificial skins.

Fringe photometric stereo

In fringe projection profilometry (FPP), a high spatial frequency of fringe typically indicates powerful error suppression capability; however, it complicates phase unwrapping, necessitating a greater number of patterns and thus compromising the real-time performance of scanning. In this letter, we report a fringe photometric stereo (FPS) to completely bypass phase unwrapping for recovering the continuous 3D surface. We reveal the linear mapping relationship between the phase gradient and depth gradient, thereby facilitating the computation of depth gradients from phase gradients. Thus, we can employ depth gradients to reconstruct high-quality 3D profiles. Experimental results demonstrate that our FPS surpasses traditional phase-shifting profilometry in mitigating Gaussian noise. Our approach enables high-quality and unambiguous 3D reconstruction using single-frequency fringe patterns, relying solely on a camera and a projector without the need for dual, multiple, or light field cameras, thereby paving a path to high-speed and precision 3D imaging.

Pixel-level Terrain Processing of the Martian Surface from Moderate Resolution Images of Tianwen-1

The moderate-resolution camera (MoRIC) of the Tianwen-1 orbiter, China’s first Mars exploration mission, is a frame-imaging scheme color camera. However, generating a precise digital elevation model (DEM) from these images faces challenges due to the presence of featureless and repetitive regions on the Martian surface. Traditional photogrammetric methods often struggle to achieve pixel-level resolution in these areas. To address this, we employ a shape from shading (SFS) algorithm that leverages shading patterns to reconstruct fine-scale terrain details, optimizing photogrammetric initial DEM. This paper presents a novel global bisection search algorithm, based on the Pearson correlation coefficient to determine the smoothing parameter, a crucial element in the SFS process. Comparisons with higher-resolution data from the Mars Express high-resolution stereo camera and Mars Reconnaissance Orbiter CTX images validate our approach, highlighting the potential of SFS to significantly enhance the scientific value of Tianwen-1 MoRIC data by generating high-resolution, three-dimensional maps of Mars.

Research on Defect Detection Method of Fusion Reactor Vacuum Chamber Based on Photometric Stereo Vision.

This paper addresses image enhancement and 3D reconstruction techniques for dim scenes inside the vacuum chamber of a nuclear fusion reactor. First, an improved multi-scale Retinex low-light image enhancement algorithm with adaptive weights is designed. It can recover image detail information that is not visible in low-light environments, maintaining image clarity and contrast for easy observation. Second, according to the actual needs of target plate defect detection and 3D reconstruction inside the vacuum chamber, a defect reconstruction algorithm based on photometric stereo vision is proposed. To optimize the position of the light source, a light source illumination profile simulation system is designed in this paper to provide an optimized light array for crack detection inside vacuum chambers without the need for extensive experimental testing. Finally, a robotic platform mounted with a binocular stereo-vision camera is constructed and image enhancement and defect reconstruction experiments are performed separately. The results show that the above method can broaden the gray level of low-illumination images and improve the brightness value and contrast. The maximum depth error is less than 24.0% and the maximum width error is less than 15.3%, which achieves the goal of detecting and reconstructing the defects inside the vacuum chamber.

Prior-shape-guided photometric stereo model for 3D damage measurement of worn surfaces

Three-dimensional (3D) damage measurement often encounters limited accuracy due to the non-Lambertian effects on worn surfaces. To address this issue, a prior-shape-guided surface reconstruction model is introduced with photometric stereo vision. Initially, an illumination calibration method is constructed to mitigate the uneven brightness on worn surface images. Subsequently, a prior-shape-guided network is developed that employs the feature aggregation strategy and prior shape uncertainty to guide worn surface reconstruction. Experimental results reveal that the proposed model can effectively recover the typical morphologies of worn surfaces, and the root mean square error is reduced from 7.39 to 1.59 when compared to existing methods. Most importantly, this model has been applied to detect aero-engine bearings and achieve 3D quantitative characterization for worn damages.

Enhanced Scratch Detection for Textured Materials Based on Optimized Photometric Stereo Vision and Fast Fourier Transform–Gabor Filtering

In the process of scratch defect detection in textured materials, there are often problems of low efficiency in traditional manual detection, large errors in machine vision, and difficulty in distinguishing defective scratches from the background texture. In order to solve these problems, we developed an enhanced scratch defect detection system for textured materials based on optimized photometric stereo vision and FFT-Gabor filtering. We designed and optimized a novel hemispherical image acquisition device that allows for selective lighting angles. This device integrates images captured under multiple light sources to obtain richer surface gradient information for textured materials, overcoming issues caused by high reflections or dark shadows under a single light source angle. At the same time, for the textured material, scratches and a textured background are difficult to distinguish; therefore, we introduced a Gabor filter-based convolution kernel, leveraging the fast Fourier transform (FFT), to perform convolution operations and spatial domain phase subtraction. This process effectively enhances the defect information while suppressing the textured background. The effectiveness and superiority of the proposed method were validated through material applicability experiments and comparative method evaluations using a variety of textured material samples. The results demonstrated a stable scratch capture success rate of 100% and a recognition detection success rate of 98.43% ± 1.0%.

GR-PSN: Learning to Estimate Surface Normal and Reconstruct Photometric Stereo Images.

In this paper, we propose a novel method, namely GR-PSN, which learns surface normals from photometric stereo images and generates the photometric images under distant illumination from different lighting directions and surface materials. The framework is composed of two subnetworks, named GeometryNet and ReconstructNet, which are cascaded to perform shape reconstruction and image rendering in an end-to-end manner. ReconstructNet introduces additional supervision for surface-normal recovery, forming a closed-loop structure with GeometryNet. We also encode lighting and surface reflectance in ReconstructNet, to achieve arbitrary rendering. In training, we set up a parallel framework to simultaneously learn two arbitrary materials for an object, providing an additional transform loss. Therefore, our method is trained based on the supervision by three different loss functions, namely the surface-normal loss, reconstruction loss, and transform loss. We alternately input the predicted surface-normal map and the ground-truth into ReconstructNet, to achieve stable training for ReconstructNet. Experiments show that our method can accurately recover the surface normals of an object with an arbitrary number of inputs, and can re-render images of the object with arbitrary surface materials. Extensive experimental results show that our proposed method outperforms those methods based on a single surface recovery network and shows realistic rendering results on 100 different materials.

The shading isophotes: Model and methods for Lambertian planes and a point light

Structure-from-Motion (SfM) and Shape-from-Shading (SfS) are complementary classical approaches to 3D vision. Broadly speaking, SfM exploits geometric primitives from textured surfaces and SfS exploits pixel intensity from the shading image. We propose an approach that exploits virtual geometric primitives extracted from the shading image, namely the level-sets, which we name shading isophotes. Our approach thus combines the strength of geometric reasoning with the rich shading information. We focus on the case of untextured Lambertian planes of unknown albedo lit by an unknown Point Light Source (PLS) of unknown intensity. We derive a comprehensive geometric model showing that the unknown scene parameters are in general all recoverable from a single image of at least two planes. We propose computational methods to detect the isophotes, to reconstruct the scene parameters in closed-form and to refine the results densely using pixel intensity. Our methods thus estimate light source, plane pose and camera pose parameters for untextured planes, which cannot be achieved by the existing approaches. We evaluate our model and methods on synthetic and real images.

Photometric stereo multi-information fusion unsupervised anomaly detection algorithm

Due to different materials, product surfaces are susceptible to light, shadow, reflection, and other factors. Coupled with the appearance of defects of various shapes and types, as well as dust, impurities, and other interfering influences, normal and abnormal samples are difficult to distinguish and a common problem in the field of defect detection. Given this, this paper proposes an end-to-end photometric stereo multi-information fusion unsupervised anomaly detection model. First, the photometric stereo feature generator is used to obtain normal, reflectance, depth, and other information to reconstruct the 3D topographic details of the object’s surface. Second, a multi-scale channel attention mechanism is constructed to fully use the feature associations of different layers of the backbone network, and the limited feature information is used to enhance the defect characterization ability. Finally, the original image is fused with normal and depth features to find the feature variability between defects and defects, as well as between defects and background. The feature differences between the source and clone networks are utilized to achieve multi-scale detection and improve detection accuracy. In this paper, the model performance is verified on the PSAD dataset. The experimental results show that the algorithm in this paper has higher detection accuracy compared with other algorithms. Among them, the multi-scale attention mechanism and multi-information fusion input improve the detection accuracy by 2.56% and 1.57%, respectively. In addition, the ablation experiments further validate the effectiveness of the detection algorithm in this paper.

Shadow-constrained shape-from-shading for pixel-wise 3D surface reconstruction at the lunar south pole

ABSTRACT High-resolution digital elevation models (DEMs) of the lunar surface are crucial for lunar exploration missions and scientific research. The emergence of advanced technologies, such as Shape-from-Shading (SfS), has enabled the generation of pixel-wise high-resolution DEMs from monocular images of the lunar surface. However, SfS encounters significant challenges in locations with limited illumination, such as the lunar south pole, where the surface is largely covered by shadows. Therefore, this paper presents a novel shadow-constrained SfS approach for pixel-wise 3D surface reconstruction at the lunar south pole. The proposed approach uses multiple high-resolution images captured under different illumination conditions and an existing low-resolution DEM as the inputs and generates a high-resolution DEM with the same resolution as that of the input image through hierarchical SfS processing incorporating shadow constraints. Experiments were conducted using actual images collected by the Lunar Reconnaissance Orbiter (LRO) Narrow Angle Camera (NAC) at the lunar south pole. Comparisons with respect to photogrammetric DEMs generated from stereo NAC images show that the DEMs generated using the proposed approach exhibit the smallest root mean square error (RMSE). Moreover, shaded relief images rendered from the DEMs generated using the proposed approach demonstrate the highest similarity to the actual NAC images. Detailed profile comparisons further validate the effectiveness of the shadow constraint in optimizing 3D reconstruction in regions in proximity to shadows and within shadowed regions. The proposed shadow-constrained SfS approach can be used to generate high-resolution DEMs to support future missions to explore the lunar south pole, with applications including landing site evaluation and route planning for lunar probes or astronauts.

Evaluation of emergency skin decontamination protocols in response to an acid attack (vitreolage)

The incidence of “acid attacks” (vitreolage) is a global concern, with those affected often receiving lifelong medical care due to physical and psychological damage. The purpose of this study was to evaluate the effectiveness of several emergency skin decontamination approaches against concentrated (>99 %) sulphuric acid and to identify the effective window of opportunity for decontamination. The effects of four decontamination methods (dry, wet, combined dry & wet and cotton cloth) were assessed using an in vitro diffusion cell system containing dermatomed porcine skin. Sulphuric acid (H2SO4) was applied to the skin with decontamination protocols performed at 10 s, 30 s, 8 min, and 30 min post exposure. Skin damage was quantified by tritiated water (3H2O) penetration, receptor fluid pH and photometric stereo imaging (PSI), with quantification of residual sulphur (by SEM-EDS) to determine overall decontamination efficiency. Skin translucency (quantified by PSI) demonstrated a time-dependent loss of dermal tissue integrity from 10 s. Quantification of dermal sulphur content confirmed the rapid (exponential) decrease in decontamination efficiency with time. The pH of the water effluent indicated complete neutralisation of acid from the skin surface after 90 s of irrigation. Wet decontamination (either alone or immediately following dry decontamination) was the most effective intervention evaluated, although no decontamination technique was statistically effective after 30 s exposure to the acid. These data demonstrate the time-critical consequences of dermal exposure to concentrated sulphuric acid: we find no practical window of opportunity for acid decontamination, as physical damage is virtually instantaneous.

Uni MS-PS: A multi-scale encoder-decoder transformer for universal photometric stereo

Photometric Stereo (PS) addresses the challenge of reconstructing a three-dimensional (3D) representation of an object by estimating the 3D normals at all points on the object’s surface. This is achieved through the analysis of at least three photographs, all taken from the same viewpoint but with distinct lighting conditions. This paper introduces a novel approach for Universal PS, i.e., when both the active lighting conditions and the ambient illumination are unknown. Our method employs a multi-scale encoder–decoder architecture based on Transformers that allows to accommodates images of any resolutions as well as varying number of input images. We are able to scale up to very high resolution images like 6000 pixels by 8000 pixels without losing performance and maintaining a decent memory footprint. Moreover, experiments on publicly available datasets establish that our proposed architecture improves the accuracy of the estimated normal field by a significant factor compared to state-of-the-art methods. Code and dataset available at: https://clement-hardy.github.io/Uni-MS-PS/index.html.

Adaptive Weighted Data Fusion for Line Structured Light and Photometric Stereo Measurement System.

Line structured light (LSL) measurement systems can obtain high accuracy profiles, but the overall clarity relies greatly on the sampling interval of the scanning process. Photometric stereo (PS), on the other hand, is sensitive to tiny features but has poor geometrical accuracy. Cooperative measurement with these two methods is an effective way to ensure precision and clarity results. In this paper, an LSL-PS cooperative measurement system is brought out. The calibration methods used in the LSL and PS measurement system are given. Then, a data fusion algorithm with adaptive weights is proposed, where an error function that contains the 3D point cloud matching error and normal vector error is established. The weights, which are based on the angles of adjacent normal vectors, are also added to the error function. Afterward, the fusion results can be obtained by solving linear equations. From the experimental results, it can be seen that the proposed method has the advantages of both the LSL and PS methods. The 3D reconstruction results have the merits of high accuracy and high clarity.

Artificial Intelligence Technology for Interactive Mobile Devices and Its Application in 3D Visual Design

The use of visual rationality in planning and designing indoor 3D spaces can effectively enhance the quality of environmental art design. For the rational planning and design of 3D spaces, it is essential to assign attribute values to the rational evaluation index of 3D space planning, determine the weight vector of rational evaluation factors for indoor 3D space planning, and accomplish the visual rationality planning and design of indoor 3D spaces. 3D reconstruction is one of the primary objectives and focal points of machine vision research. The principle is to extract 3D information from the 2D image captured by the machine’s sensors in order to reconstruct a 3D scene model. The traditional method constructs the evaluation index system for indoor 3D space planning to determine the weight of the evaluation index for alternative points. However, it overlooks the weight vector of the evaluation factors, leading to low accuracy in rational planning. This paper optimizes the effect of structured light reconstruction using photometric stereo vision. Firstly, the text delves into the impact of light sources on the 3D reconstruction effect. Subsequently, a fusion algorithm combining 3D and structured light reconstruction methods is introduced, leveraging artificial intelligence (AI) technology. Experimental results show that this algorithm has a superior 3D reconstruction effect.

Design and development of an occlusion-free surface profile measurement device

The prevalent surface profile measurement techniques in the sub-millimetre measurement scale, based on active triangulation (such as laser scanners), stereo vision and photometric stereo, may miss certain parts of the surface of the object due to occlusion. Occlusion occurs in these techniques as they rely on a top-view configuration, where the illumination rays and the surface profile lie on the same plane. In this study, a surface profile measurement device is designed and developed, which can generate occlusion-free measurements by adopting a side-view configuration. This proposed measurement technique is tested on a few surfaces, and the results are presented.

Retraction Note: Exploring a narrative-based framework for historical exhibits combining JanusVR with photometric stereo

Retraction Note: Exploring a narrative-based framework for historical exhibits combining JanusVR with photometric stereo

Morphology reconstruction and defect detection of holes inner surface based on reflection characteristics

A reconstruction method is proposed based on the reflection characteristics to measure the defect profile and depth of the inner surface of the hole. In addition to capturing geometric information of the scene, this method decomposes the distortion in the endoscope image into circumferential and axial distortions, thereby improving correction accuracy and enabling accurate measurement of the defect contour. The bidirectional reflection distribution function (BRDF) is established based on reflection characteristic theory, allowing for mapping between image brightness and normal declination. By accumulating tangent values of normal deflection angles, the height of axial contour lines inside the hole can be reconstructed to achieve defect shape reconstruction and depth measurement. This approach overcomes convex-concave ambiguity in shape from shading (SFS), as demonstrated by experimental results. The proposed detection method offers equipment versatility and high measurement efficiency, making it suitable for machine vision-based detection of internal surface defects in holes.

Shape-from-shading Refinement of LOLA and LROC NAC Digital Elevation Models: Applications to Upcoming Human and Robotic Exploration of the Moon

The Lunar Reconnaissance Orbiter (LRO) has returned a wealth of remotely sensed data of the Moon over the past 15 years. As preparations are under way to return humans to the lunar surface with the Artemis campaign, LRO data have become a cornerstone for the characterization of potential sites of scientific and exploration interest on the Moon's surface. One critical aspect of landing site selection is knowledge of topography, slope, and surface hazards. Digital elevation models derived from the Lunar Orbiter Laser Altimeter (LOLA) and Lunar Reconnaissance Orbiter Camera (LROC) instruments can provide this information at scales of meters to decameters. Shape-from-shading (SfS), or photoclinometry, is a technique for independently deriving surface height information by correlating surface reflectance with incidence angle and can theoretically approach an effective resolution equivalent to the input images themselves, typically better than 1 m per pixel with the LROC Narrow Angle Camera (NAC). We present a high-level, semiautomated pipeline that utilizes preexisting Ames Stereo Pipeline tools along with image alignment and parallel processing routines to generate SfS-refined digital elevation models using LRO data. In addition to the present focus on the lunar south pole with Artemis, we also demonstrate the usefulness of SfS for characterizing meter-scale lunar topography at lower equatorial latitudes.