White light endoscopy is the clinical gold standard for detecting diseases in the gastrointestinal tract. Most applications involve identifying visual abnormalities in tissue color, texture, and shape. Unfortunately, the contrast of these features is often subtle, causing many clinically relevant cases to go undetected. To overcome this challenge, we introduce Multi-contrast Laser Endoscopy (MLE): a platform for widefield clinical imaging with rapidly tunable spectral, coherent, and directional illumination. We demonstrate three capabilities of MLE: enhancing tissue chromophore contrast with multispectral diffuse reflectance, quantifying blood flow using laser speckle contrast imaging, and characterizing mucosal topography using photometric stereo. We validate MLE with benchtop models, then demonstrate MLE in vivo during clinical colonoscopies. MLE images from 31 polyps demonstrate an approximate three-fold improvement in contrast and a five-fold improvement in color difference compared to white light and narrow band imaging. With the ability to reveal multiple complementary types of tissue contrast while seamlessly integrating into the clinical environment, MLE shows promise as an investigative tool to improve gastrointestinal imaging.

PS-GS: Gaussian splatting for multi-view photometric stereo

Surface-related critical quality attributes (CQAs) of solid oral dosage forms influence mechanical integrity, coating uniformity, imprint readability, and downstream product performance. Conventional two-dimensional visual inspection and geometry-focused three-dimensional (3D) systems primarily assess gross deviations but often lack mechanistic surface descriptors relevant to CQA interpretation. This study tested the hypothesis that integrating complementary photometric and geometric imaging modalities can enhance sensitivity to surface-related CQAs beyond single-modality inspection. A hybrid photometric-geometric reconstruction framework was developed for manufacturing-oriented surface integrity assessment. A hybrid photometric-geometric 3D reconstruction pipeline integrating photometric stereo and structured-light acquisition captured complementary local reflectance and global geometric information. Depth maps derived from both modalities were spatially aligned and fused to generate a unified surface representation. Surface descriptors, including curvature entropy and roughness metrics, were evaluated to assess surface complexity and stability. Micro-defect detectability was examined to compare reconstruction modalities under controlled experimental conditions. The hybrid reconstruction approach demonstrated improved balance between local imprint fidelity and global surface continuity compared with single-modality methods, while maintaining reproducible surface metrics across repeated sessions. Surface descriptors provided structured measures of surface variation beyond visual inspection alone. The proposed framework is positioned as a CQA-linked surface anomaly assessment tool within a Process Analytical Technology (PAT) context, rather than as a direct surrogate for traditional release testing. By enhancing mechanistic interpretation of surface variability, the approach may support risk-based PAT strategies in manufacturing. This proof-of-concept study used a small dataset (n = 4), requiring further validation with larger, diverse samples.

Abstract Traditional photometric stereo techniques are constrained by stringent hardware calibration requirements, pronounced sensitivity to specular highlights and cast shadows, and an inherent difficulty in reconciling reconstruction fidelity with practical usability. These limitations significantly impede their deployment in high-precision 3D facial reconstruction. To address these challenges, this study presents an uncalibrated photometric stereo framework specifically designed for facial scenarios, integrating two key modules within a two-stage optimization pipeline. In the first stage, illumination directions are estimated using a shared-weight convolutional architecture with global max-pooling and self-attention mechanisms to extract illumination-invariant features. In the second stage, PCA-based whitening followed by Huber robust regression is employed to recover surface normals under uncertain illumination conditions. Comprehensive experiments on synthetic and real-world datasets demonstrate the effectiveness of the proposed approach. On the test set, the illumination estimation network achieves a mean angular error of 7.29° for lighting direction prediction, while intensity estimation errors are reduced by 25%–33% compared with existing baselines. For surface normal reconstruction, the proposed method achieves mean angular errors of 6.7° under the Oracle-Dir setting (using ground-truth lighting directions) and 7.6° under the Est-Dir setting (using estimated lighting directions). By eliminating the need for precise hardware calibration, the framework reduces system complexity and operational cost while improving robustness, making it well suited for applications such as 3D facial recognition and digital human modeling. Future work will explore unified end-to-end optimization strategies to further improve reconstruction accuracy and generalization capability.

Photometric stereo (PS) aims to recover high-fidelity surface normals by observing pixel-wise radiometric variations under different light directions. However, traditional PS methods require dense sampling of the incident light to mitigate non-Lambertian effects, such as cast shadows and specular highlights, creating a significant efficiency bottleneck for practical optical metrology. To address this efficiency bottleneck, illumination planning methods seek to identify an optimal set of light directions to maximize information gain with minimal measurements. A critical limitation of existing illumination planning paradigms is their reliance on selecting from a predefined, discrete set of candidate light directions. This discretization of the light space introduces an artificial bottleneck, severely limiting precision and adaptability. In this paper, we address this limitation by introducing a Continuous and Online ILlumination Planning framework for Photometric Stereo (COIL-PS). Instead of selecting from a fixed grid, our method formulates illumination planning as a continuous regression problem, adaptively steering light positioning in the continuous hemispherical domain. By coupling online illumination planning with feedback from intermediate normal estimates, COIL-PS adaptively navigates non-Lambertian effects such as shadows and specularities, which allow for the precise angular placement of illumination required to resolve geometric ambiguities that fall between fixed grid points. Extensive experiments on a synthetic dataset, semi-real benchmarks, and our custom-built real-world robotic validation system demonstrate that COIL-PS achieves superior normal reconstruction accuracy compared to state-of-the-art discrete planning methods, even with a budget of only ten lights, significantly outperforming discrete planning paradigms.

Accurate photometric modeling of road pavements requires knowledge of the normal distribution function (NDF), which characterizes the statistical orientation of surface microfacets. Direct measurement of the NDF on real pavements is rarely feasible; instead, we propose estimating it from topographic data acquired via laser profilometry. This paper develops the theoretical framework to compute the NDF and assess measurement quality through an inter-line correlation indicator. Experimental measurements on various road samples are performed using both laser profilometry and photometric stereo imaging for validation. Results show that anisotropies and acquisition deviations in profilometric data can strongly affect NDF estimation, while the proposed correlation analysis efficiently reveals such artifacts. This study establishes a practical methodology for evaluating surface normal distributions over extended road areas, paving the way for realistic large-scale photometric simulations of urban environments.

Capturing accurate texture maps from physical materials remains a challenge in digital prototyping and projection-based spatial augmented reality (P-SAR). This paper presents an open-source material scanning system based on photometric stereo, designed for affordability, simplicity, and efficient operation. The system combines a consumer-grade digital camera, multifaceted reflector (MR16) LED lighting, and Arduino-controlled automation to acquire material data up to A4 size within 15s. Accurate colour reproduction is achieved through a hybrid calibration workflow that integrates camera profiling with a 3D lookup table. The resulting images are processed in a streamlined Substance 3D Designer pipeline to generate albedo and normal maps compatible with physically based rendering (PBR). To evaluate performance under realistic conditions, two fabric samples were scanned and qualitatively compared with professionally digitised references. Albedo maps were assessed based on dominant colour accuracy using CIEDE2000 (ΔE00), while normal maps were evaluated through visual rendering comparisons and directional distribution analysis. Scanning and processing times were also measured to verify workflow efficiency. Results demonstrate that the proposed system produces perceptually consistent textures suitable for real-time rendering applications while offering a low-cost and customisable solution for material digitisation.

Surface roughness measurement method based on symmetric dual-light-source photometric stereo and structure tensor analysis

To enable timely, effective, and high-accuracy detection of scour around offshore wind turbine pile foundations, this study proposes a three-dimensional reconstruction and scour volume detection method based on side-scan sonar imagery. First, the sonar images of pile foundations are preprocessed through grayscale conversion, binarization, and region expansion and merging to obtain an effective grayscale representation of scour pits. An optimized Shape-from-Shading (SFS) method is then applied to reconstruct the three-dimensional geometry from the effective grayscale map, generating point cloud data of the scour pits. Subsequently, the point cloud data are filtered using curvature and normal vector constraints, followed by depth-based z-axis descent detection, clustering, and morphological restoration to extract individual scour pit point clouds. Finally, a weight-corrected AlphaShape algorithm is employed to accurately calculate the volume of each scour pit. Numerical experiments involving five simulated scour scenarios across three types demonstrate that the proposed method achieves accurate identification and extraction of scour pit point clouds, with an average volume measurement accuracy of 97.495% compared with theoretical values. Field measurements in real-world environments further validate the effectiveness of the proposed method for practical scour volume detection around offshore wind turbine foundations.

Photometric stereo is widely used to recover detailed surface normals. However, previous methods fail to balance the accuracy and efficiency. Conventional photometric stereo achieves high accuracy but suffers from low efficiency due to spectral-multiplexing and inefficient algorithms. In contrast, multispectral photometric stereo captures images efficiently with spectral-multiplexing, but its accuracy is harmed by crosstalk. In this paper, we aim to resolve the crosstalk issue to achieve fast photometric stereo (FPS) at low cost. First, we analyze the formulation and impact of crosstalk, showing that it significantly affects normal estimation, with external factors being primary contributors to crosstalk and internal factors being the secondary. Subsequently, we propose the FPS framework with a fast data capture scheme that combines time- and spectral-multiplexing to introduce constraints on crosstalk regarding both internal and external factors, along with a lightweight network, FPS-Net, to remove crosstalk caused by those factors based on constraints under such scheme. Finally, we build a real-world crosstalk-affected FPS dataset to evaluate the performance in handling crosstalk for normal estimation. Experimental results show the superior accuracy and efficiency of our method. The code and dataset are available at https://github.com/wxy-zju/FPS-Net.

A photometric stereo and Vision Transformer-based framework for automated air void analysis in hardened concrete

High Dynamic Range Phase Inpainting Method Integrating Photometric Stereo

Dynamic single-pixel 3D imaging is challenging due to the requirement of complex calibration and the inherent tradeoff between resolution and the number of measurements. In this work, we propose a calibration-free framework that integrates binocular single-pixel imaging (SPI) with a super-resolution photometric stereo network (SRPS-Net) to achieve dynamic 3D SPI video. Photometric images reconstructed from arbitrary left and right viewpoints are processed by SRPS-Net to recover accurate surface normals without calibration. Experimental results show that our system achieves dynamic 3D reconstruction at a resolution of 128×128 with a frame rate of 6.5 fps, reaching pixel-level accuracy. The proposed method demonstrates robust generalization to complex objects and gestures, providing a compact, cost-effective, and calibration-free solution for dynamic single-pixel 3D imaging.

Enhancing photometric stereo via multi-channel information processing

UPS-HSG: Uncalibrated photometric stereo based on adaptive hybrid Spherical Gaussian

Abstract The growth of information technology and the Internet has increased the demand for online art exhibitions. As the digitisation of artworks often requires highly customised equipment and techniques, this study proposes a practical method for obtaining spatially varying bidirectional reflectance distribution function parameters for oil paintings with rich impasto and varying gloss. We combined the photometric stereo algorithm with a deep learning model, which was trained based on real oil painting samples. The proposed method surpasses current inverse rendering and pure deep learning methods that are limited to specific materials or synthetic data. Our system effectively reproduced the nonhomogeneous nature of oil paintings by capturing normal vectors, albedo, roughness, and specular intensity for each pixel. This approach provides a practical solution for digitising oil paintings, enabling the reproduction of impastos and glossy appearances in virtual environments.

Real-time rendering is increasingly used in augmented and virtual reality (AR/VR), interactive design, and product visualisation, where materials must prioritise efficiency and consistency rather than the extreme accuracy required in offline rendering. In parallel, the growing demand for personalised and customised products has created a need for digital materials that can be generated in-house without relying on expensive commercial systems. To address these requirements, this paper presents a low-cost digitisation workflow based on photometric stereo. The system integrates a custom-built scanner with cross-polarised illumination, automated multi-light image acquisition, a dual-stage colour calibration process, and a node-based reconstruction pipeline that produces albedo and normal maps. A reproducible evaluation methodology is also introduced, combining perceptual colour-difference analysis using the CIEDE2000 (ΔE00) metric with angular-error assessment of normal maps on known-geometry samples. By openly providing the workflow, bill of materials, and implementation details, this work delivers a practical and replicable solution for reliable material capture in real-time rendering and product customisation scenarios.

In this paper, we introduce a novel method of Filtering and Serializing Spatial Information to tackle uncalibrated photometric stereo tasks, termed FSSI-PS. Photometric stereo aims to recover surface normals from images with varying lighting and is crucial for tasks like 3D reconstruction and defect detection. Current methods in complex surface reconstruction are costly and inaccurate due to redundant feature representations from GCN or Transformer modules, caused by the weak global information extraction capability of GCNs or the large computational cost of Transformers. Furthermore, the trainset's lack of richness in texture complexity makes reconstruction more difficult. We address these issues by optimizing feature maps and dataset richness through serializing and filtering. First, we use Mamba-RNN to optimize feature representation by directly fusing feature maps, which reduces redundancy and uses minimal computational resources. Specifically, we treat input spatial information as a sequence and serialize it by sorting. Furthermore, we introduce the Mean Angular Variation metric to assess reconstruction difficulty by measuring texture complexity. It classifies PS-Sculpture and PS-Blobby into three categories: Difficult, Normal, and Simple. We use this to construct DNS-S+B, a photometric stereo training set with rich complexity levels. Our method is compared with state-of-the-art methods on the DiLiGenT and LUCES benchmarks to highlight effectiveness.

Neural measurement method based on near-field photometric stereo with tangent estimation for machined metal surfaces

We present a reproducible, low-cost stereo perception stack for the ARMOS TurtleBot that couples two OV7670 cameras to two ESP-WROOM-32 microcontrollers and a ROS pipeline on a Raspberry-Pi. A rigid T-mount fixes a 75 mm baseline; each sensor streams QVGA grayscale via an 8-bit DVP bus to its dedicated ESP32 using I2S-parallel with DMA. A shared pixel clock and microsecond VSYNC time-stamps enforce pairing with a 1 ms threshold, and frames are transported to the host using UDP (nominal) or UART (fallback). The method comprises photometric standardization, stereo calibration and rectification, SGBM disparity, depth recovery, and publication of costmaps for navigation; we report throughput, latency, synchronization, and depth accuracy across indoor scenarios. With QVGA over UDP, the pipeline sustains 12 fps at disparity, attains a median end-to-end latency near 94 ms (95th percentile under 120 ms), and yields costmap hit rates above 95% within 1.5 m. Mean absolute depth error grows with range (1.7 cm at 0.5 m to 12.8 cm at 3.0 m), consistent with the pinhole model. Ablations show that QQVGA over UART lowers median latency but reduces frequency and accuracy, while light JPEG reduces bandwidth variance at a modest long-range penalty. Mechanical drawings, bill of materials, firmware, and ROS configurations are provided to support replication.