Lambertian Objects Research Articles

Relationship between turbulent image variance and average image gradient

Optical turbulence can cause substantial distortions in imaging over long horizontal paths. For Lambertian objects, these distortions are only seen where there is a gradient in the object’s radiance. It is possible to establish a relationship between the intensity variance of a turbulent image and the average image’s gradient squared. We test the validity of a linear relationship between these quantities using turbulent imaging data. We find that it performs reasonably well for weak and intermediate optical turbulence regimes, but that some discrepancies remain to be explained.

Light invariant photometric stereo.

An imaging setup that enables unsynchronized photometric stereo (PS) for Lambertian objects based on modulated light sources is presented. Knowing the specific frequency of the modulated light source allows to filter out any other light in the scene. This creates an image that depends only on the particular light source while ignoring the ambient light. Moreover, if the scene is illuminated by multiple modulated sources with different frequencies, repeating this process for every frequency will produce a sequence of images with the corresponding illumination. This sequence is then used as an input to the PS algorithm for 3D reconstruction. The proposed approach, named Light Invariant Photometric Stereo (LIPS), was verified on both synthetic and real-world data. LIPS eliminates the need for synchronization between the sources and the camera and significantly outperformed the classical PS method in an illuminated environment.

Impact of color on polarization-based 3D imaging and countermeasures.

Diffuse polarization-based 3D imaging has flourished with the ability to obtain the 3D shapes of objects without multiple detectors, active mode lighting, or complex mechanical structures, which are major drawbacks of other methods for 3D imaging in natural scenes. However, traditional polarization-based 3D imaging technology introduces color distortion when reconstructing the surface of multi-colored targets. We propose a polarization-based 3D imaging model to recover the 3D geometry of multi-colored Lambertian objects. In particular, chromaticity-based color removal theory is used to restore the intrinsic intensity, which is modulated only by the target shape, and we apply the recovered intrinsic intensity to address the orientation uncertainty of target normals due to azimuth ambiguity. Finally, we integrate the corrected normals to reconstruct high-precision 3D shapes. Experimental results demonstrate that the proposed model has the ability to reconstruct multi-colored Lambertian objects exhibiting non-uniform reflectance from single views under natural light conditions.

Towards Non-Lambertian Scenes for Tensor Displays

Tensor displays are screens able to render a light field with correct depth perception without wearing glasses. Such devices have already been shown to be able to accurately render a scene composed of Lambertian objects. This paper presents the model and prototyping of a tensor display with three layers, using repurposed computer monitors, and extends the light field factorization method to non-Lambertian objects. Furthermore, we examine the relation and limitations between the depth-of-field and the depth range with Lambertian and non-Lambertian scenes. Non-Lambertian scenes contain out-of-range disparities that can not be properly rendered with the usual optimization method. We propose to artificially compress the disparity range of the scene by using two light fields focused on different depths, effectively solving the problem and allowing to render the scene clearly on both simulated and prototyped tensor display.

A boundary migration model for imaging within volumetric scattering media

Effectively imaging within volumetric scattering media is of great importance and challenging especially in macroscopic applications. Recent works have demonstrated the ability to image through scattering media or within the weak volumetric scattering media using spatial distribution or temporal characteristics of the scattered field. Here, we focus on imaging Lambertian objects embedded in highly scattering media, where signal photons are dramatically attenuated during propagation and highly coupled with background photons. We address these challenges by providing a time-to-space boundary migration model (BMM) of the scattered field to convert the scattered measurements in spectral form to the scene information in the temporal domain using all of the optical signals. The experiments are conducted under two typical scattering scenarios: 2D and 3D Lambertian objects embedded in the polyethylene foam and the fog, which demonstrate the effectiveness of the proposed algorithm. It outperforms related works including time gating in terms of reconstruction precision and scattering strength. Even though the proportion of signal photons is only 0.75%, Lambertian objects located at more than 25 transport mean free paths (TMFPs), corresponding to the round-trip scattering length of more than 50 TMFPs, can be reconstructed. Also, the proposed method provides low reconstruction complexity and millisecond-scale runtime, which significantly benefits its application.

LIT: Light-field Inference of Transparency for Refractive Object Localization

Translucency is prevalent in everyday scenes. As such, perception of transparent objects is essential for robots to perform manipulation. Compared with texture-rich or texture-less Lambertian objects, transparency induces significant uncertainty on object appearances. Ambiguity can be due to changes in lighting, viewpoint, and backgrounds, each of which brings challenges to existing object pose estimation algorithms. In this work, we propose LIT, a two-stage method for transparent object pose estimation using light-field sensing and photorealistic rendering. LIT employs multiple filters specific to light-field imagery in deep networks to capture transparent material properties, with robust depth and pose estimators based on generative sampling. Along with the LIT algorithm, we introduce the light-field transparent object dataset ProLIT for the tasks of recognition, localization and pose estimation. With respect to this ProLIT dataset, we demonstrate that LIT can outperform both state-of-the-art end-to-end pose estimation methods and a generative pose estimator on transparent objects.

Solving Uncalibrated Photometric Stereo Using Fewer Images by Jointly Optimizing Low-rank Matrix Completion and Integrability

We introduce a new, integrated approach to uncalibrated photometric stereo. We perform 3D reconstruction of Lambertian objects using multiple images produced by unknown, directional light sources. We show how to formulate a single optimization that includes rank and integrability constraints, allowing also for missing data. We then solve this optimization using the Alternating Direction Method of Multipliers (ADMM). We conduct extensive experimental evaluation on real and synthetic data sets. Our integrated approach is particularly valuable when performing photometric stereo using as few as 4–6 images, since the integrability constraint is capable of improving estimation of the linear subspace of possible solutions. We show good improvements over prior work in these cases.

Reconstruction of normal and albedo of convex Lambertian objects by solving ambiguity matrices using SVD and optimization method

Photometric stereo (PMS) can reconstruct shape and albedo of an object by using multiple images captured under varied illumination directions. However, PMS may fail if light intensity is varied across different images captured under different unknown lighting directions. This paper presents a method that can estimate shapes and albedo of inhomogeneous Lambertian objects with much less constrained lighting conditions, i.e. the illumination directions are unknown and there can be arbitrary combination of different light sources and ambient light; meanwhile the light intensity can be different in different images. By placing a reference object alongside an object, the ambiguous matrix produced by SVD can be estimated effectively. This matrix is then used to generate more accurate shape and albedo. The reconstructed results are further refined using an optimization algorithm. Both synthetic and real objects are used in our experiments and the results show the effectiveness of our method.

Depth Estimation and Specular Removal for Glossy Surfaces Using Point and Line Consistency with Light-Field Cameras.

Light-field cameras have now become available in both consumer and industrial applications, and recent papers have demonstrated practical algorithms for depth recovery from a passive single-shot capture. However, current light-field depth estimation methods are designed for Lambertian objects and fail or degrade for glossy or specular surfaces. The standard Lambertian photoconsistency measure considers the variance of different views, effectively enforcing point-consistency, i.e., that all views map to the same point in RGB space. This variance or point-consistency condition is a poor metric for glossy surfaces. In this paper, we present a novel theory of the relationship between light-field data and reflectance from the dichromatic model. We present a physically-based and practical method to estimate the light source color and separate specularity. We present a new photo consistency metric, line-consistency, which represents how viewpoint changes affect specular points. We then show how the new metric can be used in combination with the standard Lambertian variance or point-consistency measure to give us results that are robust against scenes with glossy surfaces. With our analysis, we can also robustly estimate multiple light source colors and remove the specular component from glossy objects. We show that our method outperforms current state-of-the-art specular removal and depth estimation algorithms in multiple real world scenarios using the consumer Lytro and Lytro Illum light field cameras.

Inverse rendering of Lambertian surfaces using subspace methods.

We propose a vector space approach for inverse rendering of a Lambertian convex object with distant light sources. In this problem, the texture of the object and arbitrary lightings are both to be recovered from multiple images of the object and its 3D model. Our work is motivated by the observation that all possible images of a Lambertian object lie around a low-dimensional linear subspace spanned by the first few spherical harmonics. The inverse rendering can therefore be formulated as a matrix factorization, in which the basis of the subspace is encoded in a spherical harmonic matrix S associated with the object’s geometry. A necessary and sufficient condition on S for unique factorization is derived with an introduction to a new notion of matrix rank called nonseparable full rank. A singular value decomposition-based algorithm for exact factorization in the noiseless case is introduced. In the presence of noise, two algorithms, namely, alternating and optimization based are proposed to deal with two different types of noise. A random sample consensus-based algorithm is introduced to reduce the size of the optimization problem, which is equal to the number of pixels in each image. Implementations of the proposed algorithms are done on a real data set.

Towards efficient image irradiance modelling of convex Lambertian surfaces under single viewpoint and frontal illumination

Under local illumination assumption, phenomenological appearance models capture surface appearance through the mathematical modelling of the reflection process. Theoretically, due to the arbitrariness of the lighting function, the space of all possible images of a fixed-pose object under all possible illumination conditions is infinite dimensional. Nonetheless, due to their low- frequency nature, irradiance signals can be represented using low-order basis functions, where spherical harmonics (SH) has been extensively adopted. When capturing image irradiance from a single viewpoint, the visible part of the object's surface constructs the upper hemisphere of the surface normals where the SH is no longer orthonormal. In this paper, we propose the use of hemispherical harmonics (HSH) to model image irradiance of convex Lambertian objects perceived from single viewpoint under unknown distant as well as near illumination. We prove analytically, and validate experimentally, that the Lambertian reflectance kernel has a more compact harmonic expansion in the hemispherical domain when compared to its spherical counterpart. Our experiments illustrate that, despite of having poor approximation accuracy under very close lights, such behavior improves exponentially with little increase in the distance to the light source relative to the object size.

Joint Albedo Estimation and Pose Tracking from Video

The albedo of a Lambertian object is a surface property that contributes to an object's appearance under changing illumination. As a signature independent of illumination, the albedo is useful for object recognition. Single image-based albedo estimation algorithms suffer due to shadows and non-Lambertian effects of the image. In this paper, we propose a sequential algorithm to estimate the albedo from a sequence of images of a known 3D object in varying poses and illumination conditions. We first show that by knowing/estimating the pose of the object at each frame of a sequence, the object's albedo can be efficiently estimated using a Kalman filter. We then extend this for the case of unknown pose by simultaneously tracking the pose as well as updating the albedo through a Rao-Blackwellized particle filter (RBPF). More specifically, the albedo is marginalized from the posterior distribution and estimated analytically using the Kalman filter, while the pose parameters are estimated using importance sampling and by minimizing the projection error of the face onto its spherical harmonic subspace, which results in an illumination-insensitive pose tracking algorithm. Illustrations and experiments are provided to validate the effectiveness of the approach using various synthetic and real sequences followed by applications to unconstrained, video-based face recognition.

Lighting Estimation of a Convex Lambertian Object Using Redundant Spherical Harmonic Frames

An explicit lighting estimation from a single image of Lambertian objects is influenced by two factors: data incompletion and noise contamination. Measurement of lighting consistency purely using the orthogonal spherical harmonic basis cannot achieve an accurate estimation. We present a novel signal-processing framework to represent the lighting field. We construct a redundant spherical harmonic frame with geometric symmetry on the sphere S 2. Spherical harmonic frames are defined over the generating rotation matrices about symmetry axes of finite symmetry subgroups of SO(3), and the generating functions are spherical harmonic basis functions. Compared with the orthogonal spherical harmonic basis, the redundant spherical harmonic frames not only describe the multidirectional lighting distribution intuitively, but also resist the noise theoretically. Subsequently, we analyze the relationship of the irradiance to the incoming radiance in terms of spherical harmonic frames, and reconstruct the lighting function filtered by the Lambertian BRDF (bidirectional reflectance distribution function). The experiments show that the frame coefficients of spherical harmonic frames can better characterize the complex lighting environments finely and robustly.

Lighting estimation of a convex Lambertian object using weighted spherical harmonic frames

An explicit lighting estimation from a single image of Lambertian objects is influenced by two factors: data incompletion and noise contamination. Measurement of lighting consistency purely using the orthogonal spherical harmonic basis cannot achieve an accurate estimation. We present a novel signal-processing framework to represent the lighting field. We construct a weighted spherical harmonic frame with geometric symmetry on the sphere $${S^2}$$ . Weighted spherical harmonic frames are defined over the generating rotation matrices about symmetry axes of finite symmetry subgroups of $$SO(3)$$ , and the generating functions are weighted spherical harmonic basis functions. Compared with the orthogonal spherical harmonic basis, the redundant weighted spherical harmonic frames not only describe the multidirectional lighting distribution intuitively, but also resist the noise theoretically. Subsequently, we analyze the relationship of the irradiance to the incoming radiance in terms of weighted spherical harmonic frames and reconstruct the lighting function filtered by the Lambertian BRDF (bidirectional reflectance distribution function). The experiments show that the frame coefficients of weighted spherical harmonic frames can better characterize the complex lighting environments finely and robustly.

Fusing Multiview and Photometric Stereo for 3D Reconstruction under Uncalibrated Illumination

We propose a method to obtain a complete and accurate 3D model from multiview images captured under a variety of unknown illuminations. Based on recent results showing that for Lambertian objects, general illumination can be approximated well using low-order spherical harmonics, we develop a robust alternating approach to recover surface normals. Surface normals are initialized using a multi-illumination multiview stereo algorithm, then refined using a robust alternating optimization method based on the l(1) metric. Erroneous normal estimates are detected using a shape prior. Finally, the computed normals are used to improve the preliminary 3D model. The reconstruction system achieves watertight and robust 3D reconstruction while neither requiring manual interactions nor imposing any constraints on the illumination. Experimental results on both real world and synthetic data show that the technique can acquire accurate 3D models for Lambertian surfaces, and even tolerates small violations of the Lambertian assumption.

Robustly registering range images using local distribution of albedo

We propose a robust method for registering overlapping range images of a Lambertian object under a rough estimate of illumination. Because reflectance properties are invariant to changes in illumination, the albedo is promising to range image registration of Lambertian objects lacking in discriminative geometric features under variable illumination. We use adaptive regions in our method to model the local distribution of albedo, which enables us to stably extract the reliable attributes of each point against illumination estimates. We use a level-set method to grow robust and adaptive regions to define these attributes. A similarity metric between two attributes is also defined to match points in the overlapping area. Moreover, remaining mismatches are efficiently removed using the rigidity constraint of surfaces. Our experiments using synthetic and real data demonstrate the robustness and effectiveness of our proposed method.

Body color sets: A compact and reliable representation of images

This paper proposes a novel definition of color sets, called body color sets and lines. The proposed technique refers to the dichromatic reflectance model, which states that the colors of a uniform lambertian object are roughly located around straight vectors going from the black to the body color, in relation to physical properties of the object. Ends of body vectors are robustly detected, from the clearest to the darkest through a multi-level histogram analysis, a key step of the algorithm. Finally, instead of defining the topographic map along the one and only luminance direction, body lines are designed along each body vector. The resulting topographic map is more compact and requires less executing times and resources than in previous works [gouiffes08]. Experimental results show that this approach provides a better trade-off between compactness and quality of a topographic map. Furthermore, it provides increased stability under temperature changes of the illuminant.

Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions

In this paper, we present a new method to modify the appearance of a face image by manipulating the illumination condition, when the face geometry and albedo information is unknown. This problem is particularly difficult when there is only a single image of the subject available. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using a spherical harmonic representation. Moreover, morphable models are statistical ensembles of facial properties such as shape and texture. In this paper, we integrate spherical harmonics into the morphable model framework by proposing a 3D spherical harmonic basis morphable model (SHBMM). The proposed method can represent a face under arbitrary unknown lighting and pose simply by three low-dimensional vectors, i.e., shape parameters, spherical harmonic basis parameters, and illumination coefficients, which are called the SHBMM parameters. However, when the image was taken under an extreme lighting condition, the approximation error can be large, thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion-based framework that uses a Markov random field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to extreme lighting conditions, but also insensitive to partial occlusions. The performance of our framework is demonstrated through various experimental results, including the improved rates for face recognition under extreme lighting conditions.

Recovering Shape by Shading and Stereo Under Lambertian Shading Model

A method that integrates shape from shading and stereo is reported for Lambertian objects. A rectification is proposed to convert any lighting direction from oblique to orthographic. A sparse stereo method is reported that directly uses depth information and has no foreshortening problem. The method completely solves three difficult problems in stereo, namely, recovering depth at occlusion; matching at places with similar shading and matching at smooth silhouettes. The method has been tested on both synthetic and real images. It shows superior performance compared with two recent stereo algorithms. It is also a method based on the physics of image formation.

Reconstructing 3D Shape, Albedo and Illumination from a Single Face Image

AbstractThe morphable model has been employed to efficiently describe 3D face shape and the associated albedo with a reduced set of basis vectors. The spherical harmonics (SH) model provides a compact basis to well approximate the image appearance of a Lambertian object under different illumination conditions. Recently, the SH and morphable models have been integrated for 3D face shape reconstruction. However, the reconstructed 3D shape is either inconsistent with the SH bases or obtained just from landmarks only. In this work, we propose a geometrically consistent algorithm to reconstruct the 3D face shape and the associated albedo from a single face image iteratively by combining the morphable model and the SH model. The reconstructed 3D face geometry can uniquely determine the SH bases, therefore the optimal 3D face model can be obtained by minimizing the error between the input face image and a linear combination of the associated SH bases. In this way, we are able to preserve the consistency between the 3D geometry and the SH model, thus refining the 3D shape reconstruction recursively. Furthermore, we present a novel approach to recover the illumination condition from the estimated weighting vector for the SH bases in a constrained optimization formulation independent of the 3D geometry. Experimental results show the effectiveness and accuracy of the proposed face reconstruction and illumination estimation algorithm under different face poses and multiple‐light‐source illumination conditions.