Specular Reflection Components Research Articles

Computational ghost imaging with adaptive intensity illumination for scenes featuring specular surfaces

Imaging a target scene with specular surfaces is a daunting challenge for both direct imaging and indirect computational imaging techniques. The intense specular reflection component during the measurement severely degrades the quality of the reconstructed image, resulting in a substantial loss of scene information. To address this issue, we propose a computational ghost imaging (CGI) method with adaptive intensity illumination. Capitalizing on the encoded imaging feature of CGI, this method enables effective imaging of target scenes with specular surfaces through two series of measurements, eliminating the necessity for additional optical components. Based on the position and intensity information of pixels in the specular regions from the first series of measurements, our method modulates the illumination patterns to weaken the intensity of the specular region in the second series of measurements. Simulation and experimental results demonstrate that the utilization of these modulated illumination patterns for target scene measurement effectively mitigates interference from the specular surface during imaging. Consequently, the reconstructed image is capable of presenting more detailed information about the target scene other than the specular regions. Our work introduces a novel approach for imaging target scenes with specular surfaces and broadens the scope of applications for CGI in reality.

Learning physically based material and lighting decompositions for face editing

Lighting is crucial for portrait photography, yet the complex interactions between the skin and incident light are expensive to model computationally in graphics and difficult to reconstruct analytically via computer vision. Alternatively, to allow fast and controllable reflectance and lighting editing, we developed a physically based decomposition through deep learned priors from path-traced portrait images. Previous approaches that used simplified material models or low-frequency or low-dynamic-range lighting struggled to model specular reflections or relight directly without intermediate decomposition. However, we estimate the surface normal, skin albedo and roughness, and high-frequency HDRI maps, and propose an architecture to estimate both diffuse and specular reflectance components. In our experiments, we show that this approach can represent the true appearance function more effectively than simpler baseline methods, leading to better generalization and higher-quality editing.

18‐1: Combining Annulus and Variable Aperture Source Methods to Separate Specular, Haze and Lambertian Reflection Components of ePaper Displays

The annulus light source method separates specular and diffuse components of reflected glare from display surfaces. Combining it with the variable aperture source method delivers source‐size dependence. Measuring diagnostic reflection samples and ePaper with mirror‐like or scattering surfaces confirmed that the source‐size dependence of reflected glare comes from haze scatter.

Three Dimensional Shape Reconstruction via Polarization Imaging and Deep Learning

Deep-learning-based polarization 3D imaging techniques, which train networks in a data-driven manner, are capable of estimating a target’s surface normal distribution under passive lighting conditions. However, existing methods have limitations in restoring target texture details and accurately estimating surface normals. Information loss can occur in the fine-textured areas of the target during the reconstruction process, which can result in inaccurate normal estimation and reduce the overall reconstruction accuracy. The proposed method enables extraction of more comprehensive information, mitigates the loss of texture information during object reconstruction, enhances the accuracy of surface normal estimation, and facilitates more comprehensive and precise reconstruction of objects. The proposed networks optimize the polarization representation input by utilizing the Stokes-vector-based parameter, in addition to separated specular and diffuse reflection components. This approach reduces the impact of background noise, extracts more relevant polarization features of the target, and provides more accurate cues for restoration of surface normals. Experiments are performed using both the DeepSfP dataset and newly collected data. The results show that the proposed model can provide more accurate surface normal estimates. Compared to the UNet architecture-based method, the mean angular error is reduced by 19%, calculation time is reduced by 62%, and the model size is reduced by 11%.

A Fast Specular Highlight Removal Method for Smooth Liquor Bottle Surface Combined with U2-Net and LaMa Model

Highlight removal is a critical and challenging problem. In view of the complex highlight phenomenon on the surface of smooth liquor bottles in natural scenes, the traditional highlight removal algorithms cannot semantically disambiguate between all-white or near-white materials and highlights, and the recent highlight removal algorithms based on deep learning lack flexibility in network architecture, have network training difficulties and have insufficient object applicability. As a result, they cannot accurately locate and remove highlights in the face of some small sample highlight datasets with strong pertinence, which reduces the performance of some tasks. Therefore, this paper proposes a fast highlight removal method combining U2-Net and LaMa. The method consists of two stages. In the first stage, the U2-Net network is used to detect the specular reflection component in the liquor bottle input image and generate the mask map for the highlight area in batches. In the second stage, the liquor bottle input image and the mask map generated by the U2-Net are input to the LaMa network, and the surface highlights of the smooth liquor bottle are removed by relying on the powerful image inpainting performance of LaMa. Experiments on our self-made liquor bottle surface highlight dataset showed that this method outperformed other advanced methods in highlight detection and removal.

AvatarMe++: Facial Shape and BRDF Inference With Photorealistic Rendering-Aware GANs.

Over the last years, with the advent of Generative Adversarial Networks (GANs), many face analysis tasks have accomplished astounding performance, with applications including, but not limited to, face generation and 3D face reconstruction from a single "in-the-wild" image. Nevertheless, to the best of our knowledge, there is no method which can produce render-ready high-resolution 3D faces from "in-the-wild" images and this can be attributed to the: (a) scarcity of available data for training, and (b) lack of robust methodologies that can successfully be applied on very high-resolution data. In this paper, we introduce the first method that is able to reconstruct photorealistic render-ready 3D facial geometry and BRDF from a single "in-the-wild" image. To achieve this, we capture a large dataset of facial shape and reflectance, which we have made public. Moreover, we define a fast and photorealistic differentiable rendering methodology with accurate facial skin diffuse and specular reflection, self-occlusion and subsurface scattering approximation. With this, we train a network that disentangles the facial diffuse and specular reflectance components from a mesh and texture with baked illumination, scanned or reconstructed with a 3DMM fitting method. As we demonstrate in a series of qualitative and quantitative experiments, our method outperforms the existing arts by a significant margin and reconstructs authentic, 4K by 6K-resolution 3D faces from a single low-resolution image, that are ready to be rendered in various applications and bridge the uncanny valley.

Exploring the effect of pavement reflection and photometric properties on road lighting performance

The reflection properties of pavement have an impact on the lit environment and thus upon drivers’ vision and comfort and the energy consumption of the lighting installation. The reflection properties combine diffuse and specular components. The specular reflection component changes with different materials: it also changes when the surface is wet, although this is sometimes ignored in lighting design. This study used simulation (DIALux 4.13) to investigate the effect on lighting parameters (luminance, overall uniformity, longitudinal uniformity and threshold increment (TI)) of changes in pavement reflection properties using different pavement materials and under dry and wet conditions. Furthermore, comparison of lighting parameters was made with different road lighting arrangements. The results show that an increase in the specular reflection component leads to an increase in luminance and a decrease in uniformity. Of the surfaces investigated, the porous pavement had the lower luminance but better uniformity. Arranging the lighting installation based on the extreme wettest condition could make luminance and uniformity rise but with an increase of 2–2.5 kWh/m2 annual energy consumption. When trying to control glare problems during design process, it is suggested that uniformity cannot be neglected except the TI.

Identifying specular highlights: Insights from deep learning.

Specular highlights are the most important image feature for surface gloss perception. Yet, recognizing whether a bright patch in an image is due to specular reflection or some other cause (e.g., texture marking) is challenging, and it remains unclear how the visual system reliably identifies highlights. There is currently no image-computable model that emulates human highlight identification, so here we sought to develop a neural network that reproduces observers’ characteristic successes and failures. We rendered 179,085 images of glossy, undulating, textured surfaces. Given such images as input, a feedforward convolutional neural network was trained to output an image containing only the specular reflectance component. Participants viewed such images and reported whether or not specific pixels were highlights. The queried pixels were carefully selected to distinguish between ground truth and a simple thresholding of image intensity. The neural network outperformed the simple thresholding model—and ground truth—at predicting human responses. We then used a genetic algorithm to selectively delete connections within the neural network to identify variants of the network that approximated human judgments even more closely. The best resulting network shared 68% of the variance with human judgments—more than the unpruned network. As a first step toward interpreting the network, we then used representational similarity analysis to compare its inner representations to a wide variety of hand-engineered image features. We find that the network learns representations that are similar not only to directly image-computable predictors but also to more complex predictors such as intrinsic or geometric factors, as well as some indications of photo-geometrical constraints learned by the network. However, our network fails to replicate human response patterns to violations of photo-geometric constraints (rotated highlights) as described by other authors.

Single-Image Specular Highlight Removal via Real-World Dataset Construction

Specular reflections pose great challenges on various multimedia and computer vision tasks, <i>e.g.</i>, image segmentation, detection and matching. In this paper, we build a large-scale Paired Specular-Diffuse (PSD) image dataset, where the images are carefully captured by using real-world objects and the ground-truth specular-free diffuse images are provided. To the best of our knowledge, this is the first real-world benchmark dataset for specular highlight removal task, which is useful for evaluating and encouraging new deep learning-based approaches. Given this dataset, we present a novel Generative Adversarial Network (GAN) for specular highlight removal from a single image by introducing the detection of specular reflection information as a guidance. Our network also makes full use of the attention mechanism and is able to directly model the mapping relation between the diffuse area and the specular highlight area without any explicit estimation of the illumination. Experimental results demonstrate that the proposed network is more effective to remove specular reflection components with the guidance of specular highlight detection than recent state-of-the-art methods.

Glossy Appearance Editing for Heterogeneous Material Objects

Abstract With the proliferation of smartphones and social networking services, the opportunities for individuals to take photographs have increased exponentially. In a previous study, the perceived gloss of an object was reduced by its representing as a digital image and compared with a real object. It is also known that image editing, such as lossy image compression, can reduce the glossiness of an image. Therefore, the glossiness of real objects may be easily changed in digital images; thus, a method for appropriately editing the gloss in digital images is required for post-processing. In this study, we propose a gloss appearance editing method for various material objects in a single digital image. The proposed method consists of three steps: color space conversion, gloss detection, and gloss editing. The relationship between the proposed method and the respective reflection models of inhomogeneous objects, metallic objects, and translucent objects was analyzed. Consequently, we determined that the gloss editing of the proposed method is equivalent to editing the specular reflection component of an inhomogeneous object, the grazing reflection component of a metallic object, and the specular reflection component of a translucent object. We applied the proposed method to test images including objects of various materials and confirmed its effectiveness through a subjective evaluation by visual inspection and an objective evaluation using image statistics.

Optimal Material Search for Infrared Markers under Non-Heating and Heating Conditions.

Research on optimal markers for infrared imaging and differences in their characteristics in the presence of heat sources has not yet been performed. This study investigates optimal material combinations for developing an accurate and detachable infrared marker for multiple conditions in the medium wave infrared (MWIR) region. Based on four requirements, 11 material combinations are systematically evaluated. Consequently, the optimal marker differs in relation to the presence of specular reflection components. Metal–insulator markers are suitable under non-heating and hot-air heating conditions without reflection components, although a printed marker made of copier paper is captured more clearly than metal–insulator markers during heating, using an optical radiation heating source with reflection components. Our findings can be applied in structural health monitoring and multi-modal projection involving heat sources.

Development of a System to Measure the Optical Properties of Facial Skin using a 3D Camera and Projector

We have developed a system to measure both the optical properties of facial skin and the three-dimensional shape of the face. To measure the three-dimensional facial shape, our system uses a light-field camera to provide a focused image and a depth image simultaneously. The light source uses a projector that produces a high-frequency binary illumination pattern to separate the subsurface scattering and surface reflections from the facial skin. Using a dichromatic reflection model, the surface reflection image of the skin can be separated further into a specular reflection component and a diffuse reflection component. Verification using physically controlled objects showed that the separation of the optical properties by the system correlated with the subsurface scattering, specular reflection, or diffuse reflection characteristics of each object. The method presented here opens new possibilities in cosmetology and skin pharmacology for measurement of the skin's gloss and absorption kinetics and the pharmacodynamics of various external agents.

Research and implementation of digital 3D panoramic visual communication technology based on virtual reality

SummaryBased on the digital three‐dimensional (3D) panoramic technology of sequence pictures, the static pictures are stitched together to form a 360‐degree panoramic image, which can truly reproduce the real scene. Panoramic images are widely used, making this technology a hot spot in the research of virtual reality technology. This paper completes the research content of the application value of the digital 3D panorama, the key technology of digital 3D panorama technology, and the design and display of 3D panorama campus through practical application. This article comprehensively introduces the hardware equipment of panoramic technology and the technical requirements of panoramic image acquisition in the field of practical application, and combines the actual cases to systematically produce panoramic image stitching, panoramic image generation, and virtual roaming campus construction; 3D panoramic technology is a brand‐new form of viewing images, discussing and studying the artistic expression of 3D panoramic images, and researching its innovative application in visual communication art. In this paper, the chromaticity calibration of digital camera and projection equipment is used to eliminate the coupling difference between the two; then estimate the spectral reflectance of the three‐color channel of the object and separate the specular reflection component and the diffuse reflection component, at the same time correct the color and specular reflectance of the object itself, and analyze the influence of reflection on the color of the projection light source. This article describes the design ideas and key technologies of the school's 3D panoramic walkthrough system, describes in detail the process of the system from demand analysis to system design and finally realization, and focuses on the production method of panoramic images and the main functional modules in the walkthrough system. At the end, the system was tested, and the existing problems were analyzed, and solutions were provided. Research shows that when the roaming system has 3,000 concurrent users, it can read a single scene for about 3 s on average.

Preliminary study on the separation of specular reflection and backscattering components using synthetic aperture beamforming

In the early stages of atherosclerosis, the luminal surface of the arterial wall becomes rough due to injury and detachment of endothelial cells. This roughening can potentially be estimated with ultrasound since the electrical echo signal from the transducer is sensitive to both the angle of incidence to an extended surface as well as the roughness of the surface. Specifically, as the roughness of an interface increases, specular reflection is substituted by scattering. We propose a method that attempts separation of reflection and backscattering components in the received echo signals. Assuming the predominant propagation directions of the reflected and scattered waves can be somewhat controlled by the emitted sound field, separation of those components was attempted using synthetic aperture imaging with a transmit beam, focused at a point more distant than the imaging depth. Specifically, two dedicated beamforming processes were used for generation of reflection-emphasized and backscattering-emphasized images. Experimental verifications on a phantom using an ultrasound system with a limited number of active transmit-receive channels yielded a difference between these two images of 8dB. The results further showed a similar (slightly improved) lateral spatial resolution size of 0.41mm for the backscattering-emphasized image compared with conventional B-mode imaging (0.47mm). A new technique for separation of the reflection and backscattering components using synthetic aperture beamforming with a transmit beam featuring a large focal distance was proposed. The technique demonstrated a partial separation of the reflection and backscattering components, which potentially may be used to estimate surface roughness.

Use of color information for structured-light 3D shape measurement of objects with shiny surfaces

Objects with shiny surfaces cannot be directly measured using the conventional structured-light method. To cope with this problem, in this paper, we propose a novel method for removing the saturated components in an image. First, the specular pixels in the image are identified using a highlight extraction algorithm. Then, based on the reflection component separation (RCS) theory, the diffuse and specular components of these specular pixels are separated. For objects with shiny surfaces, use of the RCS approach destroys the color information of highlighted pixels with a large specular reflection component. As such, finally, the color information of the highlighted pixels is recovered using an image repair process. Experimental results indicate that 95% of the highlights in the images were eliminated. The highlight caused by the strong reflecting surface can be well suppressed. This proposed method effectively overcomes the interference of reflected light and provides a feasible solution to the problems associated with the structured-light measurement of objects with smooth surfaces.

An effective framework for 3D shape measurement of specular surface based on the dichromatic reflection model

Fringe projection profilometry based on phase-shift method has been commonly used for three-dimensional measurement. However, measuring the specular surface is always a challenging task since the specular area cannot present original intensity information. This paper presents a novel framework for measuring objects with specular surfaces. Firstly, a set of phase-shifted fringe patterns with maximum intensity value is projected onto the specular surface. Then, the dichromatic reflection model is developed to theoretically decompose an image into specular and diffuse reflection components, which identifies specular pixels more robustly and effectively than judging by saturation only. Secondly, since the location and intensity of specular areas are acquired, the best projection intensity corresponding to the specular areas can be determined by fitting a power function that transforms captured specular reflection components to projected intensities, aiming to minimize specular areas and decrease specular reflection components. It should be stated here that the assumption based on this paper is that the fewer specular reflection components, the fewer measurement errors. Subsequently, the adaptive fringe patterns, which are constructed using the best projection intensity, are reprojected for phase recovery. Thirdly, it is reasonable that there still exist regions where the specular reflection components are too high. For these information-loss areas, the pixel inpainting technique is adopted when extracting the disparity map. Finally, the ultimate result combines adaptive fringe reconstruction with pixel inpainting. The experiment results verify that the proposed measurement framework can effectively and robustly perform three-dimensional reconstruction of various objects with specular surfaces.

Study on reflection characteristics of underwater target and laser echo power

For the design of the underwater laser detection system, the echo power equation of underwater target is derived, and the effects of water’s attenuation coefficient and incident angle on target's echo power are analyzed. The bidirectional reflection distribution function (BRDF) of the single-station model is used to study the reflection characteristics of the target. The influence of the correlation coefficients of specular and diffuse reflection components on the reflected light intensity of special axisymmetric targets is studied. The signal-to-noise ratio (SNR) values of three types of target under different surface BRDF are given. The simulation results show that the larger the attenuation coefficient is, the smaller the echo power received, and the incidence angle is closer to 90°, the echo power is equal to 0. A smaller surface slope or a larger diffuse reflected coefficient will cause a smaller reflected intensity.

Polarization-based extracting diffuse reflection from light-field of object surface

&lt;sec&gt;The reflection light field of surface of the Non-Lambertian body in nature has both specular reflection and diffuse reflection components. In the process of three-dimensional(3D) reconstruction, image matching and pattern recognition are based on the ideal Lambert body. The imaging effect is limited due to the presence of specular reflection components, and the accuracy of feature recognition is low. In order to obtain the diffuse reflection component accurately, a large number of studies have been conducted for a long time, which can be mainly divided into two parts: intensity- and polarization-based separation techniques. The intensity-based separation algorithm is limited in many aspects due to the prior knowledge, such as light source chromaticity, direction and image color information. With the maturity of detection technology, the acquisition and interpretation of multi-dimensional physical properties of light-field have made great progress of the utilization of polarization characteristics of light wave. Compared with traditional intensity imaging technology, the polarization imaging technology has strongr and many advantages in highlighting targets. However, in traditional polarization-based separation techniques, it is often necessary to assume that the diffuse light is completely unpolarized, which is used in some specific cases but not universally.&lt;/sec&gt;&lt;sec&gt;In this work, we report a method to obtain the diffuse reflection components of the target surface based on the polarization characteristics of the light-field. According to Fresnel's law and Lawrence B. Wolff's reflection model, the reflected light-field on the target surface can be divided into diffuse and specular components with partial polarization. The partial polarization characteristics of diffuse and specular components are explored in depth and the Stokes vector is used to calculate the minimum light intensity of each pixel modulated by polarizer, which is completely unpolarized light. By subtracting completely unpolarized light from the obtained polarized sub-images, the diffuse and specular components in the polarized part satisfy the linear constraint model. Based on the independent component analysis (ICA) model, the diffuse and specular components in the polarized part are regarded as independent and non-interfering additive vectors. The singular value decomposition method and optical relevancy of mutual information are used to determine the optimal mix coefficients matrix of the subcomponents in the linear constrained model. Thus, the diffuse components are accurately acquired and explained from the complex reflected light-field. &lt;/sec&gt;&lt;sec&gt;Simulation and experimental results show that the algorithm mentioned above can accurately obtain the optimal mix coefficients’ matrix without the prior knowledge of illuminant chromaticity, or direction or image chromatic information, or others. This technique can accurately obtain and remove the specular reflection part, at the same time, restore the diffuse light intensity which is covered by the specular reflection and conforms to the change trend of the surface shape. Meanwhile, the pretty good results also demonstrate that the proposed separation method has the strong stability and wide applicability. This technology does not have to make the assumption that 3D imaging technology and computer vision algorithms such as pattern recognition rely on natural objects as ideal Lambert bodies, and it can eleminate the influence of complex reflected light-field on target results, which makes passive remote 3D imaging technology more applicable and more robust.&lt;/sec&gt;

Single Image Highlight Removal for Real-Time Image Processing Pipelines

This paper presents a fully automatic method for the separation of diffuse and specular reflection components from a single image. Overall, the mechanisms in which the available methods operate on are computationally costly and do not translate well to modern hardware-implemented image processing pipelines, such as the ones present in consumer electronics. Consequently, the objective of this article is to introduce a simple yet effective method for specular highlight removal. It is based on the dichromatic reflection model and operates through histogram matching in the YCbCr color space. The proposed method performs in real-time. It only uses global image statistics and point-wise intensity transformations. Experimental evaluation shows that the proposed approach has competitive results in comparison to state-of-the-art methods. Limitations of the proposed approach are seldom and are common to most methods available. The proposed method, however, achieves better quality results with much less computational cost, thus enabling feasibility in systems with limited processing power.

Enhancement characteristics of laser-induced plasma confined by hemispherical cavities in different materials

A hemispherical cavity can be used to improve the sensitivity of laser-induced breakdown spectroscopy by increasing the emission spectra of the plasma. In this work, laser-induced plasma plume images were obtained with and without confinement. The results have confirmed that the reason for the signal enhancement caused by the hemispherical cavity was the reflected shockwave “compressing” effect. The dependence of spectral signal enhancement on the material property of the cavities was investigated. Four cavities in different materials, which include K-resin, aluminum alloy, photosensitive resin and nylon, were used to confine the plasma. It has shown that the material physical properties of the cavities influence the enhancement effect. We found that the main influencing factor was surface roughness of the material. Smoother cavity surfaces induced stronger enhancement effects due to the larger specular reflection component of the reflected shockwave and the more intense subsequent interaction between the plasma and shockwave. The spectral intensities of the characteristic lines approximately doubled with surface roughness values changing from 0.2 to 16.1.