Illumination Estimation Research Articles

Forgery detection using multiple light sources for synthetic images

Image processing requires an estimation of position, direction and illumination of the objects present in a scene. Nowadays, digital image manipulations have experienced a remarkable growth. These image manipulations can be detected by various forgery detection techniques that have been developed in the past few years. We describe a forgery detection technique based on estimations of multiple light source directions. This method uses a pixel patch from the image region to estimate the source light vector. The implementation is done for images where one and more light sources are available in the scene. The proposed technique is able to identify image forgery in terms of elevation angle α obtained from a source of light and surface normal. This method is tested for both outdoor and indoor images under certain known parameters. The novelty of our method is that photo manipulation detection is done using a multiple light source detection approach. We demonstrate that the proposed technique produces accurate results by making certain assumptions about surface properties and illumination parameters.

Color Enhancement With Adaptive Illumination Estimation for Low-Backlighted Displays

In this paper, we propose an image enhancement system to maintain human visual perception for low-backlighted LCD or LED displays. Adopting the low-backlight mode can save electrical power and extend battery life. First, we examine the relationship between the image and backlight to maintain visual perceptual quality. Then, we use an adaptive illumination estimation to decompose the image intensity into an illumination layer and a reflectance layer with multiscale and parallel perspectives. We then refer to the given backlight level and our derived image-backlight relationship to compensate the illumination layer and meanwhile enhance the reflectance layer. Finally, we also boost the color saturation. Experiments using both numerical blind/referenceless image spatial quality evaluator and subjective mean opinion score demonstrate that our system outperforms existing systems.

Improving color constancy by discounting the variation of camera spectral sensitivity

It is an ill-posed problem to recover the true scene colors from a color biased image by discounting the effects of scene illuminant and camera spectral sensitivity (CSS) at the same time. Most color constancy (CC) models have been designed to first estimate the illuminant color, which is then removed from the color biased image to obtain an image taken under white light, without the explicit consideration of CSS effect on CC. This paper first studies the CSS effect on illuminant estimation arising in the inter-dataset-based CC (inter-CC), i.e., training a CC model on one dataset and then testing on another dataset captured by a distinct CSS. We show the clear degradation of existing CC models for inter-CC application. Then a simple way is proposed to overcome such degradation by first learning quickly a transform matrix between the two distinct CSSs (CSS-1 and CSS-2). The learned matrix is then used to convert the data (including the illuminant ground truth and the color-biased images) rendered under CSS-1 into CSS-2, and then train and apply the CC model on the color-biased images under CSS-2 without the need of burdensome acquiring of the training set under CSS-2. Extensive experiments on synthetic and real images show that our method can clearly improve the inter-CC performance for traditional CC algorithms. We suggest that, by taking the CSS effect into account, it is more likely to obtain the truly color constant images invariant to the changes of both illuminant and camera sensors.

Combinational illumination estimation method based on image-specific PCA filters and support vector regression

Accurate illuminant estimation from digital image data is a fundamental step of practically every image colour correction. Combinational illuminant estimation schemes have been shown to improve estimation accuracy significantly compared to other colour constancy algorithms. These schemes combine individual estimates of simpler colour constancy algorithms in some ‘intelligent’ manner into a joint and, usually, more efficient illuminant estimation. Among them, a combinational method based on Support Vector Regression (SVR) was proposed recently, demonstrating the more accurate illuminant estimation (Li et al. IEEE Trans. Image Process. 23(3), 1194–1209, 2014). We extended this method by our previously introduced convolutional framework, in which the illuminant was estimated by a set of image-specific filters generated using a linear analysis. In this work, the convolutional framework was reformulated, so that each image-specific filter obtained by principal component analysis (PCA) produced one illuminant estimate. All these individual estimates were then combined into a joint illuminant estimation by using SVR. Each illuminant estimation by using a single image-specific PCA filter within the convolutional framework actually represented one base algorithm for the combinational method based on SVR. The proposed method was validated on the well-known Gehler image dataset, reprocessed and prepared by author Shi, and, as well, on the NUS multi-camera dataset. It was shown that the median and trimean angular errors were (non-significantly) lower for our proposed method compared to the original combinational method based on SVR for which our method utilized just 6 image-specific PCA filters, while the original combinational method required 12 base algorithms for similar results. Nevertheless, a proposed method unified grey edge framework, PCA analysis, linear filtering theory, and SVR regression formally for the combinational illuminant estimation.

Illuminant estimation in multispectral imaging.

With the advancement in sensor technology, the use of multispectral imaging is gaining wide popularity for computer vision applications. Multispectral imaging is used to achieve better discrimination between the radiance spectra, as compared to the color images. However, it is still sensitive to illumination changes. This study evaluates the potential evolution of illuminant estimation models from color to multispectral imaging. We first present a state of the art on computational color constancy and then extend a set of algorithms to use them in multispectral imaging. We investigate the influence of camera spectral sensitivities and the number of channels. Experiments are performed on simulations over hyperspectral data. The outcomes indicate that extension of computational color constancy algorithms from color to spectral gives promising results and may have the potential to lead towards efficient and stable representation across illuminants. However, this is highly dependent on spectral sensitivities and noise. We believe that the development of illuminant invariant multispectral imaging systems will be a key enabler for further use of this technology.

Single and Multiple Illuminant Estimation Using Convolutional Neural Networks.

In this paper, we present a three-stage method for the estimation of the color of the illuminant in RAW images. The first stage uses a convolutional neural network that has been specially designed to produce multiple local estimates of the illuminant. The second stage, given the local estimates, determines the number of illuminants in the scene. Finally, local illuminant estimates are refined by non-linear local aggregation, resulting in a global estimate in case of single illuminant. An extensive comparison with both local and global illuminant estimation methods in the state of the art, on standard data sets with single and multiple illuminants, proves the effectiveness of our method.

Static Scene Illumination Estimation from Videos with Applications

We present a system that automatically recovers scene geometry and illumination from a video, providing a basis for various applications. Previous image based illumination estimation methods require either user interaction or external information in the form of a database. We adopt structure-from-motion and multi-view stereo for initial scene reconstruction, and then estimate an environment map represented by spherical harmonics (as these perform better than other bases). We also demonstrate several video editing applications that exploit the recovered geometry and illumination, including object insertion (e.g., for augmented reality), shadow detection, and video relighting.

Multiple illuminant estimation from the covariance of colors

In this paper we present a single and a multiple illuminant estimation physics-based algorithm. Both algorithms are based on the mean projections maximization assumption and un-centered component analysis. The proposed assumption is validated for a large collection of images and later used to estimate the illuminant color. The multiple illuminant estimator assumes that the spectral power distribution of the light source is not the same for the whole scene, which is the case for a wide range of images. In such cases, our new multiple illuminant estimator recovers an accurate illuminants estimates map for each input image while maintaining speed. The evaluation of the proposed algorithms on different real image datasets is realized. The experimental results are satisfying; our algorithms maximize the trade-off between accuracy (illuminant estimation error) and computational complexity.

Constrained low-rank gamut completion for robust illumination estimation

Illumination estimation is an important component of color constancy and automatic white balancing. According to recent survey and evaluation work, the supervised methods with a learning phase are competitive for illumination estimation. However, the robustness and performance of any supervised algorithm suffer from an incomplete gamut in training image sets because of limited reflectance surfaces in a scene. In order to address this problem, we present a constrained low-rank gamut completion algorithm, which can replenish gamut from limited surfaces in an image, for robust illumination estimation. In the proposed algorithm, we first discuss why the gamut completion is actually a low-rank matrix completion problem. Then a constrained low-rank matrix completion framework is proposed by adding illumination similarities among the training images as an additional constraint. An optimization algorithm is also given out by extending the augmented Lagrange multipliers. Finally, the completed gamut based on the proposed algorithm is fed into the support vector regression (SVR)-based illumination estimation method to evaluate the effect of gamut completion. The experimental results on both synthetic and real-world image sets show that the proposed gamut completion model not only can effectively improve the performance of the original SVR method but is also robust to the surface insufficiency in training samples.

Differences in illumination estimation in #thedress

Differences in illumination estimation in #thedress

Differences in illumination estimation in #thedress.

We investigated whether people who report different colors for #thedress do so because they have different assumptions about the illumination in #thedress scene. We introduced a spherical illumination probe (Koenderink, Pont, van Doorn, Kappers, & Todd, 2007) into the original photograph, placed in fore-, or background of the scene and-for each location-let observers manipulate the probe's chromaticity, intensity and the direction of the illumination. Their task was to adjust the probe such that it would appear as a white sphere in the scene. When the probe was located in the foreground, observers who reported the dress to be white (white perceivers) tended to produce bluer adjustments than observers who reported it as blue (blue perceivers). Blue perceivers tended to perceive the illumination as less chromatic. There were no differences in chromaticity settings between perceiver types for the probe placed in the background. Perceiver types also did not differ in their illumination intensity and direction estimates across probe locations. These results provide direct support for the idea that the ambiguity in the perceived color of the dress can be explained by the different assumptions that people have about the illumination chromaticity in the foreground of the scene. In a second experiment we explore the possibility that blue perceivers might overall be less sensitive to contextual cues, and measure white and blue perceivers' dress color matches and labels for manipulated versions of the original photo. Results indeed confirm that contextual cues predominantly affect white perceivers.

Estimating Illumination Chromaticity Based on Structured Support Vector Machine

Illumination estimation is crucial to the color constancy problem. Most existing color constancy algorithms assume uniform illumination. However, in real-world scenes, this is not often the case. Thus, we propose a novel framework for estimation the colors of multiple illuminants and their spatial distribution in the scene. We formulate this problem as an illuminant color estimate is obtained independently from distinct image main-regions. We think there is only one kind of light in each main-regions. From these main-regions a robust local illumination color is computed by consensus. So we can solve the problem of multiple illumination estimation using single illuminant color constancy method. The performance if our method is evaluated on multiple data sets. Experimental results show that we can reliably process mixed illuminant scenes.

Directional Illumination Estimation Sets and Multilevel Matching Metric for Illumination-Robust Face Recognition

It is a challenging task to improve the performance of face recognition under complex illumination conditions. Illumination estimation-based illumination invariant extraction is widely used to alleviate the adverse effects of illumination variation on face recognition. Most existing methods only used slowly changing characteristics of lighting to achieve illumination estimation, thus resulting in inaccurate illumination estimation and illumination invariant extraction under complex illumination conditions. To alleviate this issue, on the basis of the Lambertian reflectance model, we propose an innovative method of directional illumination estimation to extract directional illumination invariant sets from a facial image. The directional illumination invariant sets not only better preserve essential features of the face, but also largely reduce adverse effects of rapid light changes. Moreover, we propose a multilevel matching metric for category classification by using an inner product measure and residual matching. Experimental results on Yale $\text{B}^{+}$ , CAS-PEAL-R1, uncontrolled and AR face databases validate that the proposed method can effectively improve the accuracy of face recognition under complex illumination conditions.

Sun position estimation and tracking for virtual object placement in time-lapse videos

Realistic illumination of virtual objects placed in real videos is important in terms of achieving visual coherence. We propose a novel approach for illumination estimation on time-lapse videos and seamlessly insert virtual objects in these videos in a visually consistent way. The proposed approach works for both outdoor and indoor environments where the main light source is the Sun. We first modify an existing illumination estimation method that aims to obtain sparse radiance map of the environment in order to estimate the initial Sun position. We then track the hard ground shadows on the time-lapse video by using an energy-based pixel-wise method. The proposed method aims to track the shadows by utilizing the energy values of the pixels that forms them. We tested the method on various time-lapse videos recorded in outdoor and indoor environments and obtained successful results.

Illumination correction of dyed fabrics approach using Bagging-based ensemble particle swarm optimization–extreme learning machine

Changes in illumination will result in serious color difference evaluation errors during the dyeing process. A Bagging-based ensemble extreme learning machine (ELM) mechanism hybridized with particle swarm optimization (PSO), namely Bagging–PSO–ELM, is proposed to develop an accurate illumination correction model for dyed fabrics. The model adopts PSO algorithm to optimize the input weights and hidden biases for the ELM neural network called PSO–ELM, which enhances the performance of ELM. Meanwhile, to further increase the prediction accuracy, a Bagging ensemble scheme is used to construct an independent PSO–ELM learning machine by taking bootstrap replicates of the training set. Then, the obtained multiple different PSO–ELM learners are aggregated to establish the prediction model. The proposed prediction model is evaluated with real dyed fabric images and discussed in comparison with several related methods. Experimental results show that the ensemble color constancy method is able to generate a more robust illuminant estimation model with better generalization performance.

Estimation of illuminant in the white/gold and blue/black "dress" image

Estimation of illuminant in the white/gold and blue/black "dress" image

Multiple Illuminant Color Estimation via Statistical Inference on Factor Graphs.

This paper presents a method to recover a spatially varying illuminant color estimate from scenes lit by multiple light sources. Starting with the image formation process, we formulate the illuminant recovery problem in a statistically data-driven setting. To do this, we use a factor graph defined across the scale space of the input image. In the graph, we utilize a set of illuminant prototypes computed using a data driven approach. As a result, our method delivers a pixelwise illuminant color estimate being devoid of libraries or user input. The use of a factor graph also allows for the illuminant estimates to be recovered making use of a maximum a posteriori inference process. Moreover, we compute the probability marginals by performing a Delaunay triangulation on our factor graph. We illustrate the utility of our method for pixelwise illuminant color recovery on widely available data sets and compare against a number of alternatives. We also show sample color correction results on real-world images.

Enhanced retinal modeling for face recognition and facial feature point detection under complex illumination conditions

We improved classic retinal modeling to alleviate the adverse effect of complex illumination on face recognition and extracted robust image features. Our improvements on classic retinal modeling included three aspects. First, a combined filtering scheme was applied to simulate functions of horizontal and amacrine cells for accurate local illumination estimation. Second, we developed an optimal threshold method for illumination classification. Finally, we proposed an adaptive factor acquisition model based on the arctangent function. Experimental results on the combined Yale B; the Carnegie Mellon University poses, illumination, and expression; and the Labeled Face Parts in the Wild databases show that the proposed method can effectively alleviate illumination difference of images under complex illumination conditions, which is helpful for improving the accuracy of face recognition and that of facial feature point detection.

Approaching the computational color constancy as a classification problem through deep learning

Computational color constancy refers to the problem of computing the illuminant color so that the images of a scene under varying illumination can be normalized to an image under the canonical illumination. In this paper, we adopt a deep learning framework for the illumination estimation problem. The proposed method works under the assumption of uniform illumination over the scene and aims for the accurate illuminant color computation. Specifically, we trained the convolutional neural network to solve the problem by casting the color constancy problem as an illumination classification problem. We designed the deep learning architecture so that the output of the network can be directly used for computing the color of the illumination. Experimental results show that our deep network is able to extract useful features for the illumination estimation and our method outperforms all previous color constancy methods on multiple test datasets.

Hong Kong CIE sky classification and prediction by accessible weather data and trees-based methods

Solar irradiance and daylight illuminance are important for solar energy and daylighting designs. Recently, the International Commission of Illuminance (CIE) adopted a range of sky conditions to represent the possible sky distributions which are crucial to the estimation of solar irradiance and daylight illuminance on vertical building facades. The important issue would be whether the sky conditions are correctly identified by the accessible variables. Previously, a number of climatic parameters including sky luminance distributions, vertical solar irradiance and sky illuminance were proposed for the CIE sky classification. However, such data are not always available. This paper proposes an approach based on the readily accessible data that systematically recorded by the local meteorological station for many years. The performance was evaluated using measured vertical solar irradiance and illuminance. The results show that the proposed approach is reliable for sky classification.