Illuminant Estimation Algorithms Research Articles

Detecting Human Falls in Poor Lighting: Object Detection and Tracking Approach for Indoor Safety

Falls are one the leading causes of accidental death for all people, but the elderly are at particularly high risk. Falls are severe issue in the care of those elderly people who live alone and have limited access to health aides and skilled nursing care. Conventional vision-based systems for fall detection are prone to failure in conditions with low illumination. Therefore, an automated system that detects falls in low-light conditions has become an urgent need for protecting vulnerable people. This paper proposes a novel vision-based fall detection system that uses object tracking and image enhancement techniques. The proposed approach is divided into two parts. First, the captured frames are optimized using a dual illumination estimation algorithm. Next, a deep-learning-based tracking framework that includes detection by YOLOv7 and tracking by the Deep SORT algorithm is proposed to perform fall detection. On the Le2i fall and UR fall detection (URFD) datasets, we evaluate the proposed method and demonstrate the effectiveness of fall detection in dark night environments with obstacles.

Illumination estimation via random vector functional link based on improved arithmetic optimization algorithm

AbstractIn order to solve the problem that the existing illumination estimation methods have low accuracy in image restoration, a random vector functional link illumination estimation algorithm based on whale optimization arithmetic algorithm is proposed in this article. First, in order to improve the optimization accuracy of arithmetic optimization algorithm, the whale optimization algorithm is applied to optimize the initial population of the arithmetic optimization algorithm. Then, we utilize the improved arithmetic optimization algorithm to improve the input weight and hidden layer offset of random vector functional link, and exclude the uncertainty caused by the random generation of random vector functional link's input weight and hidden layer bias. Finally, the proposed whale arithmetic optimization algorithm‐random vector functional link model is applied to SFU lab dataset and Gehler‐Shi dataset for illumination estimation. Compared with the other earlier illumination estimation algorithms, the proposed illumination estimation model in this article has better effect and higher stability. In addition, due to the diagonal mapping method for color correction and image restoration, the proposed illumination estimation algorithm can also get better results in image restoration.

Illumination correction with optimized kernel extreme learning machine based on improved marine predators algorithm

AbstractAiming at the problem of poor image illumination correction accuracy, a kernel extreme learning machine optimized based on the differential evolution‐improved marine predators algorithm is proposed to estimate the scene illumination information and restore the image. First, in order to solve the problem of randomization of the initial population of the marine predators algorithm, differential evolutions is used to provide a set of suitable initial populations to the algorithm. Then, an improved algorithm is used to optimize the weight and bias of the kernel extreme learning machine to solve the problem of the randomness of the weight and bias, so that the learning machine converges to the global optimal value and avoids the unstable predictions. Finally, after obtaining the predicted illumination information, the image model is corrected to the image effect under standard illumination through diagonal transformation. It can be seen from the experimental results that the best predictive value of the differential evolution marine predator algorithm learning machine is 0.0135915, and the quasi‐bias is only 0.001687. Compared with traditional illumination estimation algorithms such as random vector functional link and extreme learning machine, the differential evolution marine predator algorithm learning machine algorithm proposed in this article has better stability, higher accuracy of calculated predicted values, and better image restoration effect.

On the evaluation of temporal and spatial stability of color constancy algorithms.

Computational color constancy algorithms are commonly evaluated only through angular error analysis on annotated datasets of static images. The widespread use of videos in consumer devices motivated us to define a richer methodology for color constancy evaluation. To this extent, temporal and spatial stability are defined here to determine the degree of sensitivity of color constancy algorithms to variations in the scene that do not depend on the illuminant source, such as moving subjects or a moving camera. Our evaluation methodology is applied to compare several color constancy algorithms on stable sequences belonging to the Gray Ball and Burst Color Constancy video datasets. The stable sequences, identified using a general-purpose procedure, are made available for public download to encourage future research. Our investigation proves the importance of evaluating color constancy algorithms according to multiple metrics, instead of angular error alone. For example, the popular fully convolutional color constancy with confidence-weighted pooling algorithm is consistently the best performing solution for error evaluation, but it is often surpassed in terms of stability by the traditional gray edge algorithm, and by the more recent sensor-independent illumination estimation algorithm.

An Improved White Patch Method for Image Illumination Estimation

As one of the underlying pixel-based illumination estimation algorithms, the White Patch algorithm is an algorithm for calculating the global illumination RGB value of an image based on the specific assumption that the maximum reflected light on the scene is chromatic. The algorithm is harsh on the assumptions of scene illumination, and many images are difficult to satisfy this assumption constraint. In this paper, we propose an improved White Patch image illumination estimation method. Firstly, the image patch is extracted by using sliding window method, we then use the white patch algorithm to estimate the illumination color value of each patch, and finally the kernel density estimation is adopted to obtain the overall illumination color value of the image. The experimental results show that the improved White Patch images illumination estimation method proposed to this paper performs better on the illumination estimation of natural illumination scene images.

An Illumination Estimation Algorithm based on Outdoor Scene Classification

The illumination estimation algorithm belongs to the field of color constancy, aiming to restoring the color of image through estimating the RGB of scene illumination. In different scenarios, the performance of a general algorithm varies greatly. If the scene can be predicted, it can be inferred that the scenarios related optimal algorithms is better than a general algorithm for estimating illumination. In this paper, a novel algorithm based on outdoor scene classification was proposed: firstly, the support vector machine (svm) classifiers was used to identify scene types , and then the scenarios related optimal algorithms was selected, finally used the RGB values of scene illumination were calculated.

An alternative approach to preserve naturalness with non-uniform illumination estimation for images enhancement using normalized $$L_2$$-Norm based on Retinex

In recent works, the illumination estimation based on Retinex is utilized to enhance the image details. However, to enhance the image using illumination estimation without affecting the naturalness with non-uniform illumination is very difficult. The usual algorithms used for illumination estimation failed to include these constraints such as sharp edges on illumination boundaries, spatial smoothness, and the limited range of illumination while preserving the naturalness. In this paper, an illumination estimation algorithm using normalized $$L_2$$ -Norm with the joint edge-preservation filter is proposed. The proposed algorithm efficiently estimates illumination and fulfils all the above-mentioned constraints. The normalized $$L_2$$ -Norm is used to approximate the illumination to overcome the block effect in patch-wise illumination. Then, the overall structure is refined by using the joint edge-preservation filter. Experimental results show the quality of the smoothness of illumination beyond the edges and ensure the range of the estimated illumination in comparison with the other state-of-the-art algorithms.

Providing a Single Ground-Truth for Illuminant Estimation for the ColorChecker Dataset.

The ColorChecker dataset is one of the most widely used image sets for evaluating and ranking illuminant estimation algorithms. However, this single set of images has at least 3 different sets of ground-truth (i.e., correct answers) associated with it. In the literature it is often asserted that one algorithm is better than another when the algorithms in question have been tuned and tested with the different ground-truths. In this short correspondence we present some of the background as to why the 3 existing ground-truths are different and go on to make a new single and recommended set of correct answers. Experiments reinforce the importance of this work in that we show that the total ordering of a set of algorithms may be reversed depending on whether we use the new or legacy ground-truth data.

As-projective-as-possible bias correction for illumination estimation algorithms

Illumination estimation is the key routine in a camera's onboard auto-white-balance (AWB) function. Illumination estimation algorithms estimate the color of the scene's illumination from an image in the form of an R, G, B vector in the sensor's raw-RGB color space. While learning-based methods have demonstrated impressive performance for illumination estimation, cameras still rely on simple statistical-based algorithms that are less accurate but capable of executing quickly on the camera's hardware. An effective strategy to improve the accuracy of these fast statistical-based algorithms is to apply a post-estimate bias-correction function to transform the estimated R, G, B vector such that it lies closer to the correct solution. Recent work by Finlayson [Interface Focus8, 20180008 (2018)2042-889810.1098/rsfs.2018.0008] showed that a bias-correction function can be formulated as a projective transform because the magnitude of the R, G, B illumination vector does not matter to the AWB procedure. This paper builds on this finding and shows that further improvements can be obtained by using an as-projective-as-possible (APAP) projective transform that locally adapts the projective transform to the input R, G, B vector. We demonstrate the effectiveness of the proposed APAP bias correction on several well-known statistical illumination estimation methods. We also describe a fast lookup method that allows the APAP transform to be performed with only a few lookup operations.

Rehabilitating the ColorChecker Dataset for Illuminant Estimation

In a previous work, it was shown that there is a curious problem with the benchmark ColorChecker dataset for illuminant estimation. To wit, this dataset has at least 3 different sets of ground-truths. Typically, for a single algorithm a single ground-truth is used. But then different algorithms, whose performance is measured with respect to different ground-truths, are compared against each other and then ranked. This makes no sense. We show in this paper that there are also errors in how each ground-truth set was calculated. As a result, all performance rankings based on the ColorChecker dataset - and there are scores of these - are inaccurate. In this paper, we re-generate a new 'recommended' set of ground-truth based on the calculation methodology described by Shi and Funt. We then review the performance evaluation of a range of illuminant estimation algorithms. Compared with the legacy ground-truths, we find that the difference in how algorithms perform can be large, with many local rankings of algorithms being reversed. Finally, we draw the readers attention to our new 'open' data repository which, we hope, will allow the ColorChecker set to be rehabilitated and once again to become a useful benchmark for illuminant estimation algorithms.

Colour and illumination in computer vision.

In computer vision, illumination is considered to be a problem that needs to be 'solved'. The colour cast due to illumination is removed to support colour-based image recognition and stable tracking (in and out of shadows), among other tasks. In this paper, I review historical and current algorithms for illumination estimation. In the classical approach, the illuminant colour is estimated by an ever more sophisticated analysis of simple image summary statistics often followed by a bias correction step. Bias correction is a function applied to the estimates made by a given illumination estimation algorithm to correct consistent errors in the estimations. Most recently, the full power, and much higher complexity, of deep learning has been deployed (where, effectively, the definition of the image statistics of interest and the type of analysis carried out are found as part of an overall optimization). In this paper, I challenge the orthodoxy of deep learning, i.e. that it is the best approach for illuminant estimation. We instead focus on the final bias correction stage found in many simple illumination estimation algorithms. There are two key insights in our method. First, we argue that the bias must be corrected in an exposure invariant way. Second, we show that this bias correction amounts to 'solving for a homography'. Homography-based illuminant estimation is shown to deliver leading illumination estimation performance (at a very small fraction of the complexity of deep learning methods).

Naturalness Preserved Nonuniform Illumination Estimation for Image Enhancement Based on Retinex

Illumination estimation is important for image enhancement based on Retinex. However since illumination estimation is an ill-posed problem it is difficult to achieve accurate illumination estimation for nonuniform illumination images. The conventional illumination estimation algorithms fail to comprehensively take all the constraints into the consideration such as spatial smoothness sharp edges on illumination boundaries and limited range of illumination. Thus these algorithms cannot effectively and efficiently estimate illumination while preserving naturalness. In this paper we present a naturalness preserved illumination estimation algorithm based on the proposed joint edge-preserving filter which exploits all the abovementioned constraints. Moreover a fast estimation is implemented based on the box filter. Experimental results demonstrate that the proposed algorithm can achieve the adaptive smoothness of illumination beyond edges and ensure the range of the estimated illumination. When compared with other state-of-the-art algorithms it can achieve better quality from both subjective and objective aspects.

A Psychophysical Analysis of Illuminant Estimation Algorithms

Illuminant estimation algorithms are often evaluated by calculating recovery angular error which is the angle between the RGB of the ground truth and the estimated illuminants. However, the same scene viewed under two different lights with respect to which the same algorithm delivers illuminant estimates and then identical reproductions - and so, the practical estimation error is the same - can, in fact and counterintuitively, result in quite different recovery errors. Reproduction angular error has been recently introduced as an improvement to recovery angular error. The new metric calculates the angle between the RGB values of a white surface corrected by the ground truth illuminant and corrected by the estimated illuminant. Experiments show that illuminant estimation algorithms could be ranked differently depending on whether they are evaluated by recovery or reproduction angular error. In this paper a psychophysical experiment is designed which demonstrates that observers choices on 'what makes a good reproduction' correlates with reproduction error and not recovery error.

A Curious Problem with Using the Colour Checker Dataset for Illuminant Estimation

In illuminant estimation, we attempt to estimate the RGB of the light. We then use this estimate on an image to correct for the light's colour bias. Illuminant estimation is an essential component of all camera reproduction pipelines. How well an illuminant estimation algorithm works is determined by how well it predicts the ground truth illuminant colour. Typically, the ground truth is the RGB of a white surface placed in a scene. Over a large set of images an estimation error is calculated and different algorithms are then ranked according to their average estimation performance. Perhaps the most widely used publically available dataset used in illuminant estimation is Gehler's Colour Checker set that was reprocessed by Shi and Funt. This image set comprises 568 images of typical everyday scenes. Curiously, we have found three different ground truths for the Shi-Funt Colour Checker image set. In this paper, we investigate whether adopting one ground truth over another results in different rankings of illuminant estimation algorithms. We find that, depending on the ground truth used, the ranking of different algorithms can change, and sometimes dramatically. Indeed, it is entirely possible that much of the recent 'advances' made in illuminant estimation were achieved because authors have switched to using a ground truth where better estimation performance is possible.

The Reproduction Angular Error for Evaluating the Performance of Illuminant Estimation Algorithms.

The angle between the RGBs of the measured illuminant and estimated illuminant colors-the recovery angular error-has been used to evaluate the performance of the illuminant estimation algorithms. However we noticed that this metric is not in line with how the illuminant estimates are used. Normally, the illuminant estimates are `divided out' from the image to, hopefully, provide image colors that are not confounded by the color of the light. However, even though the same reproduction results the same scene might have a large range of recovery errors. In this work the scale of the problem with the recovery error is quantified. Next we propose a new metric for evaluating illuminant estimation algorithms, called the reproduction angular error, which is defined as the angle between the RGB of a white surface when the actual and estimated illuminations are `divided out'. Our new metric ties algorithm performance to how the illuminant estimates are used. For a given algorithm, adopting the new reproduction angular error leads to different optimal parameters. Further the ranked list of best to worst algorithms changes when the reproduction angular is used. The importance of using an appropriate performance metric is established.

Color correction pipeline optimization for digital cameras

The processing pipeline of a digital camera converts the RAW image acquired by the sensor to a representation of the original scene that should be as faithful as possible. There are mainly two modules responsible for the color-rendering accuracy of a digital cam- era: the former is the illuminant estimation and correction module, and the latter is the color matrix transformation aimed to adapt the color response of the sensor to a standard color space. These two modules together form what may be called the color correction pipeline. We design and test new color correction pipelines that exploit different illuminant estimation and correction algorithms that are tuned and automatically selected on the basis of the image content. Since the illuminant estimation is an ill-posed problem, illuminant correction is not error-free. An adaptive color matrix transformation module is optimized, taking into account the behavior of the first module in order to alleviate the amplification of color errors. The proposed pipe- lines are tested on a publicly available dataset of RAW images. Experimental results show that exploiting the cross-talks between the modules of the pipeline can lead to a higher color-rendition accu- racy. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publica- tion, including its DOI. (DOI: 10.1117/1.JEI.22.2.023014)

Color constancy by chromaticity neutralization

In this paper, a robust illuminant estimation algorithm for color constancy is proposed. Considering the drawback of the well-known max-RGB algorithm, which regards only pixels with the maximum image intensities, we explore the representative pixels from an image for illuminant estimation: The representative pixels are determined via the intensity bounds corresponding to a certain percentage value in the normalized accumulative histograms. To achieve the suitable percentage, an iterative algorithm is presented by simultaneously neutralizing the chromaticity distribution and preventing overcorrection. The experimental results on the benchmark databases provided by Simon Fraser University and Microsoft Research Cambridge, as well as several web images, demonstrate the effectiveness of our approach.

Automatic color constancy algorithm selection and combination

In this work, we investigate how illuminant estimation techniques can be improved taking into account intrinsic, low level properties of the images. We show how these properties can be used to drive, given a set of illuminant estimation algorithms, the selection of the best algorithm for a given image. The algorithm selection is made by a decision forest composed of several trees on the basis of the values of a set of heterogeneous features. The features represent the image content in terms of low-level visual properties. The trees are trained to select the algorithm that minimizes the expected error in illuminant estimation. We also designed a combination strategy that estimates the illuminant as a weighted sum of the different algorithms’ estimations. Experimental results on the widely used Ciurea and Funt dataset demonstrate the effectiveness of our approach.

Improving Color Constancy Using Indoor–Outdoor Image Classification

In this work, we investigate how illuminant estimation techniques can be improved, taking into account automatically extracted information about the content of the images. We considered indoor/outdoor classification because the images of these classes present different content and are usually taken under different illumination conditions. We have designed different strategies for the selection and the tuning of the most appropriate algorithm (or combination of algorithms) for each class. We also considered the adoption of an uncertainty class which corresponds to the images where the indoor/outdoor classifier is not confident enough. The illuminant estimation algorithms considered here are derived from the framework recently proposed by Van de Weijer and Gevers. We present a procedure to automatically tune the algorithms' parameters. We have tested the proposed strategies on a suitable subset of the widely used Funt and Ciurea dataset. Experimental results clearly demonstrate that classification based strategies outperform general purpose algorithms.

Reconstructing 3D Shape, Albedo and Illumination from a Single Face Image

AbstractThe morphable model has been employed to efficiently describe 3D face shape and the associated albedo with a reduced set of basis vectors. The spherical harmonics (SH) model provides a compact basis to well approximate the image appearance of a Lambertian object under different illumination conditions. Recently, the SH and morphable models have been integrated for 3D face shape reconstruction. However, the reconstructed 3D shape is either inconsistent with the SH bases or obtained just from landmarks only. In this work, we propose a geometrically consistent algorithm to reconstruct the 3D face shape and the associated albedo from a single face image iteratively by combining the morphable model and the SH model. The reconstructed 3D face geometry can uniquely determine the SH bases, therefore the optimal 3D face model can be obtained by minimizing the error between the input face image and a linear combination of the associated SH bases. In this way, we are able to preserve the consistency between the 3D geometry and the SH model, thus refining the 3D shape reconstruction recursively. Furthermore, we present a novel approach to recover the illumination condition from the estimated weighting vector for the SH bases in a constrained optimization formulation independent of the 3D geometry. Experimental results show the effectiveness and accuracy of the proposed face reconstruction and illumination estimation algorithm under different face poses and multiple‐light‐source illumination conditions.