Color Constancy Research Articles

Effective cross-sensor color constancy using a dual-mapping strategy.

Deep neural networks (DNNs) have been widely used for illuminant estimation, which commonly requires great efforts to collect sensor-specific data. In this paper, we propose a dual-mapping strategy-the DMCC method. It only requires the white points captured by the training and testing sensors under a D65 condition to reconstruct the image and illuminant data, and then maps the reconstructed image into sparse features. These features, together with the reconstructed illuminants, were used to train a lightweight multi-layer perceptron (MLP) model, which can be directly used to estimate the illuminant for the testing sensor. The proposed model was found to have performance comparable to other state-of-the-art methods, based on the three available datasets. Moreover, the smaller number of parameters, faster speed, and not requiring data collection using the testing sensor make it ready for practical deployment. This paper is an extension of Yue and Wei [Color and Imaging Conference (2023)], with more detailed results, analyses, and discussions.

Synthetic Document Images with Diverse Shadows for Deep Shadow Removal Networks.

Shadow removal for document images is an essential task for digitized document applications. Recent shadow removal models have been trained on pairs of shadow images and shadow-free images. However, obtaining a large, diverse dataset for document shadow removal takes time and effort. Thus, only small real datasets are available. Graphic renderers have been used to synthesize shadows to create relatively large datasets. However, the limited number of unique documents and the limited lighting environments adversely affect the network performance. This paper presents a large-scale, diverse dataset called the Synthetic Document with Diverse Shadows (SynDocDS) dataset. The SynDocDS comprises rendered images with diverse shadows augmented by a physics-based illumination model, which can be utilized to obtain a more robust and high-performance deep shadow removal network. In this paper, we further propose a Dual Shadow Fusion Network (DSFN). Unlike natural images, document images often have constant background colors requiring a high understanding of global color features for training a deep shadow removal network. The DSFN has a high global color comprehension and understanding of shadow regions and merges shadow attentions and features efficiently. We conduct experiments on three publicly available datasets, the OSR, Kligler's, and Jung's datasets, to validate our proposed method's effectiveness. In comparison to training on existing synthetic datasets, our model training on the SynDocDS dataset achieves an enhancement in the PSNR and SSIM, increasing them from 23.00 dB to 25.70 dB and 0.959 to 0.971 on average. In addition, the experiments demonstrated that our DSFN clearly outperformed other networks across multiple metrics, including the PSNR, the SSIM, and its impact on OCR performance.

RGB color constancy using multispectral pixel information.

Multispectral imaging is a technique that captures data across several bands of the light spectrum, and it can be useful in many computer vision fields, including color constancy. We propose a method that exploits multispectral imaging for illuminant estimation, and then applies illuminant correction in the raw RGB domain to achieve computational color constancy. Our proposed method is composed of two steps: first, a selected number of existing camera-independent algorithms for illuminant estimation, originally designed for RGB data, are applied in generalized form to work with multispectral data. We demonstrate that the sole multispectral extension of such algorithms is not sufficient to achieve color constancy, and thus we introduce a second step, in which we re-elaborate the multispectral estimations before conversion into raw RGB with the use of the camera response function. Our results on the NUS dataset show that an improvement of 60% in the color constancy performance, measured in terms of reproduction angular error, can be obtained according to our method when compared to the traditional raw RGB pipeline.

Generative models for color normalization in digital pathology and dermatology: Advancing the learning paradigm

Color medical images introduce an additional confounding factor compared to conventional grayscale medical images: color variability. This variability can lead to inconsistent evaluation by clinicians and the misinterpretation or suboptimal learning process of automatic quantitative algorithms. To mitigate the potential negative consequences of color variability, several color normalization strategies have been developed, proving to be effective in standardizing image appearance. In this paper, we present a novel paradigm for color normalization using generative adversarial networks (GANs). Our method focuses on standardizing images in the field of digital pathology (stain normalization) and dermatology (color constancy), where high color variability is consistently observed. Specifically, we formulate the color normalization task as an image-to-image translation problem, ensuring a pixel-to-pixel correspondence between the original and normalized images. Our approach outperforms existing state-of-the-art methods in both the digital pathology and dermatology fields. Extensive validation using public datasets demonstrate the effectiveness of our color normalization results on entirely external test sets. Our framework exhibits strong generalization capability on unseen data, making it suitable for inclusion in the pipeline of automatic quantitative algorithms to reduce color variability and improve segmentation and/or classification performance. Lastly, we provide the source code of our models to encourage open science.

A Convolutional Framework for Color Constancy.

We introduce a convolutional framework (CF) for computational color constancy, building upon the established low-level image feature-based framework, which utilized simple image statistics for illuminant estimation. Our framework expands upon this through an end-to-end learnable neural architecture. This adaptation enables the learning and usage of advanced filters that are not restricted to Gaussian kernels operating on individual color channels, thus generalizing the capabilities of the original framework. Additionally, our general framework supports deeper convolutional architectures, thus increasing its computational power. It can also be efficiently applied to estimate multiple spatially varying illuminants within a single scene. Our experimental results on standard datasets demonstrate that the CF outperforms the best methods in the low-level framework, improving the illuminant estimation accuracy by up to 34% for single illuminant estimation and 30% for multiple illuminants estimation. Additionally, our framework exhibits superior performance even when the number of training images is reduced. Finally, we document the inference speedup of our implementation reaching up to 30 × , making the CF especially suitable for applications where efficiency is critical. Source code and trained models available at: https://github.com/MarcoBauzz/convolutional-color-constancy.

Spatio-Spectral Deep Color Constancy With Multi-Band NIR

Spatio-Spectral Deep Color Constancy With Multi-Band NIR

Multi-scale color constancy based on salient varying local spatial statistics

The human visual system unconsciously determines the color of the objects by “discounting” the effects of the illumination, whereas machine vision systems have difficulty performing this task. Color constancy algorithms assist computer vision pipelines by removing the effects of the illuminant, which in the end enables these pipelines to perform better on high-level vision tasks based on the color features of the scene. Due to its benefits, numerous color constancy algorithms have been developed, and existing techniques have been improved. Combining different strategies and investigating new methods might help us design simple yet effective algorithms. Thereupon, we present a color constancy algorithm based on the outcomes of our previous works. Our algorithm is built upon the biological findings that the human visual system might be discounting the illuminant based on the highest luminance patches and space-average color. We find the illuminant estimate based on the idea that if the world is gray on average, the deviation of the brightest pixels from the achromatic value should be caused by the illuminant. Our approach utilizes multi-scale operations by only considering the salient pixels. It relies on varying surface orientations by adopting a block-based approach. We show that our strategy outperforms learning-free algorithms and provides competitive results compared to the learning-based methods. Moreover, we demonstrate that using parts of our strategy can significantly improve the performance of several learning-free methods. We also briefly present an approach to transform our global color constancy method into a multi-illuminant color constancy approach.

Invited Session III: Diversity in chromatic processing across the animal kingdom: Color vision in stomatopod crustaceans.

Stomatopod crustaceans, commonly known as mantis shrimp, have perhaps the most unusual color-vision systems of any animals. The uniqueness is possible because stomatopods have compound eyes. Here, each unit, or ommatidium, acts as an independent visual detector, with its own corneal lens, internal optics, and set of photoreceptors. Ommatidia tuned to different wavelengths can be individually placed in the eye to build unusual color systems. In mantis shrimps, the receptors responsible for color vision are limited to six parallel rows of ommatidia that together form an equatorial belt, called the midband. Various receptors in these ommatidia are tuned to eight narrow-band spectral channels in the visible spectrum plus up to four additional ultraviolet channels. Thus, there is a total of twelve different color receptors for color vision. How these color channels are analyzed in the complex set of optic lobes existing behind the retina is only partly understood. It appears that stomatopods use both opponent and labelled-line color channels. Oddly, these animals appear to have limited ability to discriminate between spectral lights, but they have outstanding color constancy. Color vision, and color processing, in stomatopods is probably unlike that of any other animal group.

Misidentifying illuminant changes in natural scenes due to failures in relational colour constancy.

The colours of surfaces in a scene may not appear constant with a change in the colour of the illumination. Yet even when colour constancy fails, human observers can usually discriminate changes in lighting from changes in surface reflecting properties. This operational ability has been attributed to the constancy of perceived colour relations between surfaces under illuminant changes, in turn based on approximately invariant spatial ratios of cone photoreceptor excitations. Natural deviations in these ratios may, however, lead to illuminant changes being misidentified. The aim of this work was to test whether such misidentifications occur with natural scenes and whether they are due to failures in relational colour constancy. Pairs of scene images from hyperspectral data were presented side-by-side on a computer-controlled display. On one side, the scene underwent illuminant changes and on the other side, it underwent the same changes but with images corrected for any residual deviations in spatial ratios. Observers systematically misidentified the corrected images as being due to illuminant changes. The frequency of errors increased with the size of the deviations, which were closely correlated with the estimated failures in relational colour constancy.

LSSVM-CC: ensemble algorithm of color constancy using least square support vector machine

LSSVM-CC: ensemble algorithm of color constancy using least square support vector machine

Impact of artificial intelligence-based color constancy on dermoscopical assessment of skin lesions: A comparative study.

The quality of dermoscopic images is affected by lighting conditions, operator experience, and device calibration. Color constancy algorithms reduce this variability by making images appear as if they were acquired under the same conditions, allowing artificial intelligence (AI)-based methods to achieve better results. The impact of color constancy algorithms has not yet been evaluated from a clinical dermatologist's workflow point of view. Here we propose an in-depth investigation of the impact of an AI-based color constancy algorithm, called DermoCC-GAN, on the skin lesion diagnostic routine. Three dermatologists, with different experience levels, carried out two assignments. The clinical experts evaluated key parameters such as perceived image quality, lesion diagnosis, and diagnosis confidence. When the DermoCC-GAN color constancy algorithm was applied, the dermoscopic images were perceived to be of better quality overall. An increase in classification performance was observed, reaching a maximum accuracy of 74.67% for a six-class classification task. Finally, the use of normalized images results in an increase in the level of self-confidence in the qualitative diagnostic routine. From the conducted analysis, it is evident that the impact of AI-based color constancy algorithms, such as DermoCC-GAN, is positive and brings qualitative benefits to the clinical practitioner.

ARiRTN: A Novel Learning-Based Estimation Model for Regressing Illumination.

In computational color constancy, regressing illumination is one of the most common approaches to manifesting the original color appearance of an object in a real-life scene. However, this approach struggles with the challenge of accuracy arising from label vagueness, which is caused by unknown light sources, different reflection characteristics of scene objects, and extrinsic factors such as various types of imaging sensors. This article introduces a novel learning-based estimation model, an aggregate residual-in-residual transformation network (ARiRTN) architecture, by combining the inception model with the residual network and embedding residual networks into a residual network. The proposed model has two parts: the feature-map group and the ARiRTN operator. In the ARiRTN operator, all splits perform transformations simultaneously, and the resulting outputs are concatenated into their respective cardinal groups. Moreover, the proposed architecture is designed to develop multiple homogeneous branches for high cardinality, and an increased size of a set of transformations, which extends the network in width and in length. As a result of experimenting with the four most popular datasets in the field, the proposed architecture makes a compelling case that complexity increases accuracy. In other words, the combination of the two complicated networks, residual and inception networks, helps reduce overfitting, gradient distortion, and vanishing problems, and thereby contributes to improving accuracy. Our experimental results demonstrate this model's outperformance over its most advanced counterparts in terms of accuracy, as well as the robustness of illuminant invariance and camera invariance.

A New Interpretation of the Physical Color Theory Based on the Descartes´ Rotation Energy of Visible, Ultraviolet and Infrared Photons

During the past four hundred years the Newtonian and Goethean schools collected many experimental observations on the formation of colors. The situation became more complicated after the experiments of Land who documented that the classical color theory (= colors correspond to exact wavelength of light) is valid completely in the dark surroundings only. We are at the crossroads to find the boundary between the physical properties of colors and the perception of those photons in the retina and the brain. Therefore, the more general physical color theory should interpret those color effects where the classical physical color theory fails. As a potential candidate we present the overlooked Descartes´ color theory based on the rotation energy of visible, ultraviolet, and infrared photons. The rotation energy of colors was defined as E = hν570*(λx/λ570) where index describes the wavelength of photons in nanometers. This new mathematical description of colors enables to newly interpret situations where the average rotation energy of all reflected photons determines the color impression. The colors formed behind the triangular prism could be interpreted as the lateral diffusion of the rotation energy of ultraviolet and infrared photons (= called by Old Masters as the interplay of darkness with the light) through the field of refracted visible photons and with the modification of their rotation energy. The white color can be interpreted as the constructive interference of red, green, and blue colors with the constructive angle cos (120°) = -0,5. The black color can be interpreted as the destructive interference of cyan, magenta, and yellow colors with the destructive angle cos (180°) = -1. For the case of illumination with a range of wavelengths the resulting color is determined from the average rotation energy of all reflected photons – a model for the interpretation of the color constancy.

Ranking-Based Color Constancy With Limited Training Samples.

Computational color constancy is an important component of Image Signal Processors (ISP) for white balancing in many imaging devices. Recently, deep convolutional neural networks (CNN) have been introduced for color constancy. They achieve prominent performance improvements comparing with those statistics or shallow learning-based methods. However, the need for a large number of training samples, a high computational cost and a huge model size make CNN-based methods unsuitable for deployment on low-resource ISPs for real-time applications. In order to overcome these limitations and to achieve comparable performance to CNN-based methods, an efficient method is defined for selecting the optimal simple statistics-based method (SM) for each image. To this end, we propose a novel ranking-based color constancy method (RCC) that formulates the selection of the optimal SM method as a label ranking problem. RCC designs a specific ranking loss function, and uses a low rank constraint to control the model complexity and a grouped sparse constraint for feature selection. Finally, we apply the RCC model to predict the order of the candidate SM methods for a test image, and then estimate its illumination using the predicted optimal SM method (or fusing the results estimated by the top k SM methods). Comprehensive experiment results show that the proposed RCC outperforms nearly all the shallow learning-based methods and achieves comparable performance to (sometimes even better performance than) deep CNN-based methods with only 1/2000 of the model size and training time. RCC also shows good robustness to limited training samples and good generalization crossing cameras. Furthermore, to remove the dependence on the ground truth illumination, we extend RCC to obtain a novel ranking-based method without ground truth illumination (RCC_NO) that learns the ranking model using simple partial binary preference annotations provided by untrained annotators rather than experts. RCC_NO also achieves better performance than the SM methods and most shallow learning-based methods with low costs of sample collection and illumination measurement.

Underwater image enhancement utilizing adaptive color correction and model conversion for dehazing

Underwater imaging is typically affected by absorption and scattering, which can lead to color distortion and image blurring. However, no universal color constancy algorithm for correcting underwater images exists due to the wide variation in underwater scenes. Furthermore, methods based on learning to improve underwater image quality often suffer from a lack of ground truth, which can compromise the validity and authenticity of restored images. In this study, we propose an adaptive underwater image enhancement technique that corrects image color and reduces haze using a valid dataset with ground truth. Specifically, we first use hue channel statistics from underwater images to build a dataset of color-corrected images from different underwater scenes. We use this dataset to train an Unet-like network for adaptive color correction. After color correction, underwater images often exhibit characteristics similar to hazy terrestrial images. We refer to this phenomenon as “model conversion”. The hazy terrestrial images have corresponding ground truth. Therefore, we train a Transformer-like network with hazy terrestrial image datasets to remove haze in underwater images. Our method is more robust for different underwater scenes and elegantly solves the problem of lacking ground truth. A series of experiments demonstrate that our method achieves superior performance compared to state-of-the-art methods in terms of both visual quality and quantitative metrics.

A color constancy based flower classification method in the blockchain data lake

A color constancy based flower classification method in the blockchain data lake

Single Pixel Spectral Color Constancy

Color constancy is still one of the biggest challenges in camera color processing. Convolutional neural networks have been able to improve the situation but there are still problems in many conditions, especially in scenes where a single color is dominating. In this work, we approach the problem from a slightly different setting. What if we could have some other information than the raw RGB image data. What kind of information would help to bring significant improvements while still be feasible in a mobile device. These questions sparked an idea for a novel approach for computational color constancy. Instead of raw RGB images used by the existing algorithms to estimate the scene white points, our approach is based on the scene’s average color spectra-single pixel spectral measurement. We show that as few as 10–14 spectral channels are sufficient. Notably, the sensor output has five orders of magnitude less data than in raw RGB images of a 10MPix camera. The spectral sensor captures the “spectral fingerprints” of different light sources and the illuminant white point can be accurately estimated by a standard regressor. The regressor can be trained with generated measurements using the existing RGB color constancy datasets. For this purpose, we propose a spectral data generation pipeline that can be used if the dataset camera model is known and thus its spectral characterization can be obtained. To verify the results with real data, we collected a real spectral dataset with a commercial spectrometer. On all datasets the proposed Single Pixel Spectral Color Constancy obtains the highest accuracy in the both single and cross-dataset experiments. The method is particularly effective for the difficult scenes for which the average improvements are 40–70% compared to state-of-the-arts. The approach can be extended to multi-illuminant case for which the experimental results also provide promising results.

Green Hydrogen Production and Its Land Tenure Consequences in Africa: An Interpretive Review

Globally, a green hydrogen economy rush is underway, and many companies, investors, governments, and environmentalists consider it as an energy source that could foster the global energy transition. The enormous potential for hydrogen production, for domestic use and export, places Africa in the spotlight in the green hydrogen economy discourse. This discourse remains unsettled regarding how natural resources, such as land and water, can be sustainably utilized for such a resource-intensive project, and what implications this would have. This review argues that green hydrogen production (GHP) in Africa has consequences where land resources (and their associated natural resources) are concerned. It discusses the current trends in GHP in Africa, and the possibilities for reducing any potential pressures it may put on land and other resource use on the continent. The approach of the review is interpretive, and hinges on answering three questions, concerning the what, why, and how of GHP and its land consequences in Africa. The review is based on 41 studies identified from Google Scholar, and sources identified via snowballed recommendations from experts. The GHP implications identified relate to land and water use, mining-related land stress, and environmental, ecological, and land-related socioeconomic consequences. The paper concludes that GHP may not foster the global energy transition, as is being opined by many renewable energy enthusiasts but, rather, could help foster this transition as part of a greener energy mix. It notes that African countries that have the potential for GHP require the institutionalization of, or a change in, their existing approaches to land-related energy governance systems, in order to achieve success.

Poster Session: Chromatic Adaptation to Heterochromatic Illumination.

Heterochromatic viewing enviroments-where scenes are illuminated by two or more colors of light-are a common tool for experiments studying chromatic adaptation and color (in)constancy. This study presents a new method for separating sensory (retina-based) chromatic adaptation from cognitive mechanisms in a heterochromatic viewing environment. Using a large, bipartite light booth lit by two separate 7-channel LED systems, observers were adapted to uniform and mixed illuminations of simulated illuminant A, illuminant D65, and 12000 K daylight. A psychophysical staircase method determined observers' white points and their state of sensory adaptation along the daylight and Planckian loci under each heterochromatic or single-color lighting condition. Under heterochromatic conditions, observers' state of chromatic adaptation fell in between the points of chromatic adaptation when adapted to each individual light source, although there was significant variation between observers. These results suggest that observers' white points should be individually determined for studies of color perception in heterochromatic environments using methods similar to those used in this study. Further research will then allow us to model how cognitive discounting of the illumination color sits on top of the sensory adaptation measured here.

Past fire dynamics inferred from polycyclic aromatic hydrocarbons and monosaccharide anhydrides in a stalagmite from the archaeological site of Mayapan, Mexico

Abstract. Speleothems (cave stalagmites) contain inorganic and organic substances that can be used to infer past changes in local and regional paleoenvironmental conditions. Specific biomarkers can be employed to elucidate the history of past fires, caused by interactions among climate, regional hydrology, vegetation, humans, and fire activity. We conducted a simple solid–liquid extraction on pulverised carbonate samples to prepare them for analysis of 16 polycyclic aromatic hydrocarbons (PAHs) and three monosaccharide anhydrides (MAs). The preparation method requires only small samples (0.5–1.0 g); PAHs and MAs were measured by GC–MS and LC–HILIC–MS, respectively. Detection limits range from 0.05–2.1 ng for PAHs and 0.01–0.1 ng for MAs. We applied the method to 10 samples from a ∼ 400-year-old stalagmite from Cenote Ch'en Mul, at Mayapan (Mexico), the largest Postclassic Maya capital of the Yucatán Peninsula. We found a strong correlation (r = 0.75, p < 0.05) between the major MA (levoglucosan) and non-alkylated PAHs (Σ15). We investigated multiple diagnostic PAH and MA ratios and found that although not all were applicable as paleo-fire proxies, ratios that combine PAHs with MAs are promising tools for identifying different fire regimes and inferring the type of fuel burned. In the 1950s and 1960s, levoglucosan and Σ15 concentrations roughly doubled compared to other times in the last 400 years, suggesting greater fire activity at Mayapan during these two decades. The higher concentrations of fire markers may have been a consequence of land clearance at the site and exploration of the cave by Carnegie Institution archaeologists.