Biologically inspired image invariance guided illuminant estimation using shallow and deep models

Abstract

Estimating the illuminant from a color-biased image is an ill-posed problem without prior information or invariance about the surfaces of a scene. Based on the classical image formation model, we have developed a heuristic approach to obtain the illuminant color by computing the ratio of the average of all pixels over the color-biased scene to that over the roughly recovered scene obtained by local normalization in each channel. The computed ratio represents an estimated invariance across color channels (IACC), ranging between the average reflectance of surfaces and the maximum reflectance of surfaces of a scene, modulated by the illuminant color.This work builds a mathematical foundation for IACC and explains why it is suitable for illuminant estimation. The core discovery is that the magnitude relationship of the average reflectances of surfaces between any two color channels is opposite to that of the maximum reflectances of surfaces for most natural scenes. As a result, we have designed two approaches for explicitly learning IACC of an image, resulting in very accurate illuminant estimation. The first approach involves a novel shallow model based on diagonal or non-diagonal matrices, together with the learned model parameters, to improve IACC performance. The second approach applies IACC as a constraint to optimize a novel deep learning approach, which has achieved state-of-the-art performance on two benchmarks. An interesting finding is that the output of the learned network, constrained only by IACC loss, provides a coarse estimation of intrinsic images such as albedo from the input color-biased image.