Retinex Model Research Articles

Toward a general model for reflection recovery and single image enhancement

Images often suffer from low visual quality due to poor imaging conditions such as low light or hazy weather. The haze imaging model is widely used in contrast enhancement in daylight condition with haze, while the retinex model is universal for low-light conditions. Although their forms and applications are different, they can be unified into a more general form through the proposed observation. Based on this model, the authors can estimate the reflection of the scene more accurately in more complex imaging conditions. In this study, the authors propose a simple but effective method for estimating the reflection and enhancing the image contrast based on a general imaging model. To preserve the image details and control contrast, the authors introduce dark boundary and bright boundary to handle the high-light and low-light conditions, and a guided structure-preserving optimization algorithm is proposed to estimate them. After obtaining the dark and bright boundaries, the reflection is calculated and the image is enhanced accordingly. Different from previous approaches, which were designed for specific applications, the proposed method can be used for more diverse imaging conditions. Experiments show that the proposed method can be applied to many poor imaging conditions and maintain good performance.

Low-light enhancement based on an improved simplified Retinex model via fast illumination map refinement

Low-light enhancement is an important post-image-processing technique, as it helps to reveal hidden details from dark image regions. In this paper, we propose a fast low-light enhancement model, which is robust to various lighting conditions and imaging noise, and is computationally efficient. By using a fusion-based simplified Retinex model, our model caters to different lighting conditions. In the model, we propose an edge-preserving filter to efficiently refine the estimated illumination map. We also extend our model by equipping it with a very simple denoising step, which effectively prevents the over-boosting of imaging noise in the dark regions. We conduct the experiments on public available images as well as the ones collected by ourselves. Visual and quantitative results validate the effectiveness of our model.

Color-compensated multi-scale exposure fusion based on physical features

The dynamic range captured by ordinary digital cameras is much smaller than that in natural scenes, so a single image cannot fully capture the real scene details when shooting high dynamic scenes. Traditional multi-exposure image fusion algorithms often cause inconsistencies in local brightness and corresponding scenes, which easily leads to problems such as color distortion and loss of details. Based on this, a novel multi-scale exposure fusion approach based on physical features is presented in this paper. Specifically, we first employ a novel Retinex model to get the illumination maps of original images and adopt the obtained illuminations map to construct the exposure maps. Then, a feature descriptor named patterns of oriented edge magnitudes (POEM) is introduced to extract local contrast from source images. Moreover, the extracted feature is combined to construct a weight map, and the weight term is integrated with the Laplacian pyramid algorithm to obtain an initial fusion image. Finally, simple detail enhancement and color compensation operations are performed to get the final fusion result. The numerous experiments have revealed that the proposed approach yields comparative and even better results in comparison with some state-of-the-art techniques both in subjective and objective evaluation.

Image retinex based on the nonconvex TV-type regularization

<p style='text-indent:20px;'>Retinex theory is introduced to show how the human visual system perceives the color and the illumination effect such as Retinex illusions, medical image intensity inhomogeneity and color shadow effect etc.. Many researchers have studied this ill-posed problem based on the framework of the variation energy functional for decades. However, to the best of our knowledge, the existing models via the sparsity of the image based on the nonconvex <inline-formula><tex-math id="M1">\begin{document}$ \ell^p $\end{document}</tex-math></inline-formula>-quasinorm were limited. To deal with this problem, this paper considers a TV<inline-formula><tex-math id="M2">\begin{document}$ _p $\end{document}</tex-math></inline-formula>-HOTV<inline-formula><tex-math id="M3">\begin{document}$ _q $\end{document}</tex-math></inline-formula>-based retinex model with <inline-formula><tex-math id="M4">\begin{document}$ p, q\in(0, 1) $\end{document}</tex-math></inline-formula>. Specially, the TV<inline-formula><tex-math id="M5">\begin{document}$ _p $\end{document}</tex-math></inline-formula> term based on the total variation(TV) regularization can describe the reflectance efficiently, which has the piecewise constant structure. The HOTV<inline-formula><tex-math id="M6">\begin{document}$ _q $\end{document}</tex-math></inline-formula> term based on the high order total variation(HOTV) regularization can penalize the smooth structure called the illumination. Since the proposed model is non-convex, non-smooth and non-Lipschitz, we employ the iteratively reweighed <inline-formula><tex-math id="M7">\begin{document}$ \ell_1 $\end{document}</tex-math></inline-formula> (IRL1) algorithm to solve it. We also discuss some properties of our proposed model and algorithm. Experimental experiments on the simulated and real images illustrate the effectiveness and the robustness of our proposed model both visually and quantitatively by compared with some related state-of-the-art variational models.</p>

Retinex Based Image Enhancement via General Dictionary Convolutional Sparse Coding

Retinex theory represents the human visual system by showing the relative reflectance of an object under various illumination conditions. A feature of this human visual system is color constancy, and the Retinex theory is designed in consideration of this feature. The Retinex algorithms have been popularly used to effectively decompose the illumination and reflectance of an object. The main aim of this paper is to study image enhancement using convolution sparse coding and sparse representations of the reflectance component in the Retinex model over a learned dictionary. To realize this, we use the convolutional sparse coding model to represent the reflectance component in detail. In addition, we propose that the reflectance component can be reconstructed using a trained general dictionary by using convolutional sparse coding from a large dataset. We use singular value decomposition in limited memory to construct a best reflectance dictionary. This allows the reflectance component to provide improved visual quality over conventional methods, as shown in the experimental results. Consequently, we can reduce the difference in perception between humans and machines through the proposed Retinex-based image enhancement.

Automated Tortuosity Analysis of Nerve Fibers in Corneal Confocal Microscopy.

Precise characterization and analysis of corneal nerve fiber tortuosity are of great importance in facilitating examination and diagnosis of many eye-related diseases. In this paper we propose a fully automated method for image-level tortuosity estimation, comprising image enhancement, exponential curvature estimation, and tortuosity level classification. The image enhancement component is based on an extended Retinex model, which not only corrects imbalanced illumination and improves image contrast in an image, but also models noise explicitly to aid removal of imaging noise. Afterwards, we take advantage of exponential curvature estimation in the 3D space of positions and orientations to directly measure curvature based on the enhanced images, rather than relying on the explicit segmentation and skeletonization steps in a conventional pipeline usually with accumulated pre-processing errors. The proposed method has been applied over two corneal nerve microscopy datasets for the estimation of a tortuosity level for each image. The experimental results show that it performs better than several selected state-of-the-art methods. Furthermore, we have performed manual gradings at tortuosity level of four hundred and three corneal nerve microscopic images, and this dataset has been released for public access to facilitate other researchers in the community in carrying out further research on the same and related topics.

A Perceptually Inspired Variational Model for Enhancing and Restoring Remote Sensing Images

Perceptually inspired algorithms have captured the recent attention of scientists and engineers due to their inherent capability to enhance the contrast of images, especially from the remote sensing domain. In this letter, we propose a perceptually inspired retinex model relying on the variational framework for enhancing and denoising satellite images captured by various imaging devices. A variational framework incorporates priors and data fidelity aspects in the designed functional, whose optimized solution yields the desired output. The model respects the distribution of the noise while enhancing the data. The overall performance is demonstrated using the visual and quantitative measures.

Low Light Image Enhancement Network With Attention Mechanism and Retinex Model

Due to the uncertainty of the environment, the captured image may not only degrade but also have uneven brightness distribution. The quality of these images can not meet the input requirements of existing computer vision tasks, so they may lead to performance degradation in the completion of computer vision tasks. Past methods are often only applicable to low-light images with uniform brightness distribution, and the performance of these methods is not ideal for low-light images with uneven brightness distribution. In order to solve the problems caused by these low-light images, a new low light image enhancement model based on attention mechanism and Retinex model is proposed. The proposed method first estimates the illumination mask of the input image, which guides the network to predict the illumination distribution. Then we use a module with attention mechanism to predict the illumination map, and the initial enhanced image is estimated based on Retinex model. We modify the color distortion and suppress noise with convolution layers to obtain final enhanced results. In experiments, the performance of our methods is demonstrated by compared with the state-of-the-art existing methods. Our approach has more positive performance in some scenarios, especially uneven lighting.

A parallel operator splitting algorithm for solving constrained total-variation retinex

<p style='text-indent:20px;'>An ideal image is desirable to faithfully represent the real-world scene. However, the observed images from imaging system are typically involved in the illumination of light. As the human visual system (HVS) is capable of perceiving identical color with spatially varying illumination, retinex theory is established to probe the rationale of HVS on perceiving color. In the realm of image processing, the retinex models are devoted to diminishing illumination effects from observed images. In this paper, following the recent work by Ng and Wang (SIAM J. Imaging Sci. 4:345-356, 2011), we develop a parallel operator splitting algorithm tailored for the constrained total-variation retinex model, in which all the resulting subproblems admit closed form solutions or can be tractably solved by some subroutines without any internally nested iterations. The global convergence of the novel algorithm is analysed on the perspective of variational inequality in optimization community. Preliminary numerical simulations demonstrate the promising performance of the proposed algorithm.

Neurocomputational Lightness Model Explains the Appearance of Real Surfaces Viewed Under Gelb Illumination.

One of the primary functions of visual perception is to represent, estimate, and evaluate the properties of material surfaces in the visual environment. One such property is surface color, which can convey important information about ecologically relevant object characteristics such as the ripeness of fruit and the emotional reactions of humans in social interactions. This paper further develops and applies a neural model (Rudd, 2013, 2017) of how the human visual system represents the light/dark dimension of color-known as lightness-and computes the colors of achromatic material surfaces in real-world spatial contexts. Quantitative lightness judgments conducted with real surfaces viewed under Gelb (i.e., spotlight) illumination are analyzed and simulated using the model. According to the model, luminance ratios form the inputs to ON- and OFF-cells, which encode local luminance increments and decrements, respectively. The response properties of these cells are here characterized by physiologically motivated equations in which different parameters are assumed for the two cell types. Under non-saturating conditions, ON-cells respond in proportion to a compressive power law of the local incremental luminance in the image that causes them to respond, while OFF-cells respond linearly to local decremental luminance. ON- and OFF-cell responses to edges are log-transformed at a later stage of neural processing and then integrated across space to compute lightness via an edge integration process that can be viewed as a neurally elaborated version of Land's retinex model (Land & McCann, 1971). It follows from the model assumptions that the perceptual weights-interpreted as neural gain factors-that the model observer applies to steps in log luminance at edges in the edge integration process are determined by the product of a polarity-dependent factor 1-by which incremental steps in log luminance (i.e., edges) are weighted by the value <1.0 and decremental steps are weighted by 1.0-and a distance-dependent factor 2, whose edge weightings are estimated to fit perceptual data. The model accounts quantitatively (to within experimental error) for the following: lightness constancy failures observed when the illumination level on a simultaneous contrast display is changed (Zavagno, Daneyko, & Liu, 2018); the degree of dynamic range compression in the staircase-Gelb paradigm (Cataliotti & Gilchrist, 1995; Zavagno, Annan, & Caputo, 2004); partial releases from compression that occur when the staircase-Gelb papers are reordered (Zavagno, Annan, & Caputo, 2004); and the larger compression release that occurs when the display is surrounded by a white border (Gilchrist & Cataliotti, 1994).

STAR: A Structure and Texture Aware Retinex Model.

Retinex theory is developed mainly to decompose an image into the illumination and reflectance components by analyzing local image derivatives. In this theory, larger derivatives are attributed to the changes in reflectance, while smaller derivatives are emerged in the smooth illumination. In this paper, we utilize exponentiated local derivatives (with an exponent γ) of an observed image to generate its structure map and texture map. The structure map is produced by been amplified with γ > 1, while the texture map is generated by been shrank with γ < 1. To this end, we design exponential filters for the local derivatives, and present their capability on extracting accurate structure and texture maps, influenced by the choices of exponents γ. The extracted structure and texture maps are employed to regularize the illumination and reflectance components in Retinex decomposition. A novel Structure and Texture Aware Retinex (STAR) model is further proposed for illumination and reflectance decomposition of a single image. We solve the STAR model by an alternating optimization algorithm. Each sub-problem is transformed into a vectorized least squares regression, with closed-form solutions. Comprehensive experiments on commonly tested datasets demonstrate that, the proposed STAR model produce better quantitative and qualitative performance than previous competing methods, on illumination and reflectance decomposition, low-light image enhancement, and color correction. The code is publicly available at https://github.com/csjunxu/STAR.

Color Correction Based on CFA and Enhancement Based on Retinex With Dense Pixels for Underwater Images

Color correction and enhancement for underwater images is challenging due to attenuation and scattering. The underwater images often have low visibility and suffer from color bias. This paper presents a novel color correction method based on color filter array (CFA) and an enhancement method based on Retinex with dense pixels and adaptive linear histogram transformation for degraded color-biased underwater images. For any digital image in the RGB space, which is captured by digital camera with CFA, their RGB values are dependent and coupled because of the interpolation process. So we try to compensate red channel attenuation of underwater degraded images from the green channel and blue channel. Retinex model has been widely used to efficiently handle low brightness and blurred images. The McCann Retinex (MR) method selects a spiral path for pixel comparison to estimate illumination. However, the simple path selection doesn't include global light dark relationship of the whole image. So we design a scheme to gain much well-distributed and denser pixels to obtain more precise intensity of illumination. Besides, we design a piecewise linear function for histogram transform, which is adaptive to the whole RGB value. Experiments on a large number of underwater degraded images show that, the processed images by our method have clearer details and uniform visual effect for all channels in RGB color space and our method can also obtain good performance metrics.

LR3M: Robust Low-Light Enhancement via Low-Rank Regularized Retinex Model.

Noise causes unpleasant visual effects in low-light image/video enhancement. In this paper, we aim to make the enhancement model and method aware of noise in the whole process. To deal with heavy noise which is not handled in previous methods, we introduce a robust low-light enhancement approach, aiming at well enhancing low-light images/videos and suppressing intensive noise jointly. Our method is based on the proposed Low-Rank Regularized Retinex Model (LR3M), which is the first to inject low-rank prior into a Retinex decomposition process to suppress noise in the reflectance map. Our method estimates a piece-wise smoothed illumination and a noise-suppressed reflectance sequentially, avoiding remaining noise in the illumination and reflectance maps which are usually presented in alternative decomposition methods. After getting the estimated illumination and reflectance, we adjust the illumination layer and generate our enhancement result. Furthermore, we apply our LR3M to video low-light enhancement. We consider inter-frame coherence of illumination maps and find similar patches through reflectance maps of successive frames to form the low-rank prior to make use of temporal correspondence. Our method performs well for a wide variety of images and videos, and achieves better quality both in enhancing and denoising, compared with the state-of-the-art methods.

A Local Flatness Based Variational Approach to Retinex

A topic of continued interest in Retinex over the years has been finding ways to implement it with computational models of improved accuracy and efficiency. We have devised a new approach to digitally implementing the Retinex using a local deviation based variational model. The new model leads to improvements in the computed image quality with respect to illumination correction and image enhancement. Several contributions are made: 1) a new prior constraint, which we call local flatness, is proposed, and a new measure of Local Deviation ( LD ) is developed to quantify the degree of local illumination flatness; 2) a variational problem is defined and the solution is found by a logical sequence of steps; 3) discrete implementation of the variational solution is shown to effectively estimate and remove uneven illumination, yielding an accurate recovered image. Unlike other physical prior based variational Retinex models, which use the L2 norm of the illumination gradient to enforce smoothness of illumination, our LD prior selectively imposes local flatness on illumination by calculating the deviation between the estimated illumination surface to a reference plane. In the experiments, pseudo ground truth images are created by superimposing uneven illumination on real scenes, providing an effective way to objectively assess algorithm performance. The experimental results show that our method can reconstruct more accurate recovered images than other state-of-the-art methods, while maintaining good contrast.

Non-ideal iris segmentation using Polar Spline RANSAC and illumination compensation

In this work, we propose a robust iris segmentation method for non-ideal ocular images, referred to as Polar Spline RANSAC, which approximates the iris shape as a closed curve with arbitrary degrees of freedom. The method is robust to several nonidealities, such as poor contrast, occlusions, gaze deviations, pupil dilation, motion blur, poor focus, frame interlacing, differences in image resolution, specular reflections, and shadows. Unlike most techniques in the literature, the proposed method obtains good performance in harsh conditions with different imaging wavelengths and datasets. We also investigate the role of different illumination compensation techniques on the iris segmentation process. The experiments showed that the proposed method results in higher or comparable accuracy with respect to other competing techniques presented in the literature for images acquired in non-ideal conditions. Furthermore, the proposed segmentation method is generalizable and can achieve competitive performance with different state-of-the-art feature extraction and matching techniques. In particular, in conjunction with a well-known recognition schema, it achieved Equal Error Rate of 4.34% on DB WVU, Equal Error Rate of 5.98% on DB QFIRE, and pixel-wise classification error rate of 0.0165 on DB UBIRIS v2. Moreover, experiments using different illumination compensation techniques demonstrate that algorithms based on the Retinex model offer improved segmentation and recognition accuracy, thereby highlighting the importance of adopting illumination models for processing non-ideal ocular images.

Efficient medical image enhancement based on CNN‐FBB model

Medical image quality requirements have been increasingly stringent with the recent developments of medical technology. To meet clinical diagnosis needs, an effective medical image enhancement method based on convolutional neural networks (CNNs) and frequency band broadening (FBB) is proposed. Curvelet transform is used to deal with medical data by obtaining the curvelet coefficient in each scale and direction, and the generalised cross-validation is implemented to select the optimal threshold for performing denoising processing. Meanwhile, the cycle spinning scheme is used to wipe off the visible ringing effects along the edges of medical images. Then, FBB and a new CNN model based on the retinex model are used to improve the processed image resolution. Eventually, pixel-level fusion is made between two enhanced medical images from CNN and FBB. In the authors’ study, 50 groups of medical magnetic resonance imaging, X-ray, and computed tomography images in total have been studied. The experimental results indicate that the final enhanced image using the proposed method outperforms other methods. The resolution and the edge details of the processed image are significantly enhanced, providing a more effective and accurate basis for medical workers to diagnose diseases.

Adaptive Image Rendering Using a Nonlinear Mapping-Function-Based Retinex Model.

This paper introduces an adaptive image rendering using a parametric nonlinear mapping-function-based on the retinex model in a low-light source. For this study, only a luminance channel was used to estimate the reflectance component of an observed low-light image, therefore halo artifacts coming from the use of the multiple center/surround Gaussian filters were reduced. A new nonlinear mapping function that incorporates the statistics of the luminance and the estimated reflectance in the reconstruction process is proposed. In addition, a new method to determine the gain and offset of the mapping function is addressed to adaptively control the contrast ratio. Finally, the relationship between the estimated luminance and the reconstructed luminance is used to reconstruct the chrominance channels. The experimental results demonstrate that the proposed method leads to the promised subjective and objective improvements over state-of-the-art, scale-based retinex methods.

Efficient underwater image and video enhancement based on Retinex

The Retinex models the human visual system to perceive natural colors, which could improve the contrast and sharpness of the degraded image and also provide color constancy and dynamic range simultaneously. This endows the Retinex exceeding advantages for enhancing the underwater image. Based on the multi-scale Retinex, an efficient enhancement method for underwater image and video is presented in this paper. Firstly, the image is pre-corrected to equalize the pixel distribution and reduce the dominating color. Then, the classical multi-scale Retinex with intensity channel is applied to the pre-corrected images for further improving the contrast and the color. In addition, multi-down-sampling and infinite impulse response Gaussian filtering are adopted to increase processing speed. Subsequently, the image is restored from logarithmic domain and the illumination of the restored image is compensated based on statistical properties. Finally, the color is selectively preserved by the inverted gray world method depending on imaging conditions and application requirements. Five kinds of typical underwater images with green, blue, turbid, dark and colorful backgrounds and two underwater videos are enhanced and evaluated on Jetson TX2, respectively, to verify the effectiveness of the proposed method.

A Unified Variational Model for Single Image Dehazing

Haze is a common weather phenomenon, which hinders many outdoor computer vision applications such as outdoor surveillance, navigation control, vehicle driving, and so on. In this paper, a simple but effective unified variational model for single image dehazing is presented based on the total variation regularization. From the perspective of the relationship between image dehazing and Retinex, the dehazing problem can be formulated as the minimization of a variational Retinex model. The proposed variational model incorporates two ${\ell _{1}}$ -norm regularization terms to constrain the scene transmission and the inverted scene radiance respectively, which can be better applied into image dehazing field. Different from the conventional two-step framework, our proposed model can simultaneously obtain the accurate transmission map and the recovered scene radiance by integrating the transmission estimation stage and the image recovery stage into the unified variational model. The entire optimization of the proposed unified variational model can be solved by an alternating direction minimization scheme. The experiments on various simulated and real-world hazy images indicate that the proposed algorithm can yield considerably promising results comparative to several state-of-the-art dehazing and enhancement techniques.

A Low-Light Sensor Image Enhancement Algorithm Based on HSI Color Model.

Images captured by sensors in unpleasant environment like low illumination condition are usually degraded, which means low visibility, low brightness, and low contrast. In order to improve this kind of images, in this paper, a low-light sensor image enhancement algorithm based on HSI color model is proposed. At first, we propose a dataset generation method based on the Retinex model to overcome the shortage of sample data. Then, the original low-light image is transformed from RGB to HSI color space. The segmentation exponential method is used to process the saturation (S) and the specially designed Deep Convolutional Neural Network is applied to enhance the intensity component (I). At the end, we back into the original RGB space to get the final improved image. Experimental results show that the proposed algorithm not only enhances the image brightness and contrast significantly, but also avoids color distortion and over-enhancement in comparison with some other state-of-the-art research papers. So, it effectively improves the quality of sensor images.