Haze Removal Research Articles

Single underwater image haze removal with a learning-based approach to blurriness estimation

Underwater images are usually degraded due to light scattering and absorption. To recover the scene radiance of degraded underwater images, a new haze removal method is presented by incorporating a learning-based approach to blurriness estimation with the image formation model. Firstly, the image blurriness is estimated with a linear model trained on a set of selected grayscale images, the average Gaussian images and blurriness images. With the estimated image blurriness, three intermediate background lights (BLs) are computed to obtain the synthesized BL. Then the scene depth is calculated by using the estimated image blurriness and BL to construct a transmission map and restore the scene radiance. Compared with other haze removal methods, haze in degraded underwater images can be removed more accurately with our proposed method. Moreover, visual inspection, quantitative evaluation and application test demonstrate that our method is superior to the compared methods and beneficial to high-level vision tasks.

Semi‐MoreGAN: Semi‐supervised Generative Adversarial Network for Mixture of Rain Removal

AbstractReal‐world rain is a mixture of rain streaks and rainy haze. However, current efforts formulate image rain streaks removal and rainy haze removal as separated models, worsening the loss of image details. This paper attempts to solve the mixture of rain removal problem in a single model by estimating the scene depths of images. To this end, we propose a novel SEMI‐supervised Mixture Of rain REmoval Generative Adversarial Network (Semi‐MoreGAN). Unlike most of existing methods, Semi‐MoreGAN is a joint learning paradigm of mixture of rain removal and depth estimation; and it effectively integrates the image features with the depth information for better rain removal. Furthermore, it leverages unpaired real‐world rainy and clean images to bridge the gap between synthetic and real‐world rain. Extensive experiments show clear improvements of our approach over twenty representative state‐of‐the‐arts on both synthetic and real‐world rainy images. Source code is available at https://github.com/syy-whu/Semi-MoreGAN.

Single nighttime image dehazing based on unified variational decomposition model and multi-scale contrast enhancement

Most of existing dehazing methods are unable to deal with nighttime hazy scenarios well due to complex degraded factors such as non-uniform illumination, low light, glows and hazes. To obtain high-quality image under nighttime haze imaging conditions, we propose a single nighttime image dehazing framework based on a unified variational decomposition model and multi-scale contrast enhancement to simultaneously address these undesirable issues. First, a unified variational decomposition model consisting of three regularization terms is proposed to simultaneously decompose a nighttime hazy image into a structure layer, a detail layer and a noise layer. Concretely, we employ ℓ1 norm to constrain the structure component, adopt ℓ0 sparsity term to enforce the piece-wise continuous of the residual of the gradients between the detail layer and the modified glow-free image, and use the Frobenius norm to estimate the noise layer. Next, the hazes in the structure layer are removed by inversing the physical model and the effective details in the texture layers are enhanced while the amplified noises are suppressed in a multi-scale fashion. Finally, the dehazed structure layer and the enhanced detail layers are integrated into a haze-free image. Experiments demonstrate that the proposed framework achieves superior performance on nighttime haze removal and noise suppression compared with state-of-the-art dehazing techniques.

(Retracted) Method of defogging unmanned aerial vehicle images based on intelligent manufacturing

Intelligent manufacturing is a major trend in manufacturing innovation around the world, and it is also the main direction and key breakthrough point for the transformation and upgrading of the manufacturing industry for a long time now and in the future. Here, we mainly study the role of intelligent manufacturing in defogging unmanned aerial vehicle (UAV) images and how to analyze UAV image defogging methods based on intelligent manufacturing. In recent years, with the continuous progress of UAV technology, UAV project has also been booming. Aerial photographing is one of the most widely used functions of UAV at present. However, when photographing images in severe weather environment such as haze, it will be affected by the absorption and scattering of light by a variety of different suspended substances in the environment, resulting in poor imaging quality, color distortion, fine pitch blur, and other adverse effects. For this reason, there is a large amount of research on the image haze removal of UAV in China. At present, the mainstream methods are based on non-vision sensor and physical model. However, due to the lack of haze removal effect and severe condition limitation in the practical application of the above methods, we explore an innovative set of automatic estimation haze removal method of atmospheric light based on this method, taking the principle of atmospheric estimation direction as the main research direction, and then through the steps of setting the global transmittance, calculating the atmospheric light amplitude, adjusting the image size, and automatically modifying the image condition threshold value, etc. The man–machine image is haze removal. To compare the advantages and disadvantages of the three methods, we choose standard deviation, information entropy, and objective evaluation method to analyze the results of the three methods. Data analysis shows that among the three haze removal methods, the atmospheric light automatic estimation haze removal method has greatly improved the overall haze removal effect. Compared with the previous two kinds of haze removal imaging in the haze environment, it can better guarantee the color balance degree, and also retain more image information, which makes the imaging clarity have a significant improvement, and the overall effect is more natural. This method has played a very good complementary role to the domestic UAV demisting technology.

Trends and Prospects of Techniques for Haze Removal From Degraded Images: A Survey

For the last two decades, image processing techniques have been used frequently in computer vision applications. The most challenging task in image processing is restoring images that are degraded due to various weather conditions. Mainly, the visibility of outdoor images is corrupted due to adverse atmospheric effects. The visibility of acquired images is reduced in these circumstances. Haze is an atmospheric phenomenon that reduces the clarity of an image. Due to the presence of particles such as dust, dirt, soot, or smoke, there is significant decay in the color and contrast of captured images. Haze present in acquired images causes issues in a variety of computer vision applications. Therefore, enhancing the contrast of a hazy image and restoring the visibility of the scene is essential. Since clear images are required in every application, image dehazing is an important step. Hence, many researchers are working on it. Different methods have been presented in the literature for image dehazing. This study describes various traditional and deep learning techniques of image dehazing from an analytical perspective. The main intention behind this work is to provide an intuitive understanding of the major techniques that have made a relevant contribution to haze removal. In this paper, we have covered different types of contributions toward dehazing based on the traditional method as well as deep learning approaches. With a considerable amount of instinctive simplifications, the reader is expected to have an improved ability to visualize the internal dynamics of these processes.

Single image dehazing with an independent Detail-Recovery Network

Single image dehazing is a prerequisite that affects the performance of many visually related tasks and has attracted increasing attention in recent years. However, most existing dehazing methods place more emphasis on haze removal but less on the detail recovery of the dehazed images. In this paper, we propose a single image dehazing method with an independent Detail Recovery Network (DRN), which considers capturing the details from the input image over a separate network and then integrating them into a coarse dehazed image. The overall network consists of two independent networks, named DRN and the dehazing network. Specifically, the DRN aims to recover the dehazed image details through the joint efforts of the local branch and the global branch. The local branch can obtain local detail information through the convolution layer, and the global branch can capture multi-scale global information by Smooth Dilated Convolution (SDC). In addition, we apply multi-faceted loss to improve the stability of the dehazing model. Extensive experiments on public image dehazing datasets illustrate the effectiveness of the modules in the proposed method and reveal that our method outperforms state-of-the-art dehazing methods. The code is released in https://github.com/YanLi-LY/Dehazing-DRN.

Classification of Mammographic ROI for Microcalcification Detection Using Multifractal Approach

Microcalcifications (MCs) are the main signs of precancerous cells. The development of aided-system for their detection has become a challenge for researchers in this field. In this paper, we propose a system for MCs detection based on the multifractal approach that classifies mammographic ROIs into normal (healthy) or abnormal ROIs containing MCs. The proposed method is divided into four main steps: a mammogram pre-processing step based on breast selection, breast density reduction using haze removal algorithm and contrast enhancement using multifractal measures. The second step consists of extracting the normal and abnormal ROIs and calculating the multifractal spectrum of each ROI. The next step represents the extraction of the multifractal features from the multifractal spectrum and the GLCM characteristics of each ROI. The last step is the classification of ROIs where three classifiers are tested (KNN, DT, and SVM). The system is evaluated on images from the INbreast database (308 images) with a total of 2688 extracted ROIs (1344 normal, 1344 with MC) from different BI-RADS classes. In this study, the SVM classifier gave the best classification results with a sensitivity, specificity, and precision of 98.66%, 97.77%, and 98.20% respectively. These results are very satisfactory and remarkable compared to the literature.

Rich feature distillation with feature affinity module for efficient image dehazing

Single-image haze removal is a long-standing hurdle for computer vision applications. Several works have been focused on transferring advances from image classification, detection, and segmentation to the niche of image dehazing, primarily focusing on contrastive learning and knowledge distillation. However, these approaches prove computationally expensive, raising concern regarding their applicability to on-the-edge use-cases. This work introduces a simple, lightweight, and efficient framework for single-image haze removal, exploiting rich “dark-knowledge” information from a lightweight pre-trained super-resolution model via the notion of heterogeneous knowledge distillation. We designed a feature affinity module to maximize the flow of rich feature semantics from the super-resolution teacher to the student dehazing network. In order to evaluate the efficacy of our proposed framework, its performance as a plug-and-play setup to a baseline model is examined. Our experiments are carried out on the RESIDE-Standard dataset to demonstrate the robustness of our framework to the synthetic and real-world domains. The extensive qualitative and quantitative results provided establish the effectiveness of the framework, achieving gains of up to 15% (PSNR) while reducing the model size by ∼20 times.

Deep Multimodal Detection in Reduced Visibility Using Thermal Depth Estimation for Autonomous Driving.

Recently, the rapid development of convolutional neural networks (CNN) has consistently improved object detection performance using CNN and has naturally been implemented in autonomous driving due to its operational potential in real-time. Detecting moving targets to realize autonomous driving is an essential task for the safety of drivers and pedestrians, and CNN-based moving target detectors have shown stable performance in fair weather. However, there is a considerable drop in detection performance during poor weather conditions like hazy or foggy situations due to particles in the atmosphere. To ensure stable moving object detection, an image restoration process with haze removal must be accompanied. Therefore, this paper proposes an image dehazing network that estimates the current weather conditions and removes haze using the haze level to improve the detection performance under poor weather conditions due to haze and low visibility. Combined with the thermal image, the restored image is assigned to the two You Only Look Once (YOLO) object detectors, respectively, which detect moving targets independently and improve object detection performance using late fusion. The proposed model showed improved dehazing performance compared with the existing image dehazing models and has proved that images taken under foggy conditions, the poorest weather for autonomous driving, can be restored to normal images. Through the fusion of the RGB image restored by the proposed image dehazing network with thermal images, the proposed model improved the detection accuracy by up to 22% or above in a dense haze environment like fog compared with models using existing image dehazing techniques.

Multi-scale dehazing network via high-frequency feature fusion

Numerous learning-based methods have achieved significant improvements in haze removal. However, the dehazed results of these methods still suffer from the loss of edge details. To solve this problem, we propose a novel multi-scale dehazing network via high-frequency feature fusion (HFMDN). HFMDN is an end-to-end trainable network, which is mainly composed of four components: a base network (Backbone), a frequency branch network (FBN), a frequency attention module (FAM), and a refine block (RB). The Backbone is a multi-scale feature fusion architecture that can share useful information across different scales. For the training phase, we employ the Laplace Operator to obtain the image’s high-frequency (HF) information, which can specifically represent the details of the image (e.g., edges, textures). The FBN takes the HF derived from the original image as an additional prior and utilizes L1 norm loss to constrain the output of FBN to predict the HF of the haze-free image. We further design a frequency attention module (FAM), which automatically learns the weights map of the frequency features to enhance image recovery ability. Furthermore, a refine block (RB) is proposed to extract the features map by fusing the outputs of FBN and Backbone to produce the final haze-free image. The quantitative comparison of the ablation study shows that high-frequency information significantly improves dehazing performance. Extensive experiments also demonstrate that our proposed methods can generate more natural and realistic haze-free images, especially in the contours and details of hazy images. HFMDN performs favorably against the CNN-based state-of-the-art dehazing methods in terms of PSNR, SSIM, and visual effect.

IDBP: Image Dehazing Using Blended Priors Including Non-Local, Local, and Global Priors

In this letter, a robust and promising atmospheric scattering model (ASM)-based image dehazing technique called IDBP is developed, which overcomes the intrinsic limitation of available techniques based on single priors. It consists of two modules, i.e., an atmospheric light estimation (ALE) module and a multiple prior constraint (MPC) module. The ALE module is based on a new global brightening strategy of enhancing the brightness of image with minimum information loss. The MPC smartly blends the constrains of non-local prior, local prior, and global prior to shrink the solution space of haze removal, which avoids the limitation of using any single priors. Unlike previous works, IDBP does not require any training process, but is based on multiple priors and minimal information loss principle to impose the ASM, thereby making it easy to implement and ensuring its robustness. Numerous experiments reveal that the proposed IDBP outperforms the state-of-the-art alternates.

Non-Local Retinex Based Dehazing and Low Light Enhancement of Images

We know that a vast amount of research has recently been done on dehazing single images. More work is done on day-time images than night-time images. Also, enhancement of low light images is another area in which lots of research going on. In this paper, a simple yet effective unified variational model is proposed for dehazing of day and night images and low-light enhancement based on non-local global variational regularization. Given the relation between image dehazing and retinex, the haze removal process can minimize a variational retinex model. Estimating of ambient light and transmission maps is a key step in modern dehazing methods. Atmospheric light is not uniform and constant for hazy night images, as night scenes often contain multiple light sources. Often lit and non-illuminated regions have different colour characteristics and cause total variation colour distortion and halo artifacts. Our work directly implements a non-local retinal model based on the L2 norm that simulates the average activity of inhibitory and excitatory neuronal populations in the cortex to overcome this problem. This potential biological feasibility of the L2 norm of our work is divided into two parts using a filtered gradient approach, the reflection sparse prior and the reflection gradient fidelity before the observed image gradient. This unified framework of NLTV-Retinex and DCP efficiently performs low-light enhancement and dehazing of day and night images. We show results obtained using our method on daytime and night-time images and a low-light image dataset. We quantitatively and qualitatively compare our results with recently reported methods, which demonstrate the effectiveness of our method.

Uncertainty-Driven Dehazing Network

Deep learning has made remarkable achievements for single image haze removal. However, existing deep dehazing models only give deterministic results without discussing the uncertainty of them. There exist two types of uncertainty in the dehazing models: aleatoric uncertainty that comes from noise inherent in the observations and epistemic uncertainty that accounts for uncertainty in the model. In this paper, we propose a novel uncertainty-driven dehazing network (UDN) that improves the dehazing results by exploiting the relationship between the uncertain and confident representations. We first introduce an Uncertainty Estimation Block (UEB) to predict the aleatoric and epistemic uncertainty together. Then, we propose an Uncertainty-aware Feature Modulation (UFM) block to adaptively enhance the learned features. UFM predicts a convolution kernel and channel-wise modulation cofficients conitioned on the uncertainty weighted representation. Moreover, we develop an uncertainty-driven self-distillation loss to improve the uncertain representation by transferring the knowledge from the confident one. Extensive experimental results on synthetic datasets and real-world images show that UDN achieves significant quantitative and qualitative improvements, outperforming the state-of-the-arts.

Image and video dehazing based on transmission estimation and refinement using Jaya algorithm

Due to its in-built ill-posed nature, image and video dehazing is an immensely thought-provoking problem, and it is an essential pre-processing step for various applications such as ADAS (automatic driver-assistance systems), security and surveillance, remote sensing, and long-range imaging. We proposed a novel single image and video dehazing method, called JA-dehaze, by using Jaya Algorithm (JA) to address this problem. The technique consists of three factors: Discrete Haar Wavelet Transform (DHWT) for sub-bands, local atmospheric light estimation and transmission map estimation and refinement using Jaya Algorithm. We formulated haze removal as an optimization problem, and the fitness function for the single image is taken from the information loss term, distance and the haze density term. The JA-dehaze method improves the gradient and preserves the information optimally by minimizing the fitness function. Furthermore, we extended the single image dehazing process to real-time video dehazing. Experimental results demonstrate that the proposed JA-dehaze method can successfully improve the quality of a foggy image without sacrificing color fidelity and can retain image details sufficiently.

Rapid nighttime haze removal with color-gray layer decomposition

Nighttime haze usually reduces the visible distance and visual contrast of the nighttime scene. To resolve this problem, a color-gray layer decomposition method is presented to decompose a real-world image into color-world and gray-world layers in the RGB color space. Based on the presented layer decomposition, a new nighttime dehazing framework is proposed to remove glowing color, enhance illumination, and remove haze. Specifically, to correct nighttime hazy images, an effective glow decomposition approach is present to remove glowing color in the color-world layer. For low-light nighttime hazy scenes, gamma correction is applied to enhance illumination in the gray-world layer. After nighttime hazy image correction, to further refine the coarse estimation of the transmission and the atmospheric light, a relative weighted guided image filter is devised with a new relative edge-aware weighting. Additionally, a 2D atmospheric scattering model is established to well remove haze with the decomposed transmission and the fused local atmospheric light. Therefore, the proposed method can rapidly recover high-quality nighttime dehazing results and preserve the intrinsic color of the scene while avoiding color artifacts. Quantitative comparisons verify that the proposed nighttime dehazing framework can achieve excellent performance in nighttime haze removal against previous state-of-the-art methods.

Guided-Pix2Pix+: End-to-end spatial and color refinement network for image dehazing

Inevitable hazy contamination degrades the visibility of images, and the resulting haze removal is one of the essential prerequisites for image processing and computer vision tasks. We proposed an end-to-end dehazing network, referred to as Guided-Pix2Pix, to estimate spatially refined transmission maps and dehaze via the physical scattering equation. The remaining enhancement of color contrast, ill-posed adversarial training, and redundant backbone, however, should be thoroughly investigated, as required in the prospect of Guided-Pix2Pix. In this paper, we inherit the end-to-end structure of Guided-Pix2Pix and accordingly propose Guided-Pix2Pix+ as an update, which concatenates transmission estimation and refinement, physical scattering equation-based dehazing, together with color refinement, to achieve haze removal in a one-stage way. Specifically, we make use of the pretrained instance of EfficientNetB0 to estimate coarse-grained transmission maps and concatenate a guided filter layer to perform spatial refinement for the incoming transmission maps. Restored by the physical scattering equation, color refinement of dehazed proposals is finally performed via the standardization and clipping of pixel intensities. All the operations are differentiable, making it possible to achieve end-to-end, tight training. Furthermore, adversarial and perceptual losses are employed to regulate the performance of our model, giving rise to structurally similar but photo-realistic dehazed proposals. Extensive experiments confirm that our Guided-Pix2Pix+ yields dehazed proposals with fine-grained spatial refinement and relatively effective color contrast, compared to our previous Guided-Pix2Pix, the baseline, and advanced dehazing methods. The source code is currently available at https://github.com/92xianshen/guided-pix2pixplus.

Low‐light image haze removal with light segmentation and nonlinear image depth estimation

Hazy image obtained in the low-light environment has the characteristics of low contrast, non-uniform illumination, color cast and much noise. In this paper, a method is put forward which can be properly applied to recover low-light hazy images. The original image is first decomposed into glow layer and haze layer with a modified color channel transformation for glow artifacts and color balanced. A new light segmentation function is proposed next by using gamma correction of channel difference and setting threshold levels to determine if the pixel belongs to light source regions. Then the ambient illuminance map is estimated using maximum reflectance prior to computing the atmosphere light in the light and non-light regions. Finally, a novel nonlinear image depth estimation model is established to build the relationship between the image depth map and three image features including luminance, saturation and gradient map for the light areas. The experimental results prove that the dehazing algorithm is reliable for removing haze and glow artifacts of active light sources, reducing much noise and improving the visibility.

DesmokeNet: A Two-Stage Smoke Removal Pipeline Based on Self-Attentive Feature Consensus and Multi-Level Contrastive Regularization

In image processing, smoke may degrade visibility and deteriorate the performance of high-level vision applications. Therefore, single image smoke removal is crucial for computer vision. Currently, existing smoke removal algorithms mainly leverage handcrafted priors. Moreover, these methods usually apply haze removal methods to perform smoke removal due to the similarity between smoke and haze. However, these methods cannot sufficiently address the degradation of thick smoke and may suffer from residual smoke and color distortion problems due to the non-global and non-homogeneous distribution of smoke. In this paper, to solve the aforementioned problems, an end-to-end deep neural network called DesmokeNet is proposed. We construct a two-stage recovered pipeline to remove the smoke in different thicknesses. The light and thick smoke is first removed locally by the smoke removal network (SRN). The missing pixels in the thick smoke are then recovered by the pixel compensation network (PCN). Moreover, we proposed the thickness-aware pixel loss and the dark channel loss to suppress the residual smoke. To further increase the discriminative ability of the DesmokeNet, we proposed self-attentive feature consensus loss and multi-level contrastive regularization loss to improve the performance of smoke removal. Finally, to train the proposed method, we construct the first large-scale dataset containing synthetic and real-world data. Extensive experiments show that the proposed method outperforms favorably against other state-of-the-art methods quantitatively and qualitatively.

Single-image night haze removal based on color channel transfer and estimation of spatial variation in atmospheric light

The visible-light imaging system used in military equipment is often subjected to severe weather conditions, such as fog, haze, and smoke, under complex lighting conditions at night that significantly degrade the acquired images. Currently available image defogging methods are mostly suitable for environments with natural light in the daytime, but the clarity of images captured under complex lighting conditions and spatial changes in the presence of fog at night is not satisfactory. This study proposes an algorithm to remove night fog from single images based on an analysis of the statistical characteristics of images in scenes involving night fog. Color channel transfer is designed to compensate for the high attenuation channel of foggy images acquired at night. The distribution of transmittance is estimated by the deep convolutional network DehazeNet, and the spatial variation of atmospheric light is estimated in a point-by-point manner according to the maximum reflection prior to recover the clear image. The results of experiments show that the proposed method can compensate for the high attenuation channel of foggy images at night, remove the effect of glow from a multi-color and non-uniform ambient source of light, and improve the adaptability and visual effect of the removal of night fog from images compared with the conventional method.

Single Image Haze Removal Based on a Simple Additive Model With Haze Smoothness Prior

Single image haze removal, which is to recover the clear version of a hazy image, is a challenging task in computer vision. In this paper, an additive haze model is proposed to approximate the hazy image formation process. In contrast with the traditional optical model, it regards the haze as an additive layer to a clean image. The model thus avoids estimating the medium transmission rate and the global atmospherical light. In addition, based on a critical observation that haze changes gradually and smoothly across the image, a haze smoothness prior is proposed to constrain this model. This prior assumes that the haze layer is much smoother than the clear image. Benefiting from this prior, we can directly separate the clean image from a single hazy image. Experimental results and comparisons with synthetic images and real-world images demonstrate that the proposed method outperforms state-of-the-art single image haze removal algorithms.