Lightweight Underwater Image Enhancement via Impulse Response of Low-Pass Filter Based Attention Network

Since underwater imaging is affected by the complex water environment, it often leads to severe distortion of the underwater image. To improve the quality of underwater images, underwater image enhancement and restoration methods have been proposed. However, many underwater image enhancement and restoration methods produce over-enhancement or under-enhancement, which affects their application. To better design underwater image enhancement and restoration methods, it is necessary to research the underwater image quality evaluation (UIQE) for underwater image enhancement and restoration methods. Therefore, a subjective evaluation dataset for an underwater image enhancement and restoration method is constructed, and on this basis, an objective quality evaluation method of underwater images, based on the relative symmetry of underwater dark channel prior (UDCP) and the underwater bright channel prior (UBCP) is proposed. Specifically, considering underwater image enhancement in different scenarios, a UIQE dataset is constructed, which contains 405 underwater images, generated from 45 different underwater real images, using 9 representative underwater image enhancement methods. Then, a subjective quality evaluation of the UIQE database is studied. To quantitatively measure the quality of the enhanced and restored underwater images with different characteristics, an objective UIQE index (UIQEI) is used, by extracting and fusing four groups of features, including: (1) the joint statistics of normalized gradient magnitude (GM) and Laplacian of Gaussian (LOG) features, based on the underwater dark channel map; (2) the joint statistics of normalized gradient magnitude (GM) and Laplacian of Gaussian (LOG) features, based on the underwater bright channel map; (3) the saturation and colorfulness features; (4) the fog density feature; (5) the global contrast feature; these features capture key aspects of underwater images. Finally, the experimental results are analyzed, qualitatively and quantitatively, to illustrate the effectiveness of the proposed UIQEI method.

Images taken under water usually suffer from the problems of quality degradation, such as low contrast, blurring details, color deviations, non-uniform illumination, etc. As an important problem in image processing and computer vision, the restoration and enhancement of underwater image are necessary for numerous practical applications. Over the last few decades, underwater image restoration and enhancement have been attracting an increasing amount of research effort. However, a comprehensive and in-depth survey of related achievements and improvements is still missing, especially the survey of underwater image dataset which is a key issue in underwater image processing and intelligent application. In this exposition, we first summarize more than 120 studies about the latest progress in underwater image restoration and enhancement, including the techniques, datasets, available codes, and evaluation metrics. We analyze the contributions and limitations of existing methods to facilitate the comprehensive understanding of underwater image restoration and enhancement. Furthermore, we provide detailed objective evaluations and analysis of the representative methods on five types of underwater scenarios, which verifies the applicability of these methods in different underwater conditions. Finally, we discuss the potential challenges and open issues of underwater image restoration and enhancement and suggest possible research directions in the future.

Chapter 7 - Underwater image enhancement: past, present, and future

In turbid water, the attenuation and scattering of light caused by scatterers make underwater optical images degraded, blurred, and contrast reduced, limiting the extraction and analysis of information from images. To address such problems, a turbid underwater image enhancement method based on parameter-tuned stochastic resonance (SR) is proposed in this article. First, an SR algorithm framework for underwater image enhancement is constructed, including the dimensionality reduction and normalization of input images, the solution and parameter optimization of the SR system, the dimensionality upgrading of output images, etc. This framework can apply the SR's ability to enhance weak signals to the enhancement of turbid underwater images. Second, to measure the performance of the system, a synthetic turbid underwater image data set (UWCHIC) is constructed using the underwater imaging model and an image set with simulated scatterers. Based on this data set, the relationship between various image quality evaluation metrics and system parameters is analyzed, and then the suitable no-reference (NR) metrics for system performance evaluation are selected and an adaptive parameter tuning strategy of the SR system is proposed to guide the image enhancement. Lastly, the proposed method is evaluated on the UWCHIC, a dataset to evaluate underwater image restoration methods (TURBID), marine underwater environment database (MUED), and underwater image enhancement benchmark (UIEB) data sets and the turbid underwater images captured from natural waters. Different experimental evaluations demonstrated that the proposed method not only effectively enhances the visual quality of turbid underwater images but also improves the performance of downstream vision tasks.

Currently, optical imaging cameras are widely used on underwater vehicles to obtain images and support numerous marine exploration tasks. Many underwater image enhancement algorithms have been proposed in the past few years to suppress backscattering noise and improve the signal-to-noise ratio of underwater images. However, these algorithms are mainly focused on underwater image enhancement tasks in a bright environment. Thus, it is still unclear how these algorithms would perform on images acquired in an underwater scene with low illumination. Images obtained in a dark underwater scene usually include more noise and have very low visual quality, which may easily lead to artifacts during the process of enhancement. To bridge this gap, we thoroughly study the existing underwater image enhancement methods and low illumination image enhancement methods based on deep learning and propose a new underwater image enhancement network to solve the problem of serious degradation of underwater image quality in a low illumination environment. Due to the lack of ready-made datasets for training, we also propose the first dataset for low-light underwater image enhancement to train our model. Our method can be implemented to skillfully and simultaneously address low-light degradation and scattering degradation in low-light underwater images. Experimental results also show that our method is robust against different illumination levels, which greatly expands the applicable scenarios of our method. Compared with previous underwater image enhancement methods and low-light image enhancement methods, outstanding performance is achieved using our method in various low-light underwater scenes.

Underwater images usually suffer from colour distortion, blur, and low contrast, which hinder the subsequent processing of underwater information. To address these problems, this paper proposes a novel approach for single underwater images enhancement by integrating data‐driven deep learning and hand‐crafted image enhancement techniques. First, a statistical analysis is made on the average deviation of each channel of input underwater images to that of its corresponding ground truths, and it is found that both the red channel and the green channel of an underwater image contribute to its colour distortion. Concretely, the red channel of an underwater image is usually seriously attenuated, and the green channel is usually over strengthened. Motivated by such an observation, an attention mechanism guided residual module for underwater image colour correction is proposed, where the colour of the red channel of the underwater image and that of the green channel is compensated in a different way, respectively. Coupled with an attention mechanism, the residual module can adaptively extract and integrate the most discriminative features for colour correction. For scene contrast enhancement and scene deblurring, the traditional image enhancement techniques such as CLAHE (contrast limited adaptive histogram equalization) and Gamma correction are coupled with a multi‐scale convolutional neural network (MSCNN), where CLAHE and Gamma correction are used as complement to deal with the complex and changeable underwater imaging environment. Experiments on synthetic and real underwater images demonstrate that the proposed method performs favourably against the state‐of‐the‐art underwater image enhancement methods.

Due to complex and volatile lighting environment, underwater imaging can be readily impaired by light scattering, warping, and noises. To improve the visual quality, Underwater Image Enhancement (UIE) techniques have been widely studied. Recent efforts have also been contributed to evaluate and compare the UIE performances with subjective and objective methods. However, the subjective evaluation is time-consuming and uneconomic for all images, while existing objective methods have limited capabilities for the newly-developed UIE approaches based on deep learning. To fill this gap, we propose an Underwater Image Fidelity (UIF) metric for objective evaluation of enhanced underwater images. By exploiting the statistical features of these images in CIELab space, we present the naturalness, sharpness, and structure indexes. Among them, the naturalness and sharpness indexes represent the visual improvements of enhanced images; the structure index indicates the structural similarity between the underwater images before and after UIE. We combine all indexes with a saliency-based spatial pooling and thus obtain the final UIF metric. To evaluate the proposed metric, we also establish a first-of-its-kind large-scale UIE database with subjective scores, namely Underwater Image Enhancement Database (UIED). Experimental results confirm that the proposed UIF metric outperforms a variety of underwater and general-purpose image quality metrics. The database and source code are available at https://github.com/z21110008/UIF.

Images captured underwater usually suffer from color distortion, detail blurring, low contrast, and a bluish or greenish tone due to light scattering and absorption in the underwater medium, which in turn the visibility is adversely affected by these factors seriously. Over the last decades, various image restoration and enhancement methods have been developed by many researchers to improve the quality (visibility and highlight richer details) of underwater images. This paper introduces the overview of state-of-the-art underwater image restoration and enhancement techniques and classifies the approaches in two categories: image restoration (physical-based model) and image enhancement (nonphysical-based model). Furthermore, the classification of these two methods is elaborated. Then, the typical underwater image restoration and enhancement methods are discussed in detail, as well as a comprehensive study and fair evaluation of the methods is carried out from both qualitative and quantitative perspectives. Finally, the research process of underwater image restoration and enhancement is summarized and the suggestions for future research are prospected.

Two-step domain adaptation for underwater image enhancement

Underwater images are usually suffering from the issues of quality degradation, such as low contrast due to blurring details, color deviations, non-uniform lighting, and noise. Since last few decades, many researches are undergoing for restoration and enhancement for degraded underwater images. In this paper, we proposed a novel algorithm using modified anisotropic diffusion filter with dynamic color balancing strategy. This proposed algorithm performs based on an employing effective noise reduction as well as edge preserving technique with dynamic color correction to make uniform lighting and minimize the speckle noise. Furthermore, reanalyze the contributions and limitations of existing underwater image restoration and enhancement methods. Finally, in this research provided the detailed objective evaluations and compared with the various underwater scenarios for above said challenges also made subjective studies, which shows that our proposed method will improve the quality of the image and significantly enhanced the image.

Underwater images suffer from severe color casts, low contrast, and blurriness, which greatly degrade the visibility and color fidelity of underwater images. Recently, numerous underwater image enhancement (UIE) algorithms have been proposed. Existing synthetic datasets-based deep learning methods employ synthetic datasets to train UIE models. However, there is a gap between synthetic datasets and real underwater images, leading to poor generalization of synthetic datasets-based UIE methods. Besides, existing real datasets-based deep learning methods largely focus on minimizing the mean squared reconstruction error between UIE results and corresponding ground-truth on the real datasets, but do not take human visual perception into account. Thus, although they achieve high PSNR between UIE results and corresponding ground-truth obtained by user study on the real datasets, they often achieve unsatisfactory perceptual quality. To address these problems, we propose a Human Perceptual Quality Driven Underwater Image Enhancement Framework (HPQ-UIEF) to achieve better results in human perceptual quality and maintain satisfactory PSNR, which is trained on a real underwater enhancement quality assessment database (UEQAB). Specifically, an Underwater Image Quality Assessment Network (UIQAN) for UIE images is first proposed to assist UIE task, in which a novel depth map prior spatial attention block (DPPAB) is embedded into UIQAN. The DPPAB can adaptively recalibrate the quality-aware feature maps and model human visual attention in a data-driven manner. Then, the UIE model is proposed, in which the UIQAN is introduced as the loss function to optimize our UIE model in the direction of perceptual metrics. Moreover, since the confidence map acquired by UIQAN can effectively reflect the sensitivity of human perceptual of local area in an UIE image, the confidence map is introduced to our UIEF to help our UIEF to perceive the perceptually important regions. Thus, the confidence map is down-sampled and then concatenated into the decoder module of the UIE model, which can further improve the perceptual quality of the UIE results. Extensive experimental results show that the proposed HPQ-UIEF outperforms state-of-the-art UIE methods qualitatively and quantitatively.

Efficient underwater visual environment perception is the key to realizing the autonomous operation of underwater robots. Because of the complex and diverse underwater environment, the underwater images not only have different degrees of color cast but also produce a lot of noise. Due to the existence of noise in the underwater image and the blocking effect in the process of enhancing the image, the enhanced underwater image is still rough. Therefore, an underwater color-cast image enhancement method based on noise suppression and block effect elimination is proposed in this paper. Firstly, an automatic white balance algorithm for brightness and color balance is designed to correct the color deviation of underwater images and effectively restore the brightness and color of underwater images. Secondly, aiming at the problem of a large amount of noise in underwater images, a noise suppression algorithm for heat conduction matrix in the wavelet domain is proposed, which suppresses image noise and improves the contrast and edge detail information of underwater images. Thirdly, for the block effect existing in the process of enhancing the underwater color-cast image, a block effect elimination algorithm based on compressed domain boundary average is proposed, which eliminates the block effect in the enhancement process and balances the bright area and dark area in the image. Lastly, multi-scale image fusion is performed on the images after color correction, noise suppression, and block effect elimination, and finally, the underwater enhanced image with rich features is obtained. The results show that the proposed method is superior to other algorithms in color correction, contrast, and visibility. It also shows that the proposed method corrects the underwater color-cast image to a certain extent and effectively suppresses the noise and block effect of the underwater image, which provides theoretical support for underwater visual environment perception technology.

Self-organized underwater image enhancement

Underwater image co-enhancement based on physical-guided transformer interaction

In this article, to exclusively suppress unuseful underwater noise feature and effectively avoid overenhancement, simultaneously, an underwater attentional generative adversarial network (UAGAN) is innovatively established. Main contributions are as follows: combining dense concatenation with global maximum and average pooling techniques, a cascade dense-channel attention (CDCA) module is devised to adaptively distinguish noise feature and recalibrate channel weight, simultaneously, such that low-contribution feature map can be effectively suppressed; to sufficiently capture long-range dependence between any two nonlocal spatial patches, the position attention (PA) module is created such that the deviation among independent patches can be sufficiently eliminated, thereby avoiding overenhancement; and in conjunction with CDCA and PA modules, the entire UAGAN framework is eventually developed in an end-to-end manner. Comprehensive experiments conducted on underwater image enhancement benchmark (UIEB) and underwater robot professional contest (URPC) datasets demonstrate remarkable effectiveness and superiority of the proposed UAGAN scheme by comparing with typical underwater image enhancement approaches including unsupervised color correction method, image blurriness and light absorption, underwater dark channel prior, underwater generative adversarial network, underwater convolutional neural network, and WaterNet in terms of peak signal-to-noise ratio, underwater color image quality evaluation, underwater image quality measures, etc.