Real Underwater Images Research Articles

Underwater image restoration method based on Walsh–Hadamard transform and attenuation coefficient estimation

Abstract Underwater images often exhibit color distortion and low contrast due to the scattering and absorption of light as it travels through water. Changes in lighting conditions further complicate the restoration and enhancement of these images. Improving the quality of underwater images is crucial for advancements in fields such as marine biology research, underwater measurement, and environmental monitoring. This paper proposes an underwater image restoration method based on the Image Formation Model (IFM), utilizing the Walsh–Hadamard transform and attenuation coefficient estimation. Traditional methods rely on dark channel prior and maximum intensity prior to estimate background light (BL) and transmission maps (TMs), often performing poorly in various underwater environments. Our method uses image blur to estimate BL and depth maps and derives three-channel attenuation coefficients using the gray-world theory to obtain a more accurate TM. Experimental results on real underwater images show that our method effectively eliminates color deviation and contrast distortion while preserving image details, significantly outperforming other IFM-based restoration techniques. Compared to the closest competing algorithms, our method achieves better UIQM and UCIQE scores.

LUIE: Learnable physical model-guided underwater image enhancement with bi-directional unsupervised domain adaptation

Recently, learning-based underwater enhancement (UIE) methods have made considerable progress, significantly benefiting downstream tasks such as underwater semantic segmentation and underwater depth estimation. Most existing unsupervised UIE methods utilize the atmospheric image formation model to decompose underwater images into background color, transmission map, and scene radiance. However, they rely on simplified physical models for estimating the transmission map, over-simplifying its complex formation, which results in imprecise modeling of underwater scattering effects. Additionally, supervised UIE methods heavily depend on synthetic data or ground truth, leading to limited generalization capabilities due to the substantial domain gap presented in different underwater scenarios. To tackle these challenges, we propose a Learnable physical model-guided unsupervised domain adaptation framework for Underwater Image Enhancement, dubbed LUIE. LUIE learns to predict background light, depth, and scene radiance from an underwater image. We incorporate a learnable network to estimate the transmission map based on the predicted depth map. To minimize the inter-domain gap between synthetic and real underwater images, we introduce a bi-directional domain adaptation method that alternates the background light from each domain. Experimental results demonstrate the effectiveness of our proposed method compared to existing approaches, and high-level experiment results validate that our enhanced underwater results. Experiments in real-world settings on underwater ROVs platform with NVIDIA Jetson AGX Xavier further confirm the effectiveness and efficiency of our work.

Underwater image enhancement based on global features and prior distribution guided

Underwater images often suffer from substantial image blur and color distortion due to the variability of water conditions and the physical location of optical equipment, which significantly impacts the underwater intelligent system's environmental perception. Standard methods exhibit limited generalization capabilities, leading to considerable performance fluctuations when handling images with uncontrolled degradation. In this research, we leverage global features and the prior distribution of ground truth images to guide our enhancement model, introducing a novel conditional Variational Auto-Encoder-based model, named UWG-VAE, to address these challenges. UWG-VAE enhances model controllability by incorporating prior distribution information and classes of degraded styles into the decoder of the enhancement model. We assess the performance of UWG-VAE in underwater image enhancement tasks across four challenging real underwater image datasets, comparing it to state-of-the-art models. UWG-VAE demonstrates a substantial enhancement in visual quality, with notable improvements in UIQM, UCIQE, and URanker evaluation metrics when compared to existing state-of-the-art models.

RSUIGM: Realistic Synthetic Underwater Image Generation with Image Formation Model

In this paper, we propose to synthesize realistic underwater images with a novel image formation model, considering both downwelling depth and line of sight (LOS) distance as cue and call it as Realistic Synthetic Underwater Image Generation Model, RSUIGM. The light interaction in the ocean is a complex process and demands specific modeling of direct and backscattering phenomenon to capture the degradations. Most of the image formation models rely on complex radiative transfer models and in-situ measurements for synthesizing and restoration of underwater images. Typical image formation models consider only line of sight distance z and ignore downwelling depth d in the estimation of effect of direct light scattering. We derive the dependencies of downwelling irradiance in direct light estimation for generation of synthetic underwater images unlike state-of-the-art image formation models. We propose to incorporate the derived downwelling irradiance in estimation of direct light scattering for modeling the image formation process and generate realistic synthetic underwater images with the proposed RSUIGM, and name it as RSUIGM dataset . We demonstrate the effectiveness of the proposed RSUIGM by using RSUIGM dataset in training deep learning based restoration methods. We compare the quality of restored images with state-of-the-art methods using benchmark real underwater image datasets and achieve improved results. In addition, we validate the distribution of realistic synthetic underwater images versus real underwater images both qualitatively and quantitatively. The proposed RSUIGM dataset is available here.

Underwater image enhancement method based on denoising diffusion probabilistic model

Underwater images often suffer from severe distortion, which seriously affects the image quality and the application. Current underwater image enhancement methods have poor generalization capability and cannot be adapted to all types of underwater images. In recent years, the diffusion model based on the denoising diffusion probabilistic model (DDPM) has achieved excellent results in various fields of computer vision. Inspired by the DDPM, an underwater image enhancement method based on the DDPM (UW-DDPM) was proposed in this paper. The UW-DDPM trained on paired datasets and utilized two U-Net networks to complete image denoising as well as image distribution transformation, which effectively improved the quality of underwater images. By testing on real underwater image datasets, UW-DDPM achieved better improvement in visual effects and evaluation metrics than the existing model.

UIALN: Enhancement for Underwater Image With Artificial Light

Since underwater images are seriously degraded due to the attenuation of light, artificial light (AL) is often used to assist photography in underwater. However, the normal underwater imaging process is changed by the AL. It is observed that the AL source typically alters the light condition to a large extent, resulting in non-uniform illumination of images. In addition, the color distortion of the area affected by AL is little because the AL close to the object suffers little attenuation. However, most existing underwater image enhancement algorithms ignore this phenomenon. In their results, the areas affected by AL tend to be over-enhanced or over-exposed and even affect the overall enhancement effect. To this end, we propose a novel underwater image enhancement algorithm (UIALN) based on luminance correction and AL area color self-guided restoration. The underwater image is converted into the LAB color space, where the AL and pseudo-blur effect on the L channel are removed based on the luminance correction network, and the color casts on the AB channels is removed by the guidance of the AL area. Specifically, a luminance correction network is first designed based on the retinex decomposition to correct luminance, where the uneven luminance caused by AL is corrected in the illumination layer decomposed by the L channel because AL is usually white light. After that, the AL area is detected by the difference between before and after luminance correction. Second, an AL area self-guidance network is designed to assist the restoration of the color channels AB. The color restoration module utilizes the internal characteristics of the image, where the characteristics of the AL area are utilized as the prior to make the color easy to be restored. In addition, to facilitate the training and testing of the algorithm, a method of synthetic underwater images with AL is proposed based on underwater image imaging model, and a new underwater image dataset with artificial light (UIDWAL) is provided. Experimental results show that our UIALN outperforms the existing state-of-the-art approaches for the enhancement of both synthetic and real underwater images with AL.

AUIE–GAN: Adaptive Underwater Image Enhancement Based on Generative Adversarial Networks

Underwater optical imaging devices are often affected by the complex underwater environment and the characteristics of the water column, which leads to serious degradation and distortion of the images they capture. Deep learning-based underwater image enhancement (UIE) methods reduce the reliance on physical parameters in traditional methods and have powerful fitting capabilities, becoming a new baseline method for UIE tasks. However, the results of these methods often suffer from color distortion and lack of realism because they tend to have poor generalization and self-adaptation capabilities. Generating adversarial networks (GANs) provides a better fit and shows powerful capabilities on UIE tasks. Therefore, we designed a new network structure for the UIE task based on GANs. In this work, we changed the learning of the self-attention mechanism by introducing a trainable weight to balance the effect of the mechanism, improving the self-adaptive capability of the model. In addition, we designed a feature extractor based on residuals with multi-level residuals for better feature recovery. To further improve the performance of the generator, we proposed a dual path discriminator and a loss function with multiple weighted fusions to help model fitting in the frequency domain, improving image quality. We evaluated our method on the UIE task using challenging real underwater image datasets and a synthetic image dataset and compared it to state-of-the-art models. The method ensures increased enhancement quality, and the enhancement effect of the model for different styles of images is also relatively stable.

Autonomous boundary inspection of Posidonia oceanica meadows using an underwater robot

Monitoring provides important information for the planning and execution of marine environment preservation operations. Posidonia oceanica is one of the principal bioindicators in Mediterranean coastal areas and regular monitoring activities play a crucial role in its conservation. However, an efficient observation of vast areas colonised with P. oceanica is extremely challenging and it currently requires tedious and time consuming diving activities. Autonomous Underwater Vehicles (AUVs) endowed with optical sensors could represent a viable solution in carrying out visual inspection surveys. Nevertheless, AUVs are usually programmed to perform pre-defined trajectories, which are not effective for seagrass monitoring applications, as meadows may be fragmented and their contours may be irregular. This work proposes a framework based on machine learning and computer vision that enables an AUV equipped with a down-looking camera to autonomously inspect the boundary of P. oceanica meadows to obtain an initial estimate of the meadow size. The proposed boundary inspection solution is composed of three main modules: (1) an image segmentation relying on a Mask R-CNN model to recognise P. oceanica in underwater images, (2) a boundary tracking strategy that generates guidance references to track P. oceanica contours, (3) a loop closure detector fusing visual and navigation information to identify when a meadow boundary has been completely explored. The image segmentation model and the visual part of the loop closure detection module were validated on real underwater images. The overall inspection framework was tested in a realistic simulation environment, using mosaics obtained from real images to replicate the actual monitoring scenarios. The results show that the proposed solution enables the AUV to autonomously accomplish the boundary inspection task of P. oceanica meadows, therefore representing an effective tool towards the conservation and protection of marine environments.

Underwater image enhancement combining dual color space and contrast learning

With the continuous increase of human society’s demand for marine resources, underwater vision plays an increasingly important role in marine information acquisition. However, due to the absorption and scattering of light, underwater images have characteristics such as blurring, low contrast, and color distortion, which are not conducive to the acquisition of underwater image information. The rapid development of deep learning and powerful feature learning capabilities have been widely used in underwater image enhancement tasks, but current research still has problems such as loss of local details and oversaturation. Aiming at the above problems, this paper proposes an underwater image enhancement method combining dual color space and contrast learning, including RGB color space and Lab color space. The RGB color space is used for color correction and artifact removal, and the Lab color space is used to enhance saturation and detail texture. The contrastive learning of positive and negative samples makes the generated image closer to the clear underwater image and deviates from the real underwater image. Experimental results on real underwater datasets show that the proposed method achieves the best results on PSNR, SSIM and UIQM compared to other underwater image enhancement methods.

Extreme Underwater Image Compression Using Physical Priors

Underwater images (UWIs) require higher compression ratio than terrestrial images due to the limited bandwidth of underwater wireless acoustic channel. In many studies such as marine species, the foreground objects (FGOs) in UWIs need to be observed in detail, while the background only needs to be viewed in general. However, existing image compression methods achieve limited compression ratio and reconstruction quality, which cannot fulfill these practical applications since they do not consider the unique underwater physical priors. To overcome the limitation, we propose an underwater physical prior-based extreme compression network (PPECN) for UWIs compression, which includes an underwater physical prior-guided FGOs autoencoder (UPGAE) and a FGOs-assisted background generator (FG-BGGAN). Specifically, we design an underwater physical prior guidance structure that simulates the data flow in the underwater physical imaging process to adaptively adjust the distribution of received Gaussian features in the UPGAE to be more consistent with real UWIs. During the adjustment, some basic UWI properties can be reconstructed, which can improve the reconstruction quality and implicitly reduce bits through the end-to-end training. Furthermore, the background is generated from simple semantic map under the constraint of the perceptual consistency between background and FGOs, significantly saving coding bits and improving the perceptual quality of the generated background. Extensive experimental results on four underwater image datasets verify that, compared with state-of-the-art compression methods, our PPECN achieves both impressive improvement in the perceptual quality of the whole image and significant gain in the pixel fidelity of the FGOs at the similar low bitrate.

Two-Stage Underwater Image Restoration Algorithm Based on Physical Model and Causal Intervention

Underwater images often suffer from severe color degradation, haze and local blur, which are caused by the scattering and absorption effects of light in water. Firstly, to address the lack of annotations in underwater images, we propose a multi-degradation rate underwater image degradation model. Additionally, we use the generative adversarial network (GAN) to guide the underwater image degradation model and generate paired images that can be used for underwater image restoration (UIR) network training. Also, given the problems of poor restoration quality, serious loss of detail, and slow inference speed of existing deep learning algorithms, we construct an underwater image restoration network that can reference in real-time. Moreover, through causal interventions in the image generation process, spurious correlations between global features and detailed features are eliminated. As a result, the detail generation ability of the image is improved. Experiments on real underwater image datasets demonstrate that compared with existing methods, our proposed method more effectively solves the problem of underwater image degradation.

Underwater Image Restoration via Contrastive Learning and a Real-World Dataset

Underwater image restoration is of significant importance in unveiling the underwater world. Numerous techniques and algorithms have been developed in recent decades. However, due to fundamental difficulties associated with imaging/sensing, lighting, and refractive geometric distortions in capturing clear underwater images, no comprehensive evaluations have been conducted with regard to underwater image restoration. To address this gap, we constructed a large-scale real underwater image dataset, dubbed Heron Island Coral Reef Dataset (‘HICRD’), for the purpose of benchmarking existing methods and supporting the development of new deep-learning based methods. We employed an accurate water parameter (diffuse attenuation coefficient) to generate the reference images. There are 2000 reference restored images and 6003 original underwater images in the unpaired training set. Furthermore, we present a novel method for underwater image restoration based on an unsupervised image-to-image translation framework. Our proposed method leveraged contrastive learning and generative adversarial networks to maximize the mutual information between raw and restored images. Extensive experiments with comparisons to recent approaches further demonstrate the superiority of our proposed method. Our code and dataset are both publicly available.

Underwater enhancement based on a self-learning strategy and attention mechanism for high-intensity regions

Images acquired during underwater activities suffer from environmental properties of the water, such as turbidity and light attenuation. These phenomena cause color distortion, blurring, and contrast reduction. In addition, irregular ambient light distribution causes color channel unbalance and regions with high-intensity pixels. Recent works related to underwater image enhancement, and based on deep learning approaches, tackle the lack of paired datasets generating synthetic ground-truth. In this paper, we present a self-supervised learning methodology for underwater image enhancement based on deep learning that requires no paired datasets. The proposed method estimates the degradation present in underwater images. Besides, an autoencoder reconstructs this image, and its output image is degraded using the estimated degradation information. Therefore, the strategy replaces the output image with the degraded version in the loss function during the training phase. This procedure misleads the neural network that learns to compensate the additional degradation. As a result, the reconstructed image is an enhanced version of the input image. Also, the algorithm presents an attention module to reduce high-intensity areas generated in enhanced images by color channel unbalances and outlier regions. Furthermore, the proposed methodology requires no ground-truth. Besides, only real underwater images were used to train the neural network, and the results indicate the effectiveness of the method in terms of color preservation, color cast reduction, and contrast improvement.

Integrating deep learning and traditional image enhancement techniques for underwater image enhancement

Underwater images usually suffer from colour distortion, blur, and low contrast, which hinder the subsequent processing of underwater information. To address these problems, this paper proposes a novel approach for single underwater images enhancement by integrating data-driven deep learning and hand-crafted image enhancement techniques. First, a statistical analysis is made on the average deviation of each channel of input underwater images to that of its corresponding ground truths, and it is found that both the red channel and the green channel of an underwater image contribute to its colour distortion. Concretely, the red channel of an underwater image is usually seriously attenuated, and the green channel is usually over strengthened. Motivated by such an observation, an attention mechanism guided residual module for underwater image colour correction is proposed, where the colour of the red channel of the underwater image and that of the green channel is compensated in a different way, respectively. Coupled with an attention mechanism, the residual module can adaptively extract and integrate the most discriminative features for colour correction. For scene contrast enhancement and scene deblurring, the traditional image enhancement techniques such as CLAHE (contrast limited adaptive histogram equalization) and Gamma correction are coupled with a multi-scale convolutional neural network (MSCNN), where CLAHE and Gamma correction are used as complement to deal with the complex and changeable underwater imaging environment. Experiments on synthetic and real underwater images demonstrate that the proposed method performs favourably against the state-of-the-art underwater image enhancement methods.

Model-Based Underwater Image Simulation and Learning-Based Underwater Image Enhancement Method

Due to the absorption and scattering effects of light in water bodies and the non-uniformity and insufficiency of artificial illumination, underwater images often present various degradation problems, impacting their utility in underwater applications. In this paper, we propose a model-based underwater image simulation and learning-based underwater image enhancement method for coping with various degradation problems in underwater images. We first derive a simplified model for describing various degradation problems in underwater images, then propose a model-based image simulation method that can generate images with a wide range of parameter values. The proposed image simulation method also comes with an image-selection part, which helps to prune the simulation dataset so that it can serve as a training set for learning to enhance the targeted underwater images. Afterwards, we propose a convolutional neural network based on the encoder-decoder backbone to learn to enhance various underwater images from the simulated images. Experiments on simulated and real underwater images with different degradation problems demonstrate the effectiveness of the proposed underwater image simulation and enhancement method, and reveal the advantages of the proposed method in comparison with many state-of-the-art methods.

Underwater image restoration via backscatter pixel prior and color compensation

The use of underwater cameras instead of divers to enter complex underwater areas for real-time monitoring of fish, shrimp, and algae is significant to the aquaculture industry. However, underwater images are severely degraded due to light absorption and scattering, limiting the development of underwater computer vision and robot vision perception. To solve blurriness and color degradation issues, this paper developed a restoration method based on backscatter pixel prior and color cast removal from the physical point of view of underwater image degradation. The proposed method used only a single underwater image as an input to estimate various parameters accurately, such as depth map, backscatter map, and illuminant map. Specifically, a backscatter estimation algorithm based on a depth map was proposed to improve the contrast of underwater images. Then, an algorithm was designed to remove color deviation based on the illuminant map. In particular, a color compensation strategy was created that could completely eliminate red artifacts in underwater images that were generated by the strong attenuation of the red channel. We designed comparative experiments from multiple angles on different real underwater image datasets. Experiments showed that the proposed method improved the contrast and removed the color deviation of light absorption compared to several reported methods. Even on underwater images with severe attenuation, the proposed method showed a significant positive effectiveness and stability on color cast removal.

Human Perceptual Quality Driven Underwater Image Enhancement Framework

Underwater images suffer from severe color casts, low contrast, and blurriness, which greatly degrade the visibility and color fidelity of underwater images. Recently, numerous underwater image enhancement (UIE) algorithms have been proposed. Existing synthetic datasets-based deep learning methods employ synthetic datasets to train UIE models. However, there is a gap between synthetic datasets and real underwater images, leading to poor generalization of synthetic datasets-based UIE methods. Besides, existing real datasets-based deep learning methods largely focus on minimizing the mean squared reconstruction error between UIE results and corresponding ground-truth on the real datasets, but do not take human visual perception into account. Thus, although they achieve high PSNR between UIE results and corresponding ground-truth obtained by user study on the real datasets, they often achieve unsatisfactory perceptual quality. To address these problems, we propose a Human Perceptual Quality Driven Underwater Image Enhancement Framework (HPQ-UIEF) to achieve better results in human perceptual quality and maintain satisfactory PSNR, which is trained on a real underwater enhancement quality assessment database (UEQAB). Specifically, an Underwater Image Quality Assessment Network (UIQAN) for UIE images is first proposed to assist UIE task, in which a novel depth map prior spatial attention block (DPPAB) is embedded into UIQAN. The DPPAB can adaptively recalibrate the quality-aware feature maps and model human visual attention in a data-driven manner. Then, the UIE model is proposed, in which the UIQAN is introduced as the loss function to optimize our UIE model in the direction of perceptual metrics. Moreover, since the confidence map acquired by UIQAN can effectively reflect the sensitivity of human perceptual of local area in an UIE image, the confidence map is introduced to our UIEF to help our UIEF to perceive the perceptually important regions. Thus, the confidence map is down-sampled and then concatenated into the decoder module of the UIE model, which can further improve the perceptual quality of the UIE results. Extensive experimental results show that the proposed HPQ-UIEF outperforms state-of-the-art UIE methods qualitatively and quantitatively.

Underwater Image Enhancement Method Based on Feature Fusion Neural Network

Aiming at the problems of uneven illumination of underwater image caused by supplementary illumination in deep-sea and night waters, image noise, low contrast and color deviation caused by suspended particles in water, a new underwater image enhancement method under non-uniform illumination is proposed. The heterogeneous feature fusion module is designed to fuse different levels and different levels of features, so as to improve the overall perception ability of the network to detail information and semantic information. Secondly, a new feature attention mechanism is designed to improve the traditional channel attention mechanism, and the improved channel attention and pixel attention mechanism are added to the heterogeneous feature fusion process to strengthen the ability of the network to extract pixel features with different turbidity. Then, the dynamic feature enhancement module is designed to adaptively expand the receptive field to improve the adaptability of the network to the image distortion scene and the ability of model conversion, and strengthen the network’s learning of the region of interest. Finally, the color loss function is designed, and the absolute error loss and structural similarity loss are jointly minimized to correct the color deviation on the basis of maintaining the image texture. A multi-scale feature extraction module is designed to extract different levels of features at the beginning of the network, and the output results are obtained through the convolution layer with jump connection and the attention module. The experimental results on several data sets show that this method can have good results in processing synthetic underwater images and real underwater images, and can better restore the image color and texture details compared with the existing methods. It conforms to the characteristics of human vision, and the visual effect is better than the existing underwater image enhancement algorithms.

Underwater Image Enhancement via Physical-Feedback Adversarial Transfer Learning

This article proposes a domain adaptive learning framework based on physical model feedback for underwater image enhancement. Underwater image enhancement involves mapping from low-quality underwater images to their dewatered counterparts. Due to the lack of dewatered images as ground truth, most learning-based methods are trained using synthetic datasets. However, they usually ignored the domain gap between synthetic training data and real-world testing data, which seriously reduces the generalization ability of those models when testing on real underwater images. We solve the problem by embedding a domain adaptive mechanism in a learning framework to eliminate the domain gap. However, the basic formulation of a domain adaptive-based learning framework does not generate realistic images in color and details. Motivated by an observation that the estimated results should be consistent with the physical model of underwater imaging, we propose a physics constraint as a feedback controller so that it can guide the estimation of underwater image enhancement. Extensive experiments validate the superiority of the proposed framework.

Severely Degraded Underwater Image Enhancement with a Wavelet-based Network

Underwater images are important in marine science and ocean engineering fields owing to giving color information, low cost, and compact. Yet obtained underwater images are often degraded and restoring and enhancing wavelength selective signal attenuation of underwater images depending on complex underwater physical process is essential in practical application. While recently developed deep learning is a promising choice, constructing sufficiently large dataset covering whole real images is challenging, peculiar to underwater image processing. In order to supplement relatively small dataset, previous studies alternatively construct an artificial underwater image dataset based on a physical model or Generative Adversarial Network. Also, incorporating traditional signal processing methods into the network architecture has shown promising success, though enhancement of severely degraded underwater images remains to be a big issue. In this paper, we tackle underwater image enhancement based on an encoder-decoder based deep learning model incorporating discrete wavelet transform and whitening and coloring transform. We also construct a severely degraded real underwater image dataset. The presented model shows excellent results both qualitatively and quantitatively in the artificial and real image dataset. Constructed dataset is available at https://github.com/tkswalk/2022-IJACSA.