Underwater Image Enhancement Method Research Articles

Underwater image enhancement method via extreme enhancement and ultimate weakening

Underwater image enhancement method via extreme enhancement and ultimate weakening

PAFPT: Progressive aggregator with feature prompted transformer for underwater image enhancement

The optical characteristics of underwater environments often result in distorted underwater images, which makes it difficult to meet the needs of complex underwater environment perception. The demand for obtaining high-quality underwater images presents challenges for existing underwater image enhancement methods. In this paper, a progressive aggregator with feature prompted transformer for underwater image enhancement is presented. First, a feature prompted transformer module consisting of a feature convolutional transposed attention and a prompted feedforward neural network is proposed. It integrates feature attention mechanisms and adjusts input features through prompt weights to better focus on global information while enhancing local details, enriching texture, removing blurring, and correcting color deviation. Concurrently, a progressive aggregator consisting of four stages of up-sampling aggregation blocks is constructed to transfer features effectively and facilitate the aggregation of features of different scales at different stages to realize comprehensive attention to image details and semantic information. Besides, a region reconstruction unit is designed, and a reconstruction attention mechanism is introduced to pay attention to the bottleneck localized region to suppress noise. The performance of our method is thoroughly evaluated on public underwater image datasets: EUVP, UIEB, RUIE and SQUID. The quantitative and qualitative results indicate that our method outperforms nine state-of-the-art methods in both subjective perception and evaluation metrics, demonstrating excellent learning and generalization abilities. Additionally, the excellent results reflect the significant performance gains it brings to downstream visual applications.

Enhancing Underwater Images through Multi-Frequency Detail Optimization and Adaptive Color Correction

This paper presents a novel underwater image enhancement method addressing the challenges of low contrast, color distortion, and detail loss prevalent in underwater photography. Unlike existing methods that may introduce color bias or blur during enhancement, our approach leverages a two-pronged strategy. First, an Efficient Fusion Edge Detection (EFED) module preserves crucial edge information, ensuring detail clarity even in challenging turbidity and illumination conditions. Second, a Multi-scale Color Parallel Frequency-division Attention (MCPFA) module integrates multi-color space data with edge information. This module dynamically weights features based on their frequency domain positions, prioritizing high-frequency details and areas affected by light attenuation. Our method further incorporates a dual multi-color space structural loss function, optimizing the performance of the network across RGB, Lab, and HSV color spaces. This approach enhances structural alignment and minimizes color distortion, edge artifacts, and detail loss often observed in existing techniques. Comprehensive quantitative and qualitative evaluations using both full-reference and no-reference image quality metrics demonstrate that our proposed method effectively suppresses scattering noise, corrects color deviations, and significantly enhances image details. In terms of objective evaluation metrics, our method achieves the best performance in the test dataset of EUVP with a PSNR of 23.45, SSIM of 0.821, and UIQM of 3.211, indicating that it outperforms state-of-the-art methods in improving image quality.

CCM-Net: Color compensation and coordinate attention guided underwater image enhancement with multi-scale feature aggregation

Due to the light scattering and wavelength absorption in water, underwater images exhibit blurred details, low contrast, and color deviation. Existing underwater image enhancement methods are divided into traditional methods and deep learning-based methods. Traditional methods either rely on scene prior and lack robustness, or are not flexible enough resulting in poor enhancement effects. Deep learning methods have achieved good results in the field of underwater image enhancement due to their powerful feature representation ability. However, these methods cannot enhance underwater images with various degradations because they do not consider the inconsistent attenuation of different color channels and spatial regions. In this paper, we propose a novel asymmetric encoder-decoder network for underwater image enhancement, called CCM-Net. Concretely, we first introduce the prior knowledge-based encoder, which includes color compensation (CC) modules and feature extraction modules that consist of depth-wise separable convolution and global-local coordinate attention (GLCA). Then, we design a multi-scale feature aggregation (MFA) module to integrate shallow, middle, and deep features. Finally, we deploy a decoder to reconstruct the underwater images with the extracted features. Extensive experiments on publicly available datasets demonstrate that our CCM-Net effectively improves the visual quality of underwater images and achieves impressive performance.

An underwater image enhancement method based on multi-scale layer decomposition and fusion

High-quality underwater images can intuitively reflect the most realistic underwater conditions, guiding for underwater environmental monitoring and resource exploration strongly. But when factors like light absorption affect underwater optical imaging, it has been found that poor visibility and blurred texture details occur in acquired images, posing challenges for the identification and detection of underwater targets. To obtain natural images, an enhancement algorithm is proposed based on multi-scale layer decomposition and fusion. The algorithm employs different strategies to recover image attenuation information from both local and global perspectives, generating two complementary preprocessed fusion inputs. For fusion input 1, operations are conducted in the RGB color space. Initially, the mean proportion of each color channel is used to identify the attenuated color channel. Then, a local compensation strategy is adaptively applied to restore the pixel intensity of the attenuated color channel. Finally, a statistical color correction method is used to eliminate color cast in the image. Fusion input 2 involves two processing stages. In the Lab color space, the algorithm uses the grayscale information to reduce the deviation in the mean values of channels a and b globally. The local mean information of the component L enhances detail textures. In the RGB color space, linear stretching is applied to correct color deviations. To fuse the structural features of two complementary preprocessed inputs and avoid interference between signals from different layers, the color channels of fused input image are first decomposed into muti-scale structural layers based on structural priors. Then, the image enhancement is achieved through layer-by-layer fusion of the corresponding color channels of the two inputs. By testing and analyzing with the, it was found that the proposed method can improve the clarity of attenuated images in various underwater scenarios of UIEBD and RUIE datasets effectively, enhancing image detail and texture richness, increases contrast, and achieving natural and comfortable visual quality. Compared with the quantitative metrics of 14 other algorithms, the proposed algorithm shows an average score improvement of 10.14, 90.48, and 2.06, respectively, in metrics AG (average gradient), EI (edge intensity), and NIQE (natural image quality evaluator). In the RUIE dataset, it shows an average score improvement of 10.21, 94.76, and 1.86, respectively.

Enhancement of underwater dam crack images using multi-feature fusion

Underwater dam-crack images captured by remotely operated vehicles (ROVs) often exhibit blurred and unclear features because of the absorption and scattering of water and device shaking. To address these challenges, we propose an underwater crack image-enhancement network (UCE-CycleGAN) that outperforms existing methods on unpaired crack datasets. This network employs multi-feature fusion, incorporating edge maps and dark channels to mitigate blurring and enhance texture details. The blurriness metric and number of matched feature points in the enhanced images improved by 4.12 and 17.11 times, respectively. The mean average precision for detection and segmentation increased by 58.5 % and 39.0 %, respectively, surpassing those of other underwater image-enhancement methods. Furthermore, by integrating YOLOv8 feature recognition and a semantic segmentation network, we developed a real-time automated system (UCADS) for underwater dam-crack detection, thereby expanding the applications of enhanced underwater crack images. The effectiveness of UCADS was validated through field experiments at reservoir dams.

NUAM-Net: A Novel Underwater Image Enhancement Attention Mechanism Network

Vision-based underwater exploration is crucial for marine research. However, the degradation of underwater images due to light attenuation and scattering poses a significant challenge. This results in the poor visual quality of underwater images and impedes the development of vision-based underwater exploration systems. Recent popular learning-based Underwater Image Enhancement (UIE) methods address this challenge by training enhancement networks with annotated image pairs, where the label image is manually selected from the reference images of existing UIE methods since the groundtruth of underwater images do not exist. Nevertheless, these methods encounter uncertainty issues stemming from ambiguous multiple-candidate references. Moreover, they often suffer from local perception and color perception limitations, which hinder the effective mitigation of wide-range underwater degradation. This paper proposes a novel NUAM-Net (Novel Underwater Image Enhancement Attention Mechanism Network) that addresses these limitations. NUAM-Net leverages a probabilistic training framework, measuring enhancement uncertainty to learn the UIE mapping from a set of ambiguous reference images. By extracting features from both the RGB and LAB color spaces, our method fully exploits the fine-grained color degradation clues of underwater images. Additionally, we enhance underwater feature extraction by incorporating a novel Adaptive Underwater Image Enhancement Module (AUEM) that incorporates both local and long-range receptive fields. Experimental results on the well-known UIEBD benchmark demonstrate that our method significantly outperforms popular UIE methods in terms of PSNR while maintaining a favorable Mean Opinion Score. The ablation study also validates the effectiveness of our proposed method.

Self-organized underwater image enhancement

Underwater images captured in diverse underwater scenes exhibit varying types and degrees of degradation, including color deviations, low contrast, blurry details, etc. Single image enhancement methods tend to insufficiently address the diverse degradation issues, resulting in inappropriate results that do not align well with human visual perception or underwater color prior. To overcome these deficiencies, we develop a novel reinforcement learning framework that selects a sequence of image enhancement methods and configures their parameters in a self-organized manner for the purpose of underwater image enhancement. In contrast to end-to-end deep learning-based black-box mechanisms, the novel framework operates in a white-box fashion where the mechanisms for the method selection and parameter configuration are transparent. Furthermore, our framework incorporates the human visual perception and the underwater color prior into non-reference score increments for rewarding the underwater image enhancement. This breaks through the training limit imposed by volunteer-selected enhanced images as references. Comprehensive qualitative and quantitative experiments ultimately demonstrate that our framework outperforms nine state-of-the-art underwater image enhancement methods in terms of visual quality, and achieves better performance in five underwater image quality assessment metrics on three underwater image datasets. We release our code at https://gitee.com/wanghaoupc/Self_organized_UIE.

Scientific mapping and bibliometric analysis of research advancements in underwater image enhancement

Underwater Image Enhancement (UIE) addresses challenges in marine resource exploitation such as absorption, noise, and low contrast. This paper employs bibliometric methods and data from Web of Science to analyze UIE literature. The analysis reveals that research on UIE entered a rapid development stage starting from 2017. Three main areas of technical support for research: underwater image enhancement methods, deep learning-based methods, and underwater image restoration and depth estimation. Hotspots in research include enhancement, deep learning, restoration, feature extraction, and quality assessment. Drawing from the bibliometric analysis, this paper proposes a conceptual framework for UIE research and presents five research suggestions: enhancing method robustness and adaptability, improving real-time performance, integrating software and hardware, establishing underwater image benchmark datasets, and refining quality evaluation systems for underwater images. This study offers valuable references and guidance for the future exploration and development of the UIE field.

Convolution-transformer blend pyramid network for underwater image enhancement

Underwater images suffer from different types of degradation, where color degradation occurs in the spatial domain and edge degradation in the frequency domain. The high-quality underwater image enhancement represents a crucial milestone in advancing computer vision systems tailored for marine environments. This foundational endeavor encompasses a wide array of applications in computer vision tasks, including underwater inspection, underwater archaeology, and environmental monitoring. However, current convolutional neural network (CNN)-based pyramid frameworks primarily focus on capturing local features, often overlooking the significance of global semantic features that play a crucial role in understanding underwater scenes. Moreover, these frameworks handle spatial and frequency features independently, failing to enhance images by exploring correlation among domain-specific attributes for enabling information consistency. Besides, optimizing the model using a loss function with the same domain attributes from the ground truth may not lead to a better generalization ability. To solve these problems, we propose a new Convolution-Transformer Blend Pyramid Network (CTPN), which consists of a spatial branch and several frequency branches. The CTPN has four key components: a Swin transformer encoder, a CNN-Transformer aggregated encoder–decoder (CTED) and a blend pyramid framework. The Swin transformer encoder is employed to capture global semantic features, benefiting from its ability to extract long-range and global dependencies among features. The CTED fuses local features captured by CNN layers and global semantic features captured by the Swin transformer encoder in the spatial branch, with the help of the Cross-Model Fusion Module (CFM) and Skip-Aggregation Module (SAM). Subsequently, a blend pyramid framework is designed which not only progressively expands the transformed information of the previous domain branch to the current domain branch via the CTED-based refining operation, but also utilizes the proposed Domain Affinity Block (DAB) to explore the connection between domain attributes, ensuring information consistency. The experimental results demonstrate that the proposed method outperforms existing underwater image enhancement methods quantitatively and qualitatively.

An effective transformer based on dual attention fusion for underwater image enhancement

Underwater images suffer from color shift, low contrast, and blurred details as a result of the absorption and scattering of light in the water. Degraded quality images can significantly interfere with underwater vision tasks. The existing data-driven based underwater image enhancement methods fail to sufficiently consider the impact related to the inconsistent attenuation of spatial areas and the degradation of color channel information. In addition, the dataset used for model training is small in scale and monotonous in the scene. Therefore, our approach solves the problem from two aspects of the network architecture design and the training dataset. We proposed a fusion attention block that integrate the non-local modeling ability of the Swin Transformer block into the local modeling ability of the residual convolution layer. Importantly, it can adaptively fuse non-local and local features carrying channel attention. Moreover, we synthesize underwater images with multiple water body types and different degradations using the underwater imaging model and adjusting the degradation parameters. There are also perceptual loss functions introduced to improve image vision. Experiments on synthetic and real-world underwater images have shown that our method is superior. Thus, our network is suitable for practical applications.

Underwater Organism Color Fine-Tuning via Decomposition and Guidance

Due to the wavelength dependent light attenuation and scattering, the color of the underwater organism usually appears distorted. The existing underwater image enhancement methods mainly focus on designing networks capable of generating enhanced underwater organisms with fixed color. Due to the complexity of the underwater environment, ground truth labels are difficult to obtain, which results in the non-existence of perfect enhancement effects. Different from the existing methods, this paper proposes an algorithm with color enhancement and color fine-tuning (CECF) capabilities. The color enhancement behavior of CECF is the same as that of existing methods, aiming to restore the color of the distorted underwater organism. Beyond this general purpose, the color fine-tuning behavior of CECF can adjust the color of organisms in a controlled manner, which can generate enhanced organisms with diverse colors. To achieve this purpose, four processes are used in CECF. A supervised enhancement process learns the mapping from a distorted image to an enhanced image by the decomposition of color code. A self reconstruction process and a cross-reconstruction process are used for content-invariant learning. A color fine-tuning process is designed based on the guidance for obtaining various enhanced results with different colors. Experimental results have proven the enhancement ability and color fine-tuning ability of the proposed CECF. The source code is provided in https://github.com/Xiaofeng-life/CECF.

Underwater image enhancement using multi-task fusion.

Underwater images are often scattered due to suspended particles in the water, resulting in light scattering and blocking and reduced visibility and contrast. Color shifts and distortions are also caused by the absorption of different wavelengths of light in the water. This series of problems will make the underwater image quality greatly impaired, resulting in some advanced visual work can not be carried out underwater. In order to solve these problems, this paper proposes an underwater image enhancement method based on multi-task fusion, called MTF. Specifically, we first use linear constraints on the input image to achieve color correction based on the gray world assumption. The corrected image is then used to achieve visibility enhancement using an improved type-II fuzzy set-based algorithm, while the image is contrast enhanced using standard normal distribution probability density function and softplus function. However, in order to obtain more qualitative results, we propose multi-task fusion, in which we solve for similarity, then we obtain fusion weights that guarantee the best features of the image as much as possible from the obtained similarity, and finally we fuse the image with the weights to obtain the output image, and we find that multi-task fusion has excellent image enhancement and restoration capabilities, and also produces visually pleasing results. Extensive qualitative and quantitative evaluations show that MTF method achieves optimal results compared to ten state-of-the-art underwater enhancement algorithms on 2 datasets. Moreover, the method can achieve better results in application tests such as target detection and edge detection.

WaterPairs: a paired dataset for underwater image enhancement and underwater object detection

Due to its importance in marine engineering and aquatic robotics, underwater image enhancement works as a preprocessing step to improve the performance of high-level vision tasks such as underwater object detection and recognition. Although several studies exhibit that underwater image enhancement algorithms can boost the detection accuracy of detectors, no work has focused on studying the relationship between these two tasks. This is mainly because current underwater datasets lack either bounding box annotations or high-quality reference images, based on which detection accuracy or image quality assessment metrics are calculated. To examine how underwater image enhancement methods affect underwater object detection tasks, we provide a large-scale underwater object detection dataset with both bounding box annotations and high-quality reference images, namely, the WaterPairs dataset. The WaterPairs dataset offers a platform for researchers to comprehensively study the influence of underwater image enhancement algorithms on underwater object detection tasks. We will release our dataset at https://github.com/IanDragon/WaterPairs once this paper is accepted.

An in-situ image enhancement method for the detection of marine organisms by remotely operated vehicles

Abstract With the assistance of the visual system, remote operated vehicles (ROVs) can replace frogmen to achieve safer and more efficient capturing of marine organisms. However, the selective absorption and scattering of light lead to a decrease in the visual quality of underwater images, which hinders ROV operators from observing the operating environment. Unfortunately, most image enhancement methods only focus on image color correction rather than perceptual enhancement, which in turn prevents the object detector from quickly locating the target. Therefore, a visual-enhanced and detection-friendly underwater image enhancement method is needed. In this paper, an underwater image enhancement method called in-situ enhancement is proposed to improve the semantic information of the visual hierarchy based on current scene information in multiple stages. Mapping the underwater image to its dual space allows the enhancement equation to be applied to severely degraded underwater scenes. Moreover, it is also a detection-friendly method and has good generalization in both visual quality improvement and object detection. The experimental results show that in different underwater datasets, the in-situ enhancement effectively improves the visual quality of underwater images, and its enhanced results train different object detectors with high detection accuracy.

Underwater image enhancement method based on a cross attention mechanism

Underwater image enhancement method based on a cross attention mechanism

A real-world underwater image enhancement method based on multi-color space and two-stage adaptive fusion

A real-world underwater image enhancement method based on multi-color space and two-stage adaptive fusion

A gated cross-domain collaborative network for underwater object detection

Underwater object detection (UOD) plays a significant role in aquaculture and marine environmental protection. Considering the challenges posed by low contrast and low-light conditions in underwater environments, several underwater image enhancement (UIE) methods have been proposed to improve the quality of underwater images. However, only using the enhanced images does not improve the performance of UOD, since it may unavoidably remove or alter critical patterns and details of underwater objects. In contrast, we believe that exploring the complementary information from the two domains is beneficial for UOD. The raw image preserves the natural characteristics of the scene and texture information of the objects, while the enhanced image improves the visibility of underwater objects. Based on this perspective, we propose a Gated Cross-domain Collaborative Network (GCC-Net) to address the challenges of poor visibility and low contrast in underwater environments, which comprises three dedicated components. Firstly, a real-time UIE method is employed to generate enhanced images, which can improve the visibility of objects in low-contrast areas. Secondly, a cross-domain feature interaction module is introduced to facilitate the interaction and mine complementary information between raw and enhanced image features. Thirdly, to prevent the contamination of unreliable generated results, a gated feature fusion module is proposed to adaptively control the fusion ratio of cross-domain information. Our method presents a new UOD paradigm from the perspective of cross-domain information interaction and fusion. Experimental results demonstrate that the proposed GCC-Net achieves state-of-the-art performance on four underwater datasets.

A Multi-scale feature modulation network for efficient underwater image enhancement

Many strategies have been proposed for underwater image enhancement. However, they are often incompatible with underwater remotely operated vehicle platforms with low memory and computational power. In this paper, we address this problem by proposing a simple yet effective multi-scale feature modulation network to achieve a better trade-off between the efficiency of the model and the performance of the reconstruction. Specifically, we develop a multi-scale modulation module on top of a Visual Transformer (VIT)-like block. Within this module, we extract the features of the input image using a Multi-scale Spatial Feature Module (MSFM) to dynamically select representative features in the feature space of the image. Since the MSFM block handles the image features from a spatial point of view of the image, we further introduce the channel mixing module to perform the channel mixing. Extensive experimental results show that the proposed method is 8.5 times smaller in terms of network parameters than the state-of-the-art underwater image enhancement methods (e.g., Semi-uir), and achieves essentially the same performance with less computational cost. The code is available at https://github.com/zhengshijian1993/MFMN.

Development and validation of BVB-UIE: an underwater image-enhancement model with two-layer deep neural network

Due to the absorption and scattering of light in water, underwater visual visibility is poor, and underwater images usually suffer from color distortion and feature loss, which poses a great challenge for underwater vision tasks. To handle these issues, we present an underwater image enhancement (UIE) method. A Gaussian pyramid is constructed for the degraded underwater images based on an improved visual saliency model, with the characteristics of luminance, orientation, texture, and color. By combining channel and spatial attention mechanisms, a deep asymmetric feature enhancement network is built to preserve the high-dimensional features of the image. In addition, a polynomial loss function is used for the depth hopping supervising constraints during the enhancement process, and the gating signals are employed to control the importance of the features at different spatial locations. These innovations maximize the efficiency of the feature extraction while simplifying the network complexity. Experiments on an open benchmark dataset for UIE show that our method effectively eliminates the color bias and contrast distortion problems while preserving the complex image details compared to the existing UIE algorithms. Objective metrics show a significant improvement in the algorithm, with a 15% increment in the peak signal-to-noise ratio metric compared to the closest competing algorithm.