Underwater Datasets Research Articles

DBFNet: A Dual-Branch Fusion Network for Underwater Image Enhancement

Due to the absorption and scattering effects of light propagating through water, underwater images inevitably suffer from severe degradation, such as color casts and losses of detail. Many existing deep learning-based methods have demonstrated superior performance for underwater image enhancement (UIE). However, accurate color correction and detail restoration still present considerable challenges for UIE. In this work, we develop a dual-branch fusion network, dubbed the DBFNet, to eliminate the degradation of underwater images. We first design a triple-color channel separation learning branch (TCSLB), which balances the color distribution of underwater images by learning the independent features of the different channels of the RGB color space. Subsequently, we develop a wavelet domain learning branch (WDLB) and design a discrete wavelet transform-based attention residual dense module to fully employ the wavelet domain information of the image to restore clear details. Finally, a dual attention-based selective fusion module (DASFM) is designed for the adaptive fusion of latent features of the two branches, in which both pleasing colors and diverse details are integrated. Extensive quantitative and qualitative evaluations of synthetic and real-world underwater datasets demonstrate that the proposed DBFNet significantly improves the visual quality and shows superior performance to the compared methods. Furthermore, the ablation experiments demonstrate the effectiveness of each component of the DBFNet.

Boosting R-CNN: Reweighting R-CNN samples by RPN’s error for underwater object detection

Complicated underwater environments bring new challenges to object detection, such as unbalanced light conditions, low contrast, occlusion, and mimicry of aquatic organisms. Under these circumstances, the objects captured by the underwater camera will become vague, and the generic detectors often fail on these vague objects. This work aims to solve the problem from two perspectives: uncertainty modeling and hard example mining. We propose a two-stage underwater detector named boosting R-CNN, which comprises three key components. First, a new region proposal network named RetinaRPN is proposed, which provides high-quality proposals and considers objectness and IoU prediction for uncertainty to model the object prior probability. Second, the probabilistic inference pipeline is introduced to combine the first-stage prior uncertainty and the second-stage classification score to model the final detection score. Finally, we propose a new hard example mining method named boosting reweighting. Specifically, when the region proposal network miscalculates the object prior probability for a sample, boosting reweighting will increase the classification loss of the sample in the R-CNN head during training, while reducing the loss of easy samples with accurately estimated priors. Thus, a robust detection head in the second stage can be obtained. During the inference stage, the R-CNN has the capability to rectify the error of the first stage to improve the performance. Comprehensive experiments on two underwater datasets and two generic object detection datasets demonstrate the effectiveness and robustness of our method. The link of code:https://github.com/mousecpn/Boosting-R-CNN.

Fast underwater image enhancement based on a generative adversarial framework

Underwater image enhancement is a fundamental requirement in the field of underwater vision. Along with the development of deep learning, underwater image enhancement has made remarkable progress. However, most deep learning-based enhancement methods are computationally expensive, restricting their application in real-time large-size underwater image processing. Furthermore, GAN-based methods tend to generate spatially inconsistent styles that decrease the enhanced image quality. We propose a novel efficiency model, FSpiral-GAN, based on a generative adversarial framework for large-size underwater image enhancement to solve these problems. We design our model with equal upsampling blocks (EUBs), equal downsampling blocks (EDBs) and lightweight residual channel attention blocks (RCABs), effectively simplifying the network structure and solving the spatial inconsistency problem. Enhancement experiments on many real underwater datasets demonstrate our model's advanced performance and improved efficiency.

Semantic Segmentation of Fish and Underwater Environments Using Deep Convolutional Neural Networks and Learned Active Contours

The conservation of marine resources requires constant monitoring of the underwater environment by researchers. For this purpose, visual automated monitoring systems are of great interest, especially those that can describe the environment using semantic segmentation based on deep learning. Although they have been successfully used in several applications, such as biomedical ones, obtaining optimal results in underwater environments is still a challenge due to the heterogeneity of water and lighting conditions, and the scarcity of labeled datasets. Even more, the existing deep learning techniques oriented to semantic segmentation only provide low resolution results, lacking the enough spatial details for a high performance monitoring. To address these challenges, a combined loss function based on the active contour theory and level set methods is proposed to refine the spatial segmentation resolution and quality. To evaluate the method, a new underwater dataset with pixel annotations for three classes (fish, seafloor, and water) was created using images from publicly accessible datasets like SUIM, RockFish, and DeepFish. The performance of architectures of convolutional neural networks (CNNs), such as UNet and DeepLabV3+, trained with different loss functions (cross entropy, dice, and active contours) was compared, finding that the proposed combined loss function improved the segmentation results by around 3%, both in the metric Intercept Over Union (IoU) as in Hausdorff Distance (HD).

U-shape Transformer for Underwater Image Enhancement.

The light absorption and scattering of underwater impurities lead to poor underwater imaging quality. The existing data-driven based underwater image enhancement (UIE) techniques suffer from the lack of a large-scale dataset containing various underwater scenes and high-fidelity reference images. Besides, the inconsistent attenuation in different color channels and space areas is not fully considered for boosted enhancement. In this work, we built a large scale underwater image (LSUI) dataset, which covers more abundant underwater scenes and better visual quality reference images than existing underwater datasets. The dataset contains 4279 real-world underwater image groups, in which each raw image's clear reference images, semantic segmentation map and medium transmission map are paired correspondingly. We also reported an U-shape Transformer network where the transformer model is for the first time introduced to the UIE task. The U-shape Transformer is integrated with a channel-wise multi-scale feature fusion transformer (CMSFFT) module and a spatial-wise global feature modeling transformer (SGFMT) module specially designed for UIE task, which reinforce the network's attention to the color channels and space areas with more serious attenuation. Meanwhile, in order to further improve the contrast and saturation, a novel loss function combining RGB, LAB and LCH color spaces is designed following the human vision principle. The extensive experiments on available datasets validate the state-of-the-art performance of the reported technique with more than 2dB superiority. The dataset and demo code are available on https://bianlab.github.io/.

A multi‐scale feature representation and interaction network for underwater object detection

AbstractCompared with natural images, underwater images are usually degraded with blur, scale variation, colour shift and texture distortion, which bring much challenge for computer vision tasks like object detection. In this case, generic object detection methods usually fail to achieve satisfactory performance. The main reason is considered that the current methods lack sufficient discriminativeness of feature representation for the degraded underwater images. A a novel multi‐scale feature representation and interaction network for underwater object detection is proposed, in which two core modules are elaborately designed to enhance the discriminativeness of feature representation for underwater images. The first is the Context Integration Module, which extracts rich context information from high‐level features and is integrated with the feature pyramid network to enhance the feature representation in a multi‐scale way. The second is the Dual‐refined Attention Interaction Module, which further enhances the feature representation by sufficient interactions between different levels of features both in channel and spatial domains based on attention mechanism. The proposed model is evaluated on four public underwater datasets. The experimental results compared with state‐of‐the‐art object detection methods show that the proposed model has leading performance, which verifies that it is effective for underwater object detection. In addition, object detection experiments on a foggy dataset of Real‐world Task‐driven Testing Set (RTTS) and the natural image dataset of pattern analysis statistical modelling and computational learning, visual object classes (PASCAL VOC) are conducted. The results show that the proposed model can be applied on the degraded dataset of RTTS but fails on PASCAL VOC.

High Speed and Precision Underwater Biological Detection Based on the Improved YOLOV4-Tiny Algorithm

Realizing high-precision real-time underwater detection has been a pressing issue for intelligent underwater robots in recent years. Poor quality of underwater datasets leads to low accuracy of detection models. To handle this problem, an improved YOLOV4-Tiny algorithm is proposed. The CSPrestblock_body in YOLOV4-Tiny is replaced with Ghostblock_body, which is stacked by Ghost modules in the CSPDarknet53-Tiny backbone network to reduce the computation complexity. The convolutional block attention module (CBAM) is integrated to the algorithm in order to find the attention region in scenarios with dense objects. Then, underwater data is effectively improved by combining the Instance-Balanced Augmentation, underwater image restoration, and Mosaic algorithm. Finally, experiments demonstrate that the YOLOV4-Tinier has a mean Average Precision (mAP) of 80.77% on the improved underwater dataset and a detection speed of 86.96 fps. Additionally, compared to the baseline model YOLOV4-Tiny, YOLOV4-Tinier reduces about model size by about 29%, which is encouraging and competitive.

Toward underwater image enhancement: new dataset and white balance priors-based fusion network

Raw underwater images are degraded due to light attenuation and particle scattering. It is challenging to obtain authentic reference images. Hence, existing underwater datasets may be short of paired and trainable images. Many data-driven approaches are therefore difficult to implement. To address the above issues, we mainly make two contributions. First, a paired artificial underwater dataset (i.e., PAU-Wild) is constructed, including more than 3500 pairs of images of seven types. PAU-Wild is the first attempt to simulate real-world underwater images with external filters. Second, to enhance underwater images, a convolutional neural network using multistage fusion is proposed. Our model introduces two white balance (WB) priors into the fusion strategy. In addition, a joint loss function is designed to preserve content features and edge features. The experimental results demonstrate that our method is effective and even outperforms several state-of-the-art methods.

Systematic review and analysis on underwater image enhancement methods, datasets, and evaluation metrics

Underwater environments can be used to explore new resources that can be employed in the fields of medical science and energy resources. Humans are dependent on the valuable resources that exist beneath Earth’s surface. Underwater exploration requires enhanced images that are obtained using enhancement methods. So, it is important that underwater image enhancement (UIE) methods work well in terms of performance and accuracy. As a result, research in UIE has increased in the past few years. An extensive survey is conducted on existing UIE methods along with their broad classification, underwater datasets, and evaluation metrics, respectively. The experimental analysis is conducted to compare the existing UIE methods in terms of qualitative and quantitative evaluation metrics. The real-world applications and future scope of existing enhancement methods are highlighted and discussed.

Research on Underwater Complex Scene SLAM Algorithm Based on Image Enhancement.

Underwater images typically suffer from less explicit feature point information and more redundant information due to wild conditions. To solve these degradation problems, we propose the VINS-MONO algorithm to enhance the quality of the underwater image. Specifically, we first used the FAST feature point extraction algorithm to improve the extraction speed. Then, the inverse optical flow method was used to improve the accuracy of feature extraction. At the same time, several kinds of residual information were extracted and marginalized, separately, in the marginalization part of the back-end, in order to improve the marginalization speed. Extensive experiments on underwater dataset HAUD-Dataset and public dataset EuRoC show that our approach is superior to the original VINS-MONO algorithm. In addition, the original algorithm optimizes the situation in which the feature point information is not obvious, and the redundant information is more complex in the underwater environment, which effectively improves the visual quality of the underwater image.

Self-supervised acoustic representation learning via acoustic-embedding memory unit modified space autoencoder for underwater target recognition.

Since the expensive annotation of high-quality signals obtained from passive sonars and the weak generalization ability of the single feature in the ocean, this paper proposes the self-supervised acoustic representation learning under acoustic-embedding memory unit modified space autoencoder (ASAE) and performs the underwater target recognition task. In the manner of the animal-like acoustic auditory system, the first step is to design a self-supervised representation learning method called space autoencoder (SAE) to merge Mel filter-bank (FBank) with the acoustic discrimination and gammatone filter-bank (GBank) with the anti-noise robustness into SAE spectrogram (SAE Spec). Meanwhile, due to poor high-level semantic information in SAE Spec, an acoustic-embedding memory unit (AEMU) is introduced as the strategy of adversarial enhancement. During the auxiliary task, more negative samples are joined in the improved contrastive loss function to obtain adversarial enhanced features called ASAE spectrogram (ASAE Spec). Ultimately, the comprehensive contrast experiments and ablation experiments on two underwater datasets show that ASAE Spec increases by more than 0.96% in accuracy, convergence rate, and anti-noise robustness of other mainstream acoustic features. The results prove the potential value of ASAE in practical applications.

Refraction calibration of housing parameters for a flat-port underwater camera

In underwater computer vision systems, the camera is usually housed in a watertight case with a glass interface between water and air, which causes the light rays to refract, affecting image formation geometrically. In this paper, we propose an approach to calibrate the housing parameters of a flat-port underwater camera for a reasonable underwater refractive camera model. Our approach is specifically designed to deal with the refraction effect caused by the light traveling in different mediums. In this camera model, the calibrated housing parameters include the distance between the camera and glass interface and the normal of the glass interface in the camera coordinate system. We employ an underwater binocular vision system to calibrate the housing parameters in two steps. First, the distance to the glass interface is estimated by capturing a checkerboard image and computing the distance from the camera center to a known flat plane. Then, the normal of the glass interface is calculated by a multiobjective optimization method using geometric constraints in the underwater binocular stereo system. Extensive experiments have been conducted on our underwater dataset and the results verify the effectiveness of the proposed scheme for underwater camera calibration and three-dimensional measurements.

Deep images enhancement for turbid underwater images based on unsupervised learning

In agriculture, aquaculture technologies such as precise feeding, fish identification and fishing based on underwater machine vision all rely on the analysis of underwater images. However, due to the scatting and attenuation of the illumination in the real-world underwater environment, turbid underwater images are inevitably degraded, limiting their applicability in many vision tasks. In this paper, we present an unsupervised deep learning framework, called Underwater Loop Enhancement Network (ULENet), to improve the quality of turbid underwater images. We first propose an underwater dataset construction scheme and construct the dataset on which the network proposed above is trained. The underwater dataset contains images of three different scenes: lake and reservoir scene data (no label), pool scene data (weakly correlated label), and laboratory scene data (strongly correlated label). Then we propose a loop enhancement structure that uses the approximate candidates as labels and improves the visual quality of the image through the iterative training process. We formulate a new underwater visual perception loss function that evaluates the perceptual image quality based on its color, contrast, saturation and clarity. During the training process, a more realistic, higher-contrast, and clearer underwater image is gradually generated. Qualitative and quantitative evaluations show that the proposed method can effectively enhance image clarity. Moreover, the enhanced images are applied to several vision tasks to achieve better results, such as edge detection, key point matching, fish target detection and saliency prediction etc.

Underwater self-supervised depth estimation

Accurate underwater depth estimation is a cornerstone of reaching autonomous underwater exploration. However, it is incredibly tricky due to the inherent attenuation character and heavy noise. Fortunately, the depth-changing trend and underwater light attenuation are closely correlated, providing powerful clues for underwater depth estimation. Rather than simulating the underwater attenuation through formulas, we propose an underwater self-supervised depth estimation neural network in our work. With the guidance of multiple constraints, which are meticulously designed based on the comprehensive analyses of underwater characters, this network can learn the depth-changing trend by itself from attenuation information in underwater monocular videos. Our detailed experiments on underwater datasets prove that the proposed framework can obtain accurate and fine-grained depth maps. We believe the work may provide an economical solution for underwater perception.

Virtual Underwater Datasets for Autonomous Inspections

Underwater Vehicles have become more sophisticated, driven by the off-shore sector and the scientific community’s rapid advancements in underwater operations. Notably, many underwater tasks, including the assessment of subsea infrastructure, are performed with the assistance of Autonomous Underwater Vehicles (AUVs). There have been recent breakthroughs in Artificial Intelligence (AI) and, notably, Deep Learning (DL) models and applications, which have widespread usage in a variety of fields, including aerial unmanned vehicles, autonomous car navigation, and other applications. However, they are not as prevalent in underwater applications due to the difficulty of obtaining underwater datasets for a specific application. In this sense, the current study utilises recent advancements in the area of DL to construct a bespoke dataset generated from photographs of items captured in a laboratory environment. Generative Adversarial Networks (GANs) were utilised to translate the laboratory object dataset into the underwater domain by combining the collected images with photographs containing the underwater environment. The findings demonstrated the feasibility of creating such a dataset, since the resulting images closely resembled the real underwater environment when compared with real-world underwater ship hull images. Therefore, the artificial datasets of the underwater environment can overcome the difficulties arising from the limited access to real-world underwater images and are used to enhance underwater operations through underwater object image classification and detection.

Underwater Accompanying Robot Based on SSDLite Gesture Recognition

Underwater robots are often used in marine exploration and development to assist divers in underwater tasks. However, the underwater robots on the market have some problems, such as only a single function of object detection or tracking, the use of traditional algorithms with low accuracy and robustness, and the lack of effective interaction with divers. To this end, we designed a type of gesture recognition based on interaction, using person tracking as an auxiliary means for an underwater accompanying robot (UAR). We train and test the SSDLite detection algorithm using the self-labeled underwater datasets, and combine the kernelized correlation filters (KCF) tracking algorithm with the “Active Control” target tracking rule to continuously track the underwater human body. Our experiments show that the use of underwater datasets and target tracking can effectively improve gesture recognition accuracy by 40–105%. In the outfield experiment, the performance of the algorithm was good. It achieved target tracking and gesture recognition at 29.4 FPS on Jetson Xavier NX, and the UAR made corresponding actions according to the diver gesture command.

A Robust Underwater Multiclass Fish-School Tracking Algorithm

State-of-the-art multiple-object tracking methods are frequently applied to people or vehicle tracking, but rarely involve underwater-object tracking. Compared with the processing in non-underwater photos or videos, underwater fish tracking is challenging due to variations in light conditions, water turbidity levels, shape deformations, and the similar appearances of fish. This article proposes a robust underwater fish-school tracking algorithm (FSTA). The FSTA is based on the tracking-by-detection paradigm. To solve the problem of low recognition accuracy in an underwater environment, we add an amendment detection module that uses prior knowledge to modify the detection result. Second, we introduce an underwater data association algorithm for aquatic non-rigid organisms that recombines representation and location information to refine the data matching process and improve the tracking results. The Resnet50-IBN network is used as a re-identification network to track fish. We introduce a triplet loss function based on a centroid to train the feature extraction network. The multiple-object tracking accuracy (MOTA) of the FSTA is 79.1% on the underwater dataset, which shows that it can achieve state-of-the-art performance in a complex real-world marine environment.

Deblurring Ghost Imaging Reconstruction Based on Underwater Dataset Generated by Few-Shot Learning.

Underwater ghost imaging based on deep learning can effectively reduce the influence of forward scattering and back scattering of water. With the help of data-driven methods, high-quality results can be reconstructed. However, the training of the underwater ghost imaging requires enormous paired underwater datasets, which are difficult to obtain directly. Although the Cycle-GAN method solves the problem to some extent, the blurring degree of the fuzzy class of the paired underwater datasets generated by Cycle-GAN is relatively unitary. To solve this problem, a few-shot underwater image generative network method is proposed. Utilizing the proposed few-shot learning image generative method, the generated paired underwater datasets are better than those obtained by the Cycle-GAN method, especially under the condition of few real underwater datasets. In addition, to reconstruct high-quality results, an underwater deblurring ghost imaging method is proposed. The reconstruction method consists of two parts: reconstruction and deblurring. The experimental and simulation results show that the proposed reconstruction method has better performance in deblurring at a low sampling rate, compared with existing underwater ghost imaging methods based on deep learning. The proposed reconstruction method can effectively increase the clarity degree of the underwater reconstruction target at a low sampling rate and promotes the further applications of underwater ghost imaging.

An Improved YOLO Algorithm for Fast and Accurate Underwater Object Detection

Due to the abundant natural resources of the underwater world, autonomous exploration using underwater robots has become an effective technological tool in recent years. Real-time object detection is critical when employing robots for independent underwater exploration. However, when a robot detects underwater, its computing power is usually limited, which makes it challenging to detect objects effectively. To solve this problem, this study presents a novel algorithm for underwater object detection based on YOLOv4-tiny to achieve better performance with less computational cost. First, a symmetrical bottleneck-type structure is introduced into the YOLOv4-tiny’s backbone network based on dilated convolution and 1 × 1 convolution. It captures contextual information in feature maps with reasonable computational cost and improves the mAP score by 8.74% compared to YOLOv4-tiny. Second, inspired by the convolutional block attention module, a symmetric FPN-Attention module is constructed by integrating the channel-attention module and the spatial-attention module. Features extracted by the backbone network can be fused more efficiently by the symmetric FPN-Attention module, achieving a performance improvement of 8.75% as measured by mAP score compared to YOLOv4-tiny. Finally, this work proposed the YOLO-UOD for underwater object detection through the fusion of the YOLOv4-tiny structure, symmetric FPN-Attention module, symmetric bottleneck-type dilated convolutional layers, and label smoothing training strategy. It can efficiently detect underwater objects in an embedded system environment with limited computing power. Experiments show that the proposed YOLO-UOD outperforms the baseline model on the Brackish underwater dataset, with a detection mAP of 87.88%, 10.5% higher than that of YOLOv4-tiny’s 77.38%, and the detection result exceeds YOLOv5s’s 83.05% and YOLOv5m’s 84.34%. YOLO-UOD is deployed on the embedded system Jetson Nano 2 GB with a detection speed of 9.24 FPS, which shows that it can detect effectively in scenarios with limited computing power.

A Survey on Visual Navigation and Positioning for Autonomous UUVs

Autonomous navigation and positioning are key to the successful performance of unmanned underwater vehicles (UUVs) in environmental monitoring, oceanographic mapping, and critical marine infrastructure inspections in the sea. Cameras have been at the center of attention as an underwater sensor due to the advantages of low costs and rich content information in high visibility ocean waters, especially in the fields of underwater target recognition, navigation, and positioning. This paper is not only a literature overview of the vision-based navigation and positioning of autonomous UUVs but also critically evaluates the methodologies which have been developed and that directly affect such UUVs. In this paper, the visual navigation and positioning algorithms are divided into two categories: geometry-based methods and deep learning-based. In this paper, the two types of SOTA methods are compared experimentally and quantitatively using a public underwater dataset and their potentials and shortcomings are analyzed, providing a panoramic theoretical reference and technical scheme comparison for UUV visual navigation and positioning research in the highly dynamic and three-dimensional ocean environments.