Underwater Datasets Research Articles

RSE-YOLOv8: An Algorithm for Underwater Biological Target Detection.

Underwater target detection is of great significance in underwater ecological assessment and resource development. To better protect the environment and optimize the development of underwater resources, we propose a new underwater target detection model with several innovations based on the YOLOv8 framework. Firstly, the SAConv convolutional operation is introduced to redesign C2f, the core module of YOLOv8, to enhance the network's feature extraction capability for targets of different scales. Secondly, we propose the RFESEConv convolution module instead of the conventional convolution operation in neural networks to cope with the degradation of image channel information in underwater images caused by light refraction and reflection. Finally, we propose an ESPPF module to further enhance the model's multi-scale feature extraction efficiency. Simultaneously, the overall parameters of the model are reduced. Compared to the baseline model, the proposed one demonstrates superior advantages when deployed on underwater devices with limited computational resources. The experimental results show that we have achieved significant detection accuracy on the underwater dataset, with an mAP@50 of 78% and an mAP@50:95 of 43.4%. Both indicators are 2.1% higher compared to the baseline models. Additionally, the proposed model demonstrates superior performance on other datasets, showcasing its strong generalization capability and robustness. This research provides new ideas and methods for underwater target detection and holds important application value.

A novel domain independent scene text localizer

Text localization across multiple domains is crucial for applications like autonomous driving and tracking marathon runners. This work introduces DIPCYT, a novel model that utilizes Domain Independent Partial Convolution and a Yolov5-based Transformer for text localization in scene images from various domains, including natural scenes, underwater, and drone images. Each domain presents unique challenges: underwater images suffer from poor quality and degradation, drone images suffer from tiny text and loss of shapes, and scene images suffer from arbitrarily oriented, shaped text. Additionally, license plates in drone images may not provide rich semantic information compared to other text types due to loss of contextual information between characters. To tackle these challenges, DIPCYT employs new partial convolution layers within Yolov5 and integrates Transformer detection heads with a novel Fourier Positional Convolutional Block Attention Module (FPCBAM). This approach leverages common text properties across domains, such as contextual (global) and spatial (local) relationships. Experimental results demonstrate that DIPCYT outperforms existing methods, achieving F-scores of 0.90, 0.90, 0.77, 0.85, 0.85, and 0.88 on Total-Text, ICDAR 2015, ICDAR 2019 MLT, CTW1500, Drone, and Underwater datasets, respectively.

Contextual embedding generation of underwater images using deep learning techniques

<p>This article delves into the cutting-edge realm of artificial intelligence, specifically focusing on its application in marine research via underwater image analysis. It introduces an innovative, integrated approach that combines object detection with image captioning tailored for the aquatic domain. Central to this approach is the advanced technique of image feature extraction, complemented by the strategic implementation of attention mechanisms within neural networks. These mechanisms are key in enhancing the precision and contextual understanding of underwater imagery. The efficacy of this method is underscored by extensive experiments on diverse underwater datasets. Results show notable improvements in detecting and describing complex underwater scenes, thereby providing invaluable insights for marine biologists, environmentalists, and the broader scientific community. This exploration marks a significant advancement in marine research, offering a new lens through which the underwater world can be understood and preserved.</p>

Transformer-based improved U-net for high-performance underwater polarization imaging

Polarization imaging plays an important role in underwater imaging technology, as it allows for the suppression of backscattering light and thus enhances the quality of underwater images. Furthermore, with advancements in computer hardware technology, deep learning (DL) has experienced rapid development in recent years. However, existing methods based on convolutional neural networks (CNN) have the limitations of local feature extraction and fail to full use of the salient target’s polarization features. In this paper, we propose a transformer-based improved U-Net (TIU-Net) to further improve the performance of underwater polarization imaging. The proposed TIU-Net leverages CNN and transformer module to extract short-range and long-range features respectively for underwater target reconstruction. Meanwhile, by entering multi-dimensional information, each of which has a different emphasis on the expression of target features, to enhance the performance and stability of the reconstruction model. Experimental results on our established polarization underwater dataset show the superiority of our proposed method for underwater imaging, achieving the efficient imaging and high-quality generalization imaging in highly turbid underwater environments.

Underwater small target detection under YOLOv8-LA model

In the realm of marine environmental engineering, the swift and accurate detection of underwater targets is of considerable significance. Recently, methods based on Convolutional Neural Networks (CNN) have been applied to enhance the detection of such targets. However, deep neural networks usually require a large number of parameters, resulting in slow processing speed. Meanwhile, existing methods present challenges in accurate detection when facing small and densely arranged underwater targets. To address these issues, we propose a new neural network model, YOLOv8-LA, for improving the detection performance of underwater targets. First, we design a Lightweight Efficient Partial Convolution (LEPC) module to optimize spatial feature extraction by selectively processing input channels to improve efficiency and significantly reduce redundant computation and storage requirements. Second, we developed the AP-FasterNet architecture for small targets that are commonly found in underwater datasets. By integrating depth-separable convolutions with different expansion rates into FasterNet, AP-FasterNet enhances the model’s ability to capture detailed features of small targets. Finally, we integrate the lightweight and efficient content-aware reorganization (CARAFE) up-sampling operation into YOLOv8 to enhance the model performance by aggregating contextual information over a large perceptual field and mitigating information loss during up-sampling.Evaluation results on the URPC2021 dataset show that the YOLOv8-LA model achieves 84.7% mean accuracy (mAP) on a single Nvidia GeForce RTX 3090 and operates at 189.3 frames per second (FPS), demonstrating that it outperforms existing state-of-the-art methods in terms of performance. This result demonstrates the model’s ability to ensure high detection accuracy while maintaining real-time processing capabilities.

DeformableFishNet: a high-precision lightweight target detector for underwater fish identification

The application of computer vision in fish identification facilitates researchers and managers to better comprehend and safeguard the aquatic ecological environment. Numerous researchers have harnessed deep learning methodologies for studying fish species identification. Nonetheless, this endeavor still encounters challenges such as high computational costs, a substantial number of parameters, and limited practicality. To address these issues, we propose a lightweight network architecture incorporating deformable convolutions, termed DeformableFishNet. Within DeformableFishNet, an efficient global coordinate attention module (EGCA) is introduced alongside a deformable convolution network (EDCN/EC2f), which is grounded in EGCA, to tackle the deformation of fish bodies induced by swimming motions. Additionally, an EC2f-based feature pyramid network (EDBFPN) and an efficient multi-scale decoupling head (EMSD Head) are proposed to extract multi-scale fish features within a lightweight framework. DeformableFishNet was deployed on our freshwater fish dataset, with experimental outcomes illustrating its efficacy, achieving a mean average precision (mAP) of 96.3%. The model comprises 1.7 million parameters and entails 4.7 billion floating-point operations (FLOPs). Furthermore, we validated DeformableFishNet on three public underwater datasets, yielding respective mAPs of 98%, 99.4%, and 83.6%. The experiments show that DeformableFishNet is suitable for underwater identification of various scenes.

A learnable full-frequency transformer dual generative adversarial network for underwater image enhancement

Underwater applications present unique challenges such as color deviation, noise, and low contrast, which can degrade image quality. Addressing these issues, we propose a novel approach called the learnable full-frequency transformer dual generative adversarial network (LFT-DGAN). Our method comprises several key innovations. Firstly, we introduce a reversible convolution-based image decomposition technique. This method effectively separates underwater image information into low-, medium-, and high-frequency domains, enabling more thorough feature extraction. Secondly, we employ image channels and spatial similarity to construct a learnable full-frequency domain transformer. This transformer facilitates interaction between different branches of information, enhancing the overall image processing capabilities. Finally, we develop a robust dual-domain discriminator capable of learning spatial and frequency domain characteristics of underwater images. Extensive experimentation demonstrates the superiority of the LFT-DGAN method over state-of-the-art techniques across multiple underwater datasets. Our approach achieves significantly improved quality and evaluation metrics, showcasing its effectiveness in addressing the challenges posed by underwater imaging. The code can be found at https://github.com/zhengshijian1993/LFT-DGAN.

DyFish-DETR: Underwater Fish Image Recognition Based on Detection Transformer

Due to the complexity of underwater environments and the lack of training samples, the application of target detection algorithms to the underwater environment has yet to provide satisfactory results. It is crucial to design specialized underwater target recognition algorithms for different underwater tasks. In order to achieve this goal, we created a dataset of freshwater fish captured from multiple angles and lighting conditions, aiming to improve underwater target detection of freshwater fish in natural environments. We propose a method suitable for underwater target detection, called DyFish-DETR (Dynamic Fish Detection with Transformers). In DyFish-DETR, we propose a DyFishNet (Dynamic Fish Net) to better extract fish body texture features. A Slim Hybrid Encoder is designed to fuse fish body feature information. The results of ablation experiments show that DyFishNet can effectively improve the mean Average Precision (mAP) of model detection. The Slim Hybrid Encoder can effectively improve Frame Per Second (FPS). Both DyFishNet and the Slim Hybrid Encoder can reduce model parameters and Floating Point Operations (FLOPs). In our proposed freshwater fish dataset, DyFish-DETR achieved a mAP of 96.6%. The benchmarking experimental results show that the Average Precision (AP) and Average Recall (AR) of DyFish-DETR are higher than several state-of-the-art methods. Additionally, DyFish-DETR, respectively, achieved 99%, 98.8%, and 83.2% mAP in other underwater datasets.

YOLOv8-MU: An Improved YOLOv8 Underwater Detector Based on a Large Kernel Block and a Multi-Branch Reparameterization Module.

Underwater visual detection technology is crucial for marine exploration and monitoring. Given the growing demand for accurate underwater target recognition, this study introduces an innovative architecture, YOLOv8-MU, which significantly enhances the detection accuracy. This model incorporates the large kernel block (LarK block) from UniRepLKNet to optimize the backbone network, achieving a broader receptive field without increasing the model's depth. Additionally, the integration of C2fSTR, which combines the Swin transformer with the C2f module, and the SPPFCSPC_EMA module, which blends Cross-Stage Partial Fast Spatial Pyramid Pooling (SPPFCSPC) with attention mechanisms, notably improves the detection accuracy and robustness for various biological targets. A fusion block from DAMO-YOLO further enhances the multi-scale feature extraction capabilities in the model's neck. Moreover, the adoption of the MPDIoU loss function, designed around the vertex distance, effectively addresses the challenges of localization accuracy and boundary clarity in underwater organism detection. The experimental results on the URPC2019 dataset indicate that YOLOv8-MU achieves an mAP@0.5 of 78.4%, showing an improvement of 4.0% over the original YOLOv8 model. Additionally, on the URPC2020 dataset, it achieves 80.9%, and, on the Aquarium dataset, it reaches 75.5%, surpassing other models, including YOLOv5 and YOLOv8n, thus confirming the wide applicability and generalization capabilities of our proposed improved model architecture. Furthermore, an evaluation on the improved URPC2019 dataset demonstrates leading performance (SOTA), with an mAP@0.5 of 88.1%, further verifying its superiority on this dataset. These results highlight the model's broad applicability and generalization capabilities across various underwater datasets.

MAM[formula omitted]SLAM: Towards underwater-robust multi-agent visual SLAM

Some underwater applications involve deploying multiple underwater Remotely Operated Vehicles in a common area. Such applications require the localization of these vehicles, not only with respect to each other but also with respect to a previously unknown environment. To this end, this work presents MAM3SLAM, a new fully centralized multi-agent and multi-map monocular Visual Simultaneous Localization And Mapping (VSLAM) framework. Multi-agent evaluation metrics are introduced to provide an extensive evaluation of MAM3SLAM compared to the state-of-the-art multi-agent VSLAM on four two-agent scenarios: one standard airborne dataset and three new underwater datasets recorded in a pool and the sea. The results show that MAM3SLAM is robust to underwater visual conditions and tracking failures, outperforms the other evaluated methods in estimating the individual and relative poses of the agents and in collaborative mapping accuracy. MAM3SLAM successfully estimates the individual and relative localization of the agents with an error lower than 5 cm on three out of the four test sequences, and is twice as accurate as competing multi-agent works in challenging visual conditions with frequent visual dropouts, poor textures, low framerate and fast motion. MAM3SLAM’s source code is made available, as are the underwater datasets.

Edge‐guided representation learning for underwater object detection

AbstractUnderwater object detection (UOD) is crucial for marine economic development, environmental protection, and the planet's sustainable development. The main challenges of this task arise from low‐contrast, small objects, and mimicry of aquatic organisms. The key to addressing these challenges is to focus the model on obtaining more discriminative information. The authors observe that the edges of underwater objects are highly unique and can be distinguished from low‐contrast or mimicry environments based on their edges. Motivated by this observation, an Edge‐guided Representation Learning Network, termed ERL‐Net is proposed, that aims to achieve discriminative representation learning and aggregation under the guidance of edge cues. Firstly, an edge‐guided attention module is introduced to model the explicit boundary information, which generates more discriminative features. Secondly, a hierarchical feature aggregation module is proposed to aggregate the multi‐scale discriminative features by regrouping them into three levels, effectively aggregating global and local information for locating and recognising underwater objects. Finally, a wide and asymmetric receptive field block is proposed to enable features to have a wider receptive field, allowing the model to focus on smaller object information. Comprehensive experiments on three challenging underwater datasets show that our method achieves superior performance on the UOD task.

Dual branch Transformer-CNN parametric filtering network for underwater image enhancement

Due to the absorption and scattering of light on the water surface, underwater images often face challenges such as low contrast, color deviation, insufficient exposure, and blurred details, exacerbating the difficulty of underwater tasks. In recent years, underwater image enhancement has become increasingly crucial in marine applications. Among existing underwater enhancement methods, the focus has often been on pixel-level learning, which may lead to image noise and an inability to finely adjust the image. In this paper, we propose a dual-branch Transformer-CNN Parameter Filtering network for underwater image enhancement, referred to as DTCPF. Specifically, to better aggregate window information, we introduce an overlapping window self-attention module to enhance interaction between windows. Additionally, we employ an improved Transformer encoder and decoder, utilizing long-distance attention and reversible neural networks to extract low-frequency and high-frequency information from the image. Moreover, we introduce a regression parameter filtering group for regression prediction, using the predicted parameters to enhance the image and obtain a reliable underwater enhancement model. Our approach undergoes qualitative and quantitative analyses on four real underwater datasets, demonstrating outstanding performance.

FUZ-SMO: A fuzzy slime mould optimizer for mitigating false alarm rates in the classification of underwater datasets using deep convolutional neural networks

Sonar sound datasets are of significant importance in the domains of underwater surveillance and marine research as they enable experts to discern intricate patterns within the depths of the water. Nevertheless, the task of classifying sonar sound datasets continues to pose significant challenges. In this study, we present a novel approach aimed at enhancing the precision and efficacy of sonar sound dataset classification. The integration of deep long-short-term memory (DLSTM) and convolutional neural networks (CNNs) models is employed in order to capitalize on their respective advantages while also utilizing distinctive feature engineering techniques to achieve the most favorable outcomes. Although DLSTM networks have demonstrated effectiveness in tasks involving sequence classification, attaining their optimal performance necessitates careful adjustment of hyperparameters. While traditional methods such as grid and random search are effective, they frequently encounter challenges related to computational inefficiencies. This study aims to investigate the unexplored capabilities of the fuzzy slime mould optimizer (FUZ-SMO) in the context of LSTM hyperparameter tuning, with the objective of addressing the existing research gap in this area. Drawing inspiration from the adaptive behavior exhibited by slime moulds, the FUZ-SMO proposes a novel approach to optimization. The amalgamated model, which combines CNN, LSTM, fuzzy, and SMO, exhibits a notable improvement in classification accuracy, outperforming conventional LSTM architectures by a margin of 2.142%. This model not only demonstrates accelerated convergence milestones but also displays significant resilience against overfitting tendencies.

Projector deep feature extraction-based garbage image classification model using underwater images

Marine and ocean pollution is one of the most serious environmental problems in the world. Marine plastics pose a significant threat to the marine ecosystem due to their negative effects. After passing through various processes, plastic waste accumulates on the seafloor and fragments into very small pieces known as microplastics. These microplastics are to blame for the extinction and death of aquatic life. This study obtained a hybrid underwater dataset containing 13,089 images, sized 300 × 300, including garbage and sea animals. In the proposed method, this dataset is used to develop our example projector deep feature generator. In this study, using the Resnet101 network in a sample projector build, the feature generator creates 6,000 features. Using NCA (Neighborhood Component Analysis), the best 1000 features from a pool of 6,000 are selected. The kNN (k-nearest neighbor) algorithm is then used to classify the resulting feature vectors. As validation techniques, both tenfold cross-validations were used. The hybrid dataset's best accuracy was calculated to be 99.35%. Our recommendation is successful based on the comparisons and calculated performance measures.

WaterPairs: a paired dataset for underwater image enhancement and underwater object detection

Due to its importance in marine engineering and aquatic robotics, underwater image enhancement works as a preprocessing step to improve the performance of high-level vision tasks such as underwater object detection and recognition. Although several studies exhibit that underwater image enhancement algorithms can boost the detection accuracy of detectors, no work has focused on studying the relationship between these two tasks. This is mainly because current underwater datasets lack either bounding box annotations or high-quality reference images, based on which detection accuracy or image quality assessment metrics are calculated. To examine how underwater image enhancement methods affect underwater object detection tasks, we provide a large-scale underwater object detection dataset with both bounding box annotations and high-quality reference images, namely, the WaterPairs dataset. The WaterPairs dataset offers a platform for researchers to comprehensively study the influence of underwater image enhancement algorithms on underwater object detection tasks. We will release our dataset at https://github.com/IanDragon/WaterPairs once this paper is accepted.

An in-situ image enhancement method for the detection of marine organisms by remotely operated vehicles

Abstract With the assistance of the visual system, remote operated vehicles (ROVs) can replace frogmen to achieve safer and more efficient capturing of marine organisms. However, the selective absorption and scattering of light lead to a decrease in the visual quality of underwater images, which hinders ROV operators from observing the operating environment. Unfortunately, most image enhancement methods only focus on image color correction rather than perceptual enhancement, which in turn prevents the object detector from quickly locating the target. Therefore, a visual-enhanced and detection-friendly underwater image enhancement method is needed. In this paper, an underwater image enhancement method called in-situ enhancement is proposed to improve the semantic information of the visual hierarchy based on current scene information in multiple stages. Mapping the underwater image to its dual space allows the enhancement equation to be applied to severely degraded underwater scenes. Moreover, it is also a detection-friendly method and has good generalization in both visual quality improvement and object detection. The experimental results show that in different underwater datasets, the in-situ enhancement effectively improves the visual quality of underwater images, and its enhanced results train different object detectors with high detection accuracy.

Dual stream fusion network for underwater image enhancement of multi-scale turbidity restoration and multi-path color correction.

Underwater images frequently experience color distortion and blurred details due to the absorption and scattering of light, which can hinder underwater visual tasks. To address these challenges, we propose a dual-stream fusion network for enhancing underwater images. Our multi-scale turbidity restoration module (MTRM) adopts a two-stage dehazing process from coarse to fine, while employing the SOS boosting strategy and frequency-based dense connections to further improve the performance of the U-Net. The multi-path color correction module (MCCM) utilizes the multi-path residual block as the basic unit to construct RGB enhancement paths. It selectively establishes inter-color channels through attention-based cross connections, which efficiently harness the distinctive features from various color channels. Additionally, non-local spatial and channel attention provide essential correlation information for the final fusion stage. Qualitative and quantitative evaluations conducted on various underwater datasets have demonstrated the excellent performance of our method.

Efficient Small-Object Detection in Underwater Images Using the Enhanced YOLOv8 Network

Underwater object detection plays a significant role in marine ecosystem research and marine species conservation. The improvement of related technologies holds practical significance. Although existing object-detection algorithms have achieved an excellent performance on land, they are not satisfactory in underwater scenarios due to two limitations: the underwater objects are often small, densely distributed, and prone to occlusion characteristics, and underwater embedded devices have limited storage and computational capabilities. In this paper, we propose a high-precision, lightweight underwater detector specifically optimizing for underwater scenarios based on the You Only Look Once Version 8 (YOLOv8) model. Firstly, we replace the Darknet-53 backbone of YOLOv8s with FasterNet-T0, reducing model parameters by 22.52%, FLOPS by 23.59%, and model size by 22.73%, achieving model lightweighting. Secondly, we add a Prediction Head for Small Objects, increase the number of channels for high-resolution feature map detection heads, and decrease the number of channels for low-resolution feature map detection heads. This results in a 1.2% improvement in small-object detection accuracy, while the remaining model parameters and memory consumption are nearly unchanged. Thirdly, we use Deformable ConvNets and Coordinate Attention in the neck part to enhance the accuracy in the detection of irregularly shaped and densely occluded small targets. This is achieved by learning convolution offsets from feature maps and emphasizing the regions of interest (RoIs). Our method achieves 52.12% AP on the underwater dataset UTDAC2020, with only 8.5 M parameters, 25.5 B FLOPS, and 17 MB model size. It surpasses the performance of large model YOLOv8l, at 51.69% AP, with 43.6 M parameters, 164.8 B FLOPS, and 84 MB model size. Furthermore, by increasing the input image resolution to 1280 × 1280 pixels, our model achieves 53.18% AP, making it the state-of-the-art (SOTA) model for the UTDAC2020 underwater dataset. Additionally, we achieve 84.4% mAP on the Pascal VOC dataset, with a substantial reduction in model parameters compared to previous, well-established detectors. The experimental results demonstrate that our proposed lightweight method retains effectiveness on underwater datasets and can be generalized to common datasets.

YOLO-based marine organism detection using two-terminal attention mechanism and difficult-sample resampling

The presence of various types of noise in images of marine-life datasets, as well as the class imbalances in underwater datasets, can exacerbate the difficulty in achieving effective object detection. To address this problem, we proposed you only look once (YOLO)-based marine organism detection using a two-terminal attention mechanism and difficult-sample resampling process. First, a residual building unit (RBU) module with a two-terminal attention mechanism (RBU-TA) was proposed, incorporating a reinforced channel attention mechanism into a shortcut of the residual structure. The proposed method adaptively compressed noisy feature map channels, providing rich shallow image information for high-level deep convolutional features while avoiding shallow noise pollution. To address the imbalance of marine biological image classes, difficult-sample resampling was combined with a focal loss function to suppress excessive background negative samples and retrain targets that could be difficult to distinguish, thus improving their detection accuracy. Finally, the proposed method was validated using the underwater robot professional competition (URPC) and real-world underwater object detection (RUOD) datasets, and the mean average precision (MAP) values of the results improved by 10% and 7%, respectively. The proposed method greatly improved the target detection accuracy of organisms in complex marine environments.

Instance Segmentation of Underwater Images by Using Deep Learning

Based on deep learning, an underwater image instance segmentation method is proposed. Firstly, in view of the scarcity of underwater related data sets, the size of the data set is expanded by measures including image rotation and flipping, and image generation by a generative adversarial network (GAN). Next, the underwater image data set is finally constructed by manual labeling. Then, in order to solve the problems of color shift, blur and the poor contrast of optical images caused by the complex underwater environment and the attenuation and scattering of light, an underwater image enhancement algorithm is used to first preprocess the data set, and several algorithms are discussed, including multi-scale Retinex (MSRCR) with color recovery, integrated color model (ICM), relative global histogram stretching (RGHS) and unsupervised color correction (UCM), as well as the color shift removal proposed in this work. Specifically, the results indicate that the proposed method can largely increase the segmentation mAP (mean average precision) by 85.7% compared with without the pretreatment method. In addition, based on the characteristics of the constructed underwater dataset, the feature pyramid network (FPN) is improved to some extent, and the preprocessing method is further combined with the improved network for experiments and compared with other neural networks to verify the effectiveness of the proposed method, thus achieving the effect and purpose of improving underwater image instance segmentation and target recognition. The experimental analysis results show that the proposed model can achieve a mAP of 0.245, which is about 1.1 times higher than other target recognition models.