Underwater Object Recognition Research Articles

Multi-scale aware turbulence network for underwater object recognition

Underwater imagery is subject to distortion, and the presence of turbulence in the fluid medium poses difficulties in accurately discerning objects. To tackle these challenges pertaining to feature extraction, this research paper presents a novel approach called the multi-scale aware turbulence network (MATNet) method for underwater object identification. More specifically, the paper introduces a module known as the multi-scale feature extraction pyramid network module, which incorporates dense linking strategies and position learning strategies to preprocess object contour features and texture features. This module facilitates the efficient extraction of multi-scale features, thereby enhancing the effectiveness of the identification process. Following that, the extracted features undergo refinement through comparison with positive and negative samples. Ultimately, the study introduces multi-scale object recognition techniques and establishes a multi-scale object recognition network for the precise identification of underwater objects, utilizing the enhanced multi-scale features. This process entails rectifying the distorted image and subsequently recognizing the rectified object. Extensive experiments conducted on an underwater distorted image enhancement dataset demonstrate that the proposed method surpasses state-of-the-art approaches in both qualitative and quantitative evaluations.

WaterPairs: a paired dataset for underwater image enhancement and underwater object detection

Due to its importance in marine engineering and aquatic robotics, underwater image enhancement works as a preprocessing step to improve the performance of high-level vision tasks such as underwater object detection and recognition. Although several studies exhibit that underwater image enhancement algorithms can boost the detection accuracy of detectors, no work has focused on studying the relationship between these two tasks. This is mainly because current underwater datasets lack either bounding box annotations or high-quality reference images, based on which detection accuracy or image quality assessment metrics are calculated. To examine how underwater image enhancement methods affect underwater object detection tasks, we provide a large-scale underwater object detection dataset with both bounding box annotations and high-quality reference images, namely, the WaterPairs dataset. The WaterPairs dataset offers a platform for researchers to comprehensively study the influence of underwater image enhancement algorithms on underwater object detection tasks. We will release our dataset at https://github.com/IanDragon/WaterPairs once this paper is accepted.

An Underwater Organism Image Dataset and a Lightweight Module Designed for Object Detection Networks

Long-term monitoring and recognition of underwater organism objects are of great significance in marine ecology, fisheries science and many other disciplines. Traditional techniques in this field, including manual fishing-based ones and sonar-based ones, are usually flawed. Specifically, the method based on manual fishing is time-consuming and unsuitable for scientific researches, while the sonar-based one, has the defects of low acoustic image accuracy and large echo errors. In recent years, the rapid development of deep learning and its excellent performance in computer vision tasks make vision-based solutions feasible. However, the researches in this area are still relatively insufficient in mainly two aspects. First, to our knowledge, there is still a lack of large-scale datasets of underwater organism images with accurate annotations. Second, in consideration of the limitation on hardware resources of underwater devices, an underwater organism detection algorithm that is both accurate and lightweight enough to be able to infer in real time is still lacking. As an attempt to fill in the aforementioned research gaps to some extent, we established the Multiple Kinds of Underwater Organisms (MKUO) dataset with accurate bounding box annotations of taxonomic information, which consists of 10,043 annotated images, covering eighty-four underwater organism categories. Based on our benchmark dataset, we evaluated a series of existing object detection algorithms to obtain their accuracy and complexity indicators as the baseline for future reference. In addition, we also propose a novel lightweight module, namely Sparse Ghost Module, designed especially for object detection networks. By substituting the standard convolution with our proposed one, the network complexity can be significantly reduced and the inference speed can be greatly improved without obvious detection accuracy loss. To make our results reproducible, the dataset and the source code are available online at https://cslinzhang.github.io/MKUO-and-Sparse-Ghost-Module/ .

Underwater image classification based on image enhancement and information quality evaluation

Underwater target imaging is widely used in oceans, rivers and lakes detection fields, but due to the existence of water on light absorption scattering attenuation effect, the diffraction limit of imaging system, aberration distortion and underwater turbulence, underwater images has serious degradation, mainly manifested in noise, fuzzy and low resolution, etc. In recent years, some scholars have started from the image enhancement and restoration to improve the image degradation and improve the image clarity, but the effect still needs to be improved, and the resolution can not be improved, and some algorithms will even reduce the resolution.Resolution is an important parameter for the image evaluation, improving the image quality in the underwater imaging, especially in the human eye vision, which is largely determined by the improved resolution. Therefore, this paper starts from each aspects of image processing, studies, proposes and experiments a large number of image recovery algorithms. This paper analyzes various factors introducing noise in underwater imaging process, proposes block mixed filter image denoising method based on noise statistical characteristics, analyzes the necessity of evaluating image quality by using objective indicators, and gives the corresponding objective evaluation parameters according to the characteristics of the image recovery algorithm. According to the noise preprocessing method and the image quality evaluation method, the underwater image synthesis dataset is screened to improve the performance of the underwater object recognition model.

Synthetic data formation for machine learning recognition of underwater objects

The main problem of machine learning for control systems of unmanned underwater vehicles is objectively very small samples of real data. The study aim was to develop an approach to the creation of synthetic data describing underwater objects for as the samples for training and validation in machine learning of control systems for auton-omous unmanned underwater vehicles. The subject of the study was a variety of underwater objects because real information about the shape, size and their external images is very limited. The data augmentation method was used, which makes it possible to obtain additional data on underdetermined objects of observation from the initial data while maintaining the classification features. Eight models have been developed that imitate the influence of various factors of the aquatic environment and allow using various augmentation methods (changing the position, adding noise, glare to the image, defocusing to create fuzziness; fragmentation, etc.) to obtain an almost unlimited number of images of any objects of man-made activity immersed in underwater environment, to varying degrees similar to the reference. Examples of the use of augmentation models that take into account changes in illumina-tion, transparency and the presence of an underwater landscape are given. Such synthetic (model) images may be the basis of a training set for machine learning to recognize and identify underwater objects. The trained model can be used as the basis of a decision support system for operators of remote-controlled unmanned underwater vehicles and as the basis for building control systems for autonomous uninhabited underwater vehicles for moni-toring underwater spaces.

Evaluation of Laser Image Enhancement and Restoration for Underwater Object Recognition

Evaluation of Laser Image Enhancement and Restoration for Underwater Object Recognition

Monitoring marine pollution for carbon neutrality through a deep learning method with multi-source data fusion

IntroductionMarine pollution can have a significant impact on the blue carbon, which finally affect the ocean’s ability to sequester carbon and contribute to achieving carbon neutrality. Marine pollution is a complex problem that requires a great deal of time and effort to measure. Existing machine learning algorithms cannot effectively solve the detection time problem and provide limited accuracy. Moreover, marine pollution can come from a variety of sources. However, most of the existing research focused on a single ocean indicator to analyze marine pollution. In this study, two indicators, marine organisms and debris, are used to create a more complete picture of the extent and impact of pollution in the ocean.MethodsTo effectively recognize different marine objects in the complex marine environment, we propose an integrated data fusion approach where deep convolutional neural networks (CNNs) are combined to conduct underwater object recognition. Through this multi-source data fusion approach, the accuracy of object recognition is significantly improved. After feature extraction, four machine and deep learning classifiers’ performances are used to train on features extracted with deep CNNs.ResultsThe results show that VGG-16 achieves better performance than other feature extractors when detecting marine organisms. When detecting marine debris, AlexNet outperforms other deep CNNs. The results also show that the LSTM classifier with VGG-16 for detecting marine organisms outperforms other deep learning models.DiscussionFor detecting marine debris, the best performance was observed with the AlexNet extractor, which obtained the best classification result with an LSTM. This information can be used to develop policies and practices aimed at reducing pollution and protecting marine environments for future generations.

Underwater occluded object recognition with two-stage image reconstruction strategy

Underwater occluded object recognition with two-stage image reconstruction strategy

Simultaneous restoration and super-resolution GAN for underwater image enhancement

Underwater images are generally of low quality, limiting the performance of subsequent perceptual tasks, such as underwater object detection and recognition. However, only a few methods can improve the quality of underwater images by simultaneously restoring and super-resolving underwater images. In this paper, we propose an end-to-end trainable model based on generative adversarial networks (GANs) called Simultaneous Restoration and Super-Resolution GAN (SRSRGAN) to obtain clear super-resolution underwater images automatically. In particular, our model leverages a cascaded architecture with two stages of carefully designed generative adversarial networks to restore and super-resolve corrupted underwater images in a coarse-to-fine manner. The major advantages of SRSRGAN are twofold. First, it is a unified solution that can simultaneously restore and super-resolve images. Second, SRSRGAN is not limited by the prior experience of the types and levels of underwater degraded images but can perform the inference using only observed corrupted images. These two advantages enable SRSRGAN to enjoy better flexibility and higher practicability in realistic underwater scenarios. Extensive experimental results demonstrate the superiority of SRSRGAN in underwater image restoration, super-resolution, and simultaneous restoration and super-resolution.

SO-YOLOv5: Small object recognition algorithm for sea cucumber in complex seabed environment

Underwater object recognition is a prerequisite for the realization of automated seafood harvesting. To solve the problems of low recognition accuracy and poor generalization ability of existing underwater small object, this paper takes sea cucumber as research objects to research the small object recognition approach in complex underwater environment. In this paper, a sea cucumber dataset is established to solve the problem of lacking sea cucumber dataset in the real seabed environment. A deep learning model SO-YOLOv5 is proposed based on YOLOv5 for underwater small object recognition, which improves the recognition ability of the algorithm for different sizes of objects in complex environment. The proposed model embeds a large-scale feature extraction layer in structure to increase the detection ability of small objects by referring to the idea of the Bi-directional Feature Pyramid Network. SO-YOLOv5 fuses the features between deep and shallow layers and balances the feature information of different scales, and integrates the Coordinate Attention mechanism to enhance the sensitivity of the algorithm to the direction and position information. The experimental results illustrate that the mAP of the proposed approach achieves 95.47% for sea cucumber recognition in the complex seabed environment, which is 3.42%, 6.79%, and 5.46% higher than the traditional Faster R-CNN, SSD, and YOLOv5 algorithms, respectively. This research not only provides an effective approach for sea cucumber recognition in complex environment but also has certain reference significance for the recognition of small objects in other complex environments. In addition, the proposed approach has practical application value to improve intelligence level in aquaculture.

Development of an intelligent underwater recognition system based on the deep reinforcement learning algorithm in an autonomous underwater vehicle

This study’s objective was to design an intelligent underwater recognition system and apply it in an autonomous underwater vehicle (AUV) for the recognition and tracking of underwater objects. The intelligent underwater recognition system predicted the depth map with the stereo matching algorithm based on semi-global block matching (SGBM) through the images of voyage records. It used the Deep Q-Network (DQN) algorithm based on deep reinforcement learning so that the agent may focus on the localization area of objects on the disparity map. Next, the intelligent underwater recognition system performed depth estimation according to the disparity map to obtain the stereo point clouds of the underwater object. After obtaining the depth information, the intelligent underwater recognition system constructed a deep network based on Faster Region-based Convolutional Neural Network (R-CNN) to detect the underwater object. Eventually, the system was successfully verified by a series of diving-depth tracking experiments.

In Situ Sea Cucumber Detection across Multiple Underwater Scenes Based on Convolutional Neural Networks and Image Enhancements.

Recently, rapidly developing artificial intelligence and computer vision techniques have provided technical solutions to promote production efficiency and reduce labor costs in aquaculture and marine resource surveys. Traditional manual surveys are being replaced by advanced intelligent technologies. However, underwater object detection and recognition are suffering from the image distortion and degradation issues. In this work, automatic monitoring of sea cucumber in natural conditions is implemented based on a state-of-the-art object detector, YOLOv7. To depress the image distortion and degradation issues, image enhancement methods are adopted to improve the accuracy and stability of sea cucumber detection across multiple underwater scenes. Five well-known image enhancement methods are employed to improve the detection performance of sea cucumber by YOLOv7 and YOLOv5. The effectiveness of these image enhancement methods is evaluated by experiments. Non-local image dehazing (NLD) was the most effective in sea cucumber detection from multiple underwater scenes for both YOLOv7 and YOLOv5. The best average precision (AP) of sea cucumber detection was 0.940, achieved by YOLOv7 with NLD. With NLD enhancement, the APs of YOLOv7 and YOLOv5 were increased by 1.1% and 1.6%, respectively. The best AP was 2.8% higher than YOLOv5 without image enhancement. Moreover, the real-time ability of YOLOv7 was examined and its average prediction time was 4.3 ms. Experimental results demonstrated that the proposed method can be applied to marine organism surveying by underwater mobile platforms or automatic analysis of underwater videos.

A Multi-Layer Perceptron Model for Underwater Object Recognition

The more human gets interested in sea exploration, the more research to detect objects under the water is done. Therefore, the ability to detect, classify, recognize and track all kinds of objects is evolving daily. This paper aims to introduce a computer model for underwater object classification and recognition based on a Multilayer Perceptron network. The model was constructed with a mixed dataset for the training phase, combining artificial and natural objects, and it reached approximately 99.97% classifying accuracy.

B-YOLOX-S: A Lightweight Method for Underwater Object Detection Based on Data Augmentation and Multiscale Feature Fusion

With the increasing maturity of underwater agents-related technologies, underwater object recognition algorithms based on underwater robots have become a current hotspot for academic and applied research. However, the existing underwater imaging conditions are poor, the images are blurry, and the underwater robot visual jitter and other factors lead to lower recognition precision and inaccurate positioning in underwater target detection. A YOLOX-based underwater object detection model, B-YOLOX-S, is proposed to detect marine organisms such as echinus, holothurians, starfish, and scallops. First, Poisson fusion is used for data amplification at the input to balance the number of detected targets. Then, wavelet transform is used to perform Style Transfer on the enhanced images to achieve image restoration. The clarity of the images and detection targets is further increased and the generalization of the model is enhanced. Second, a combination of BIFPN-S and FPN is proposed to fuse the effective feature layer obtained by the Backbone layer to enhance the detection precision and accelerate model detection. Finally, the localization loss function of the prediction layer in the network is replaced by EIoU_Loss to heighten the localization precision in detection. Experimental results comparing the B-YOLOX-S algorithm model with mainstream algorithms such as FasterRCNN, YOLOV3, YOLOV4, YOLOV5, and YOLOX on the URPC2020 dataset show that the detection precision and detection speed of the algorithm model have obvious advantages over other algorithm networks. The average detection accuracy mAP value is 82.69%, which is 5.05% higher than the benchmark model (YOLOX-s), and the recall rate is 8.03% higher. Thus, the validity of the algorithmic model proposed in this paper is demonstrated.

Enhancement and detection of objects in underwater images using image super-resolution and effective object detection model

It is imperative to build an automatic underwater object recognition system in place to reduce the costs of underwater inspections as well as the associated risks. An effective method of detecting underwater objects from underwater images of aquatic after enhancing them using the Image Super-resolution technique is proposed in this study. The proposed approach comprises of two major sections, Underwater Image Enhancement, and Object detection. To enhance the underwater images, a lightweight Reduced Cascading Residual Network (RCARN) is proposed that imposes the Image Super-resolution technique. Later, the enhanced images generated by the RCARN model are supplied for the object detection process, where a significant object detection model, YOLOv3 is employed in this study. To improve its performance, this YOLOv3 is trained on one of the largest datasets, the COCO data, followed by being fine-tuned using enhanced Underwater images. The dataset utilized in this work contains 6 classes of underwater objects namely dolphin, jellyfish, octopus, seahorse, starfish, and turtle. All these images are actual real field images collected from various sources. With this proposed approach, a better overall ACS and mAP of 95.44% and 75.33% are achieved here, which are improved by ~8.75% and ~15%, respectively when compared to actual collected low-resolution images.

Hovering control of UUV through underwater object detection based on deep learning

It is difficult to find a target object in water and hover an UUV at a relative position from the object to work on it. To solve this problem in this study, we used the geometric principles of camera image mapping to find the real-time position of an underwater object, based on which we investigated the hovering control of an UUV. For underwater object recognition, we used YOLOv2, a deep learning-based object detection algorithm, which has excellent real-time performance. To recognize objects in various underwater environments, the training was conducted on data obtained under different underwater environment conditions, such as illuminance, distance, and obstacles, and the performance of underwater object recognition was increased by using UUV to acquire training data in this study. An UUV was designed and fabricated to implement the proposed algorithm, and a hovering control algorithm was developed for the UUV to recognize a star-shaped object and control its relative distance and direction. The learning data were built, and the object recognition rate was increased using many real sea tests. Moreover, the camera mapping algorithm and the UUV control algorithm proposed in this paper were applied together and tested at sea, to achieve a stable control of the UUV.

Underwater occlusion object recognition with fusion of significant environmental features

Complicated underwater environments, such as occlusion by foreign objects and dim light, cause serious loss of underwater object features. This certainly increases the challenge of accurate detection of underwater objects. To tackle the above problems, we proposed an underwater occlusion object recognition algorithm combined with significant environmental features. First, the salient features of the underwater object were extracted by the salient feature extraction network. Second, salient environmental features associated with the object were extracted by the environmental feature attention mechanism. Finally, the contrast graph structure of the object’s salient features and related environmental features was constructed. The model proposed is compared with the existing object recognition model in the simulation of the three datasets: the underwater image enhancement benchmark dataset, video diver dataset, and underwater image dataset. The average recognition accuracy of the proposed model is 80.07%, which is improved by 2.38%. Experiments show that the object recognition algorithm proposed is effective and superior to existing algorithms.

DeepRecog: Threefold underwater image deblurring and object recognition framework for AUV vision systems

DeepRecog: Threefold underwater image deblurring and object recognition framework for AUV vision systems

Optimization of Underwater Marker Detection Based on YOLOv3

The research on the detection and recognition technology of marker is of great significance for some underwater operations, such as marine resource exploration, underwater robot operation and so on. The existing image processing methods can effectively detect and recognize the markers in the air. Nevertheless, in the underwater environment, the complex imaging environment of the ocean leads to serious degradation of underwater images obtained by the optical vision system. Due to the lack of effective information for object recognition, the severely degraded underwater images increases the difficulty of detection and recognition of underwater objects. With the development of high-tech underwater imaging equipment, the quality of underwater images has been improved to a certain extent, but there are still some phenomena such as color fading, low contrast and blurred details. Solutions to overcome these problems are important for the exploration of the ocean. In this paper, we introduce a deep learning model to optimize the performance of detection, and make a unique marker dataset for the application scene of our experiment. We first use the deep learning network to pre-train the marker images in the air. Next, we use the underwater marker images for fine-tuning. Finally, after the target marker is detected, the traditional image processing method is used to recognize the marker. Experimental results show that the optimization method we proposed achieves better performance on the dataset.

Sonar-based yaw estimation of target object using shape prediction on viewing angle variation with neural network

This paper proposes a method to estimate the underwater target object's yaw angle using a sonar image. A simulator modeling imaging mechanism of a sonar sensor and a generative adversarial network for style transfer generates realistic template images of the target object by predicting shapes according to the viewing angles. Then, the target object's yaw angle can be estimated by comparing the template images and a shape taken in real sonar images. We verified the proposed method by conducting water tank experiments. The proposed method was also applied to AUV in field experiments. The proposed method, which provides bearing information between underwater objects and the sonar sensor, can be applied to algorithms such as underwater localization or multi-view-based underwater object recognition.