Underwater Image Segmentation Research Articles

Target recognition and segmentation in turbid water using data from non-turbid conditions: a unified approach and experimental validation

Semantic segmentation of targets in underwater images within turbid water environments presents significant challenges, hindered by factors such as environmental variability, difficulties in acquiring datasets, imprecise data annotation, and the poor robustness of conventional methods. This paper addresses this issue by proposing a novel joint method using deep learning to effectively perform semantic segmentation tasks in turbid environments, with the practical case of efficiently collecting polymetallic nodules in deep-sea while minimizing damage to the seabed environment. Our approach includes a novel data expansion technique and a modified U-net based model. Drawing on the underwater image formation model, we introduce noise to clear water images to simulate images captured under varying degrees of turbidity, thus providing an alternative to the required data. Furthermore, traditional U-net-based modified models have shown limitations in enhancing performance in such tasks. Based on the primary factors underlying image degradation, we propose a new model which incorporates an improved dual-channel encoder. Our method significantly advances the fine segmentation of underwater images in turbid media, and experimental validation demonstrates its effectiveness and superiority under different turbidity conditions. The study provides new technical means for deep-sea resource development, holding broad application prospects and scientific value.

Instance Segmentation of Underwater Images by Using Deep Learning

Based on deep learning, an underwater image instance segmentation method is proposed. Firstly, in view of the scarcity of underwater related data sets, the size of the data set is expanded by measures including image rotation and flipping, and image generation by a generative adversarial network (GAN). Next, the underwater image data set is finally constructed by manual labeling. Then, in order to solve the problems of color shift, blur and the poor contrast of optical images caused by the complex underwater environment and the attenuation and scattering of light, an underwater image enhancement algorithm is used to first preprocess the data set, and several algorithms are discussed, including multi-scale Retinex (MSRCR) with color recovery, integrated color model (ICM), relative global histogram stretching (RGHS) and unsupervised color correction (UCM), as well as the color shift removal proposed in this work. Specifically, the results indicate that the proposed method can largely increase the segmentation mAP (mean average precision) by 85.7% compared with without the pretreatment method. In addition, based on the characteristics of the constructed underwater dataset, the feature pyramid network (FPN) is improved to some extent, and the preprocessing method is further combined with the improved network for experiments and compared with other neural networks to verify the effectiveness of the proposed method, thus achieving the effect and purpose of improving underwater image instance segmentation and target recognition. The experimental analysis results show that the proposed model can achieve a mAP of 0.245, which is about 1.1 times higher than other target recognition models.

Deep Learning for Enhanced Marine Vision: Object Detection in Underwater Environments

This study leverages the Semantic Segmentation of Underwater Imagery (SUIM) dataset, encompassing over 1,500 meticulously annotated images that delineate eight distinct object categories. These categories encompass a diverse array, ranging from vertebrate fish and invertebrate reefs to aquatic vegetation, wreckage, human divers, robots, and the seafloor. The use of this dataset involves a methodical synthesis of data through extensive oceanic expeditions and collaborative experiments, featuring both human participants and robots. The research extends its scope to evaluating cutting-edge semantic segmentation techniques, employing established metrics to gauge their performance comprehensively. Additionally, we introduce a fully convolutional encoder-decoder model designed with a dual purpose: delivering competitive performance and computational efficiency. Notably, this model boasts a remarkable accuracy of 88%, underscoring its proficiency in underwater image segmentation. Furthermore, this model's integration within the autonomy pipeline of visually-guided underwater robots presents its tangible applicability. Its rapid end-to-end inference capability addresses the exigencies of real-time decision-making, vital for autonomous systems. This study elucidates the model's practical benefits across diverse applications like visual serving, saliency prediction, and intricate scene comprehension. Crucially, the utilization of the Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) elevates image quality, enriching the foundation upon which our model's success rests. This research establishes a solid groundwork for future exploration in underwater robot vision by presenting the model and the benchmark dataset.

Underwater Marine Life Study using Yolo V8

Abstract: Underwater target detection, object classification, and image segmentation play an important role in ocean exploration and marine life studies for which the improvement of relevant technology is of much practical significance. Although existing target detection, image classification, and segmentation algorithms have achieved excellent performance on land they often fail to achieve satisfactory outcomes of detection and classification when in the underwater environment. In this paper, one of the most advanced target detection algorithms, YOLO v8 (You Only Look Once), was first applied in the underwater environment before being improved by combining it with some methods characteristic of the underwater environment. The state-of-the-art backbone and neck architectures and TC-YOLO/SAM were treated as the basic backbone network of YOLO v8, which makes the network suitable for underwater images.

Research on an underwater image segmentation algorithm based on YOLOv8

Underwater image segmentation faces complex problems such as target type, number, scale, and environmental interference. The traditional segmentation method relies on much manual annotation and expertise, extracts only the surface information of the image, and has low segmentation accuracy and efficiency, so the image segmentation method based on deep learning is chosen. Deep learning algorithms are used on top of the YOLOv8, and the adaptive self-supervised learning (Convnext V2) module, the lightweight network (Slim Neck), and the dynamic sparse attention mechanism (bilevel routing attention, Biformer) are added, which improves the YOLOv8 algorithm for single-stage instance segmentation of different underwater targets. The Convnext V2 module introduces the global response normalization (GRN) unit, which enhances the feature competition and reduces the model computation. A slim neck network is designed, which achieves a double improvement in model inference speed and accuracy by the operation of dense convolution in the lightweight convolution (GSConv) module and the design of a cross-level network structure (VoV-GSCSP). The introduction of Biformer’s dynamic sparse attention mechanism achieves more flexible computational allocation and content awareness. Comparison experiments show that the improved algorithm dramatically increases the segmentation speed while improving the segmentation accuracy. On the dual-target dataset, the mAP is improved by 3.5% from 0.753 to 0.779, and the FPS is improved by 86% from 63 to 117. The experiments validate that the improved algorithm can be used for the segmentation of different underwater images.

Underwater image enhancement method based on golden jackal optimization

The propagation of light underwater is affected by water absorption and scattering, resulting in poor visibility of obtained underwater images, and exhibiting degradation phenomena such as color deviation and low contrast. To improve the quality and visibility of underwater images, a method of underwater image enhancement based on Golden Jackal optimization is proposed. The algorithm consists of two processing stages. The first stage is mainly processed in the RGB color space. Based on the attenuation characteristics of underwater color channels, a constructed color compensation and correction strategy is used to eliminate color deviation caused by water absorption. In second stage, it is mainly processed in the Lab color space, using the Golden Jackal optimization algorithm and fitness function to search the optimal parameters of transformation function to improve the image contrast by optimizing the grey intensity of L component and its histogram distribution. The feasibility and effectiveness of the proposed method have been verified through testing and analysis of much underwater degraded image data. In subjective and objective analysis with multiple state-of-the-art underwater image clarity methods, the proposed method performs excellent in color restoration and detail texture enhancement. Further experiments have shown that the proposed method has strong robustness and universality. It is not only suitable for enhancing various underwater scene images, but also for enhancing environmental images with fog, low lighting, and sandstorms. Meanwhile, the proposed method can effectively improve the performance of underwater image segmentation, keypoint detection, and saliency detection.

Deep learning-based marine big data fusion for ocean environment monitoring: Towards shape optimization and salient objects detection

ObjectiveDuring the last few years, underwater object detection and marine resource utilization have gained significant attention from researchers and become active research hotspots in underwater image processing and analysis domains. This research study presents a data fusion-based method for underwater salient object detection and ocean environment monitoring by utilizing a deep model.MethodologyA hybrid model consists of an upgraded AlexNet with Inception v-4 for salient object detection and ocean environment monitoring. For the categorization of spatial data, AlexNet is utilized, whereas Inception V-4 is employed for temporal data (environment monitoring). Moreover, we used preprocessing techniques before the classification task for underwater image enhancement, segmentation, noise and fog removal, restoration, and color constancy.ConclusionThe Real-Time Underwater Image Enhancement (RUIE) dataset and the Marine Underwater Environment Database (MUED) dataset are used in this research project’s data fusion and experimental activities, respectively. Root mean square error (RMSE), computing usage, and accuracy are used to construct the model’s simulation results. The suggested model’s relevance form optimization and conspicuous item prediction issues in the seas is illustrated by the greatest accuracy of 95.7% and low RMSE value of 49 when compared to other baseline models.

Kapur’s entropy underwater image segmentation based on multi-strategy Manta ray foraging optimization

Kapur’s entropy underwater image segmentation based on multi-strategy Manta ray foraging optimization

Data augmentation using image translation for underwater sonar image segmentation

In underwater environment, the study of object recognition is an important basis for implementing an underwater unmanned vessel. For this purpose, abundant experimental data to train deep learning model is required. However, it is very difficult to obtain these data because the underwater experiment itself is very limited in terms of preparation time and resources. In this study, the image transformation model, Pix2Pix is utilized to generate data similar to experimental one obtained by our ROV named SPARUS between the pool and reservoir. These generated data are applied to train the other deep learning model, FCN for a pixel segmentation of images. The original sonar image and its mask image have to be prepared for all training data to train the image segmentation model and it takes a lot of effort to do it what if all training data are supposed to be real sonar images. Fortunately, this burden can be released here, for the pairs of mask image and synthesized sonar image are already consisted in the image transformation step. The validity of the proposed procedures is verified from the performance of the image segmentation result. In this study, when only real sonar images are used for training, the mean accuracy is 0.7525 and the mean IoU is 0.7275. When the both synthetic and real data is used for training, the mean accuracy is 0.81 and the mean IoU is 0.7225. Comparing the results, the performance of mean accuracy increase to 6%, performance of the mean IoU is similar value.

An underwater dam crack image segmentation method based on multi-level adversarial transfer learning

Crack detection is necessary to ensure the health of dams. Traditional detection methods perform poorly because of weak adaptability and poor image quality. Deep learning shows excellent performance in crack image detection. However, it is difficult to realize supervised learning due to the lack of labelled underwater crack image datasets. Thus, a transfer learning method named MA-AttUNet is proposed. The proposed method realizes knowledge transfer of crack image features using a multi-level adversarial transfer network. With this method, prior knowledge learned from the source domain can be applied to underwater crack image segmentation. Additionally, the attention mechanism is integrated into the segmentation network to eliminate noise interference during detection by assigning different weights to target and background pixels. Experiments show that the proposed method achieves higher segmentation precision than existing works.

NOVEL APPROACHES TO ENHANCE CORAL REEFS MONITORING WITH UNDERWATER IMAGE SEGMENTATION

Abstract. Coral reefs not only inhabiting millions of species that are primarily or completely associated with them, but also produce economic and cultural benefits to coastal societies around the world. In recent years, affected by climate change and human factors, coral reef ecosystem has been experiencing accelerated degradation. Coral reef monitoring activities are therefore required to assess the impact of adverse factors on corals and to track subsequent recovery or decline. The collection of image data has become a common approach in the field of underwater monitoring, but traditional coral image data analysis mainly has high time and labor costs. We need to investigate the spatial distribution of different coral populations in the study area through image segmentation methods to help oceanographers develop effective management and conservation strategies. In fact, deep learning has shown better prediction performance than traditional image processing or traditional machine learning algorithms in coral image segmentation tasks. Starting from classification of random point annotations, segmentation of sparsely labeled data, and segmentation of densely labeled data, this paper summarizes state-of-the-art techniques of segmentation in deep learning applied to underwater images. Then, we discuss the problems and challenges of CNN-based underwater image segmentation of coral reefs, and make corresponding solutions or possible directions for future.

Automatic Semantic Segmentation of Benthic Habitats Using Images from Towed Underwater Camera in a Complex Shallow Water Environment

Underwater image segmentation is useful for benthic habitat mapping and monitoring; however, manual annotation is time-consuming and tedious. We propose automated segmentation of benthic habitats using unsupervised semantic algorithms. Four such algorithms––Fast and Robust Fuzzy C-Means (FR), Superpixel-Based Fast Fuzzy C-Means (FF), Otsu clustering (OS), and K-means segmentation (KM)––were tested for accuracy for segmentation. Further, YCbCr and the Commission Internationale de l’Éclairage (CIE) LAB color spaces were evaluated to correct variations in image illumination and shadow effects. Benthic habitat field data from a geo-located high-resolution towed camera were used to evaluate proposed algorithms. The Shiraho study area, located off Ishigaki Island, Japan, was used, and six benthic habitats were classified. These categories were corals (Acropora and Porites), blue corals (Heliopora coerulea), brown algae, other algae, sediments, and seagrass (Thalassia hemprichii). Analysis showed that the K-means clustering algorithm yielded the highest overall accuracy. However, the differences between the KM and OS overall accuracies were statistically insignificant at the 5% level. Findings showed the importance of eliminating underwater illumination variations and outperformance of the red difference chrominance values (Cr) in the YCbCr color space for habitat segmentation. The proposed framework enhanced the automation of benthic habitat classification processes.

Underwater sonar image segmentation combining pixel-level and region-level information

Side-scan sonar (SSS) has been increasingly utilized in underwater exploration, due to the low energy attenuation of the acoustic wave in water. However, SSS imaging is characterized by its high speckle noises, strong intensity bias and low spatial resolution, which pose challenges to accurately delineate objects in underwater sonar images. This issue is handled by our proposed model that segments sonar images via combining pixel-level and region-level information. Pixel-to-pixel and region-to-region neighboring relations are considered in this model. Formally, the unified Markov random field (UMRF) is combined with the level set (LS) to establish a novel energy function for segmenting underwater sonar images. In such a model, the pixel-level information and region-level information are synergistically integrated by UMRF to force LS evolution. Results obtained by our model convergence can accurately segment underwater sonar images into three partitions: the object, shadow and background. From experimental analysis, our proposed model achieves an excellent tradeoff between the segmentation accuracy and noise resistance in underwater sonar image segmentation. The average correctness of our model reaches 0.9431 which has more than 7% improved to the best conventional method.

Underwater Sonar Image Segmentation by a Novel Joint Level Set Model

This paper proposes a novel joint level set model for underwater sonar image segmentation. Combining features in points and regions in our novel joint level set (LS), it can achieve excellent performance for underwater sonar image segmentation. Regional information guides the model to locate the object of interest, whereas the point information accurately delineates contours. In addition, the unified Markov random field (UMRF) is taken to measure the neighboring relation between points and regions, which can overcome the problems of the high speckle noise, strong bias and low resolution of underwater sonar images. Our novel model can segment underwater sonar images into three partitions, such as the objects of interest, shadow and backgrounds. In contrast to current segmentation methods, outstanding results are demonstrated by our model. Moreover, another advantage of our model lies in its high efficiency.

Underwater Image Enhancement via Minimal Color Loss and Locally Adaptive Contrast Enhancement.

Underwater images typically suffer from color deviations and low visibility due to the wavelength-dependent light absorption and scattering. To deal with these degradation issues, we propose an efficient and robust underwater image enhancement method, called MLLE. Specifically, we first locally adjust the color and details of an input image according to a minimum color loss principle and a maximum attenuation map-guided fusion strategy. Afterward, we employ the integral and squared integral maps to compute the mean and variance of local image blocks, which are used to adaptively adjust the contrast of the input image. Meanwhile, a color balance strategy is introduced to balance the color differences between channel a and channel b in the CIELAB color space. Our enhanced results are characterized by vivid color, improved contrast, and enhanced details. Extensive experiments on three underwater image enhancement datasets demonstrate that our method outperforms the state-of-the-art methods. Our method is also appealing in its fast processing speed within 1s for processing an image of size 1024×1024×3 on a single CPU. Experiments further suggest that our method can effectively improve the performance of underwater image segmentation, keypoint detection, and saliency detection. The project page is available at https://li-chongyi.github.io/proj_MMLE.html.

Deep-sea Nodule Mineral Image Segmentation Algorithm Based on Pix2PixHD

Deep-sea mineral image segmentation plays an important role in deep-sea mining and underwater mineral resource monitoring and evaluation. The application of artificial intelligence technology to deep-sea mining projects can effectively improve the quality and efficiency of mining. The existing deep learning-based underwater image segmentation algorithms have problems such as the accuracy rate is not high enough and the running time is slightly longer. In order to improve the segmentation performance of underwater mineral images, this paper uses the Pix2PixHD (Pixel to Pixel High Definition) algorithm based on Conditional Generative Adversarial Network (CGAN) to segment deep-sea mineral images. The model uses a coarse-to-fine generator composed of a global generation network and two local enhancement networks, and multiple multi-scale discriminators with same network structures but different input pictures to generate high-quality images. The test results on the deep-sea mineral datasets show that the Pix2PixHD algorithm can identify more target minerals under certain other conditions. The evaluation index shows that the Pix2PixHD algorithm effectively improves the accuracy rate and the recall rate of deep-sea mineral image segmentation compared with the CGAN algorithm and the U-Net algorithm. It is important for expanding the application of deep learning techniques in the field of deep-sea exploration and mining.

Retraction Note: Underwater image segmentation based on computer vision and research on recognition algorithm

Retraction Note: Underwater image segmentation based on computer vision and research on recognition algorithm

Underwater image segmentation in the wild using deep learning

Image segmentation is an important step in many computer vision and image processing algorithms. It is often adopted in tasks such as object detection, classification, and tracking. The segmentation of underwater images is a challenging problem as the water and particles present in the water scatter and absorb the light rays. These effects make the application of traditional segmentation methods cumbersome. Besides that, to use the state-of-the-art segmentation methods to face this problem, which are based on deep learning, an underwater image segmentation dataset must be proposed. So, in this paper, we develop a dataset of real underwater images, and some other combinations using simulated data, to allow the training of two of the best deep learning segmentation architectures, aiming to deal with segmentation of underwater images in the wild. In addition to models trained in these datasets, fine-tuning and image restoration strategies are explored too. To do a more meaningful evaluation, all the models are compared in the testing set of real underwater images. We show that methods obtain impressive results, mainly when trained with our real dataset, comparing with manually segmented ground truth, even using a relatively small number of labeled underwater training images.

RETRACTED ARTICLE: Underwater image segmentation based on computer vision and research on recognition algorithm

Due to the continuous growth of the world’s population, the development and utilization of marine resources have received great attention. At present, marine fishing relies heavily on divers for underwater operations, which have the disadvantages of high risk, low efficiency, and high cost. Therefore, the development of underwater capture robot which can automatically detect, locate, and capture targets is of great significance for the development of marine economy. Underwater robot is an indispensable equipment for deep-sea operation and plays an irreplaceable role in the development of the ocean. When autonomous underwater vehicles perform underwater operations, they can use computer vision system to obtain clear underwater images and accurate target category information, which can help the manipulator select different grasping parts for different shapes and categories and improve work efficiency. The current underwater vision technology includes “acoustic vision” and “light vision.” Due to the influence of multichannel effect and blind area, the acoustic vision research in detecting and tracking underwater targets is not deep enough. Compared with the acoustic image processing system, the underwater optical vision system has the advantages of image and video capture and has higher real-time performance, which can aim at the target faster and more conveniently. Underwater vision optical system plays an important leading role in the detailed research of underwater vehicle sensing system, which can further improve the autonomous performance of underwater vehicle. In addition, considering the nature of underwater image imaging, we are developing an underwater image segmentation and recognition system based on image processing.

Modified water wave optimization algorithm for underwater multilevel thresholding image segmentation

Multilevel thresholding is a simple and important method for image segmentation in various applications that has drawn widespread attention in recent years. However, the computational complexity increases correspondingly when the threshold levels increase. To overcome this drawback, a modified water wave optimization (MWWO) algorithm with the elite opposition-based learning strategy and the ranking-based mutation operator for underwater image segmentation is proposed in this paper. The elite opposition-based learning strategy increases the diversity of the population and prevents the search from stagnating to improve the calculation accuracy. The ranking-based mutation operator increases the selection probability. MWWO can effectively balance exploration and exploitation to obtain the optimal solution in the search space. To objectively evaluate the overall performance of the proposed algorithm, MWWO is compared with six state-of-the-art meta-heuristic algorithms by maximizing the fitness value of Kapur’s entropy method to obtain the optimal threshold through experiments on ten test images. The fitness value, the best threshold values, the execution time, the peak signal to noise ratio (PSNR), the structure similarity index (SSIM), and the Wilcoxon’s rank-sum test are used as important metrics to evaluate the segmentation effect of underwater images. The experimental results show that MWWO has a better segmentation effect and stronger robustness compared with other algorithms and an effective and feasible method for solving underwater multilevel thresholding image segmentation.