Side-scan Sonar Images Research Articles

Side-Scan Sonar Image Generation Under Zero and Few Samples for Underwater Target Detection

The acquisition of side-scan sonar (SSS) images is complex, expensive, and time-consuming, making it difficult and sometimes impossible to obtain rich image data. Therefore, we propose a novel image generation algorithm to solve the problem of insufficient training datasets for SSS-based target detection. For zero-sample detection, we proposed a two-step style transfer approach. The ray tracing method was first used to obtain an optically rendered image of the target. Subsequently, UA-CycleGAN, which combines U-net, soft attention, and HSV loss, was proposed for generating high-quality SSS images. A one-stage image-generation approach was proposed for few-sample detection. The proposed ADA-StyleGAN3 incorporates an adaptive discriminator augmentation strategy into StyleGAN3 to solve the overfitting problem of the generative adversarial network caused by insufficient training data. ADA-StyleGAN3 generated high-quality and diverse SSS images. In simulation experiments, the proposed image-generation algorithm was evaluated subjectively and objectively. We also compared the proposed algorithm with other classical methods to demonstrate its advantages. In addition, we applied the generated images to a downstream target detection task, and the detection results further demonstrated the effectiveness of the image generation algorithm. Finally, the generalizability of the proposed algorithm was verified using a public dataset.

Improvement of Criminisi’s Stripe Noise Suppression Method for Side-Scan Sonar Images

In response to the problem of stripe noise significantly reducing the clarity and details of side-scan sonar images due to various factors, the authors of this paper propose an improved Criminisi method for stripe noise suppression. To address the issues encountered in the Criminisi algorithm during the suppression of stripe noise in side-scan sonar images, the following steps are suggested: firstly, introduce dynamic weights in the priority calculation to adaptively adjust the confidence and data term weights based on the current patch’s texture complexity; secondly, utilize the Sobel operator in the data term calculation to capture the image edge information more accurately; and, thirdly, optimize the matching block search process by introducing the Manhattan distance in addition to the Sum of Squared Differences (SSD) criterion to further select the best matching block while transitioning from a global search to a local search. Experimental validation was conducted using simulated stripe noise images, comparing the proposed method with four traditional denoising techniques. The results demonstrate that the denoising effectiveness of the proposed method is superior, effectively restoring texture in noisy regions while preserving texture structure integrity. Ablation experiments validate the effectiveness of the proposed improvements. Denoising experiments on real noisy images show satisfactory results with this method, and combining it with Fourier transform for additional smoothing in certain cases may yield even better results.

Prediction-Based Submarine Cable-Tracking Strategy for Autonomous Underwater Vehicles with Side-Scan Sonar

This study investigates the tracking of underwater cables using autonomous underwater vehicles (AUVs) equipped with side-scan sonar (SSS). AUV motion stability is crucial for effective SSS imaging, which is essential for continuous cable tracking. Traditional methods that derive AUV guidance rates directly from measured cable states often cause unnecessary jitter when imaging, complicating accurate detection. To address this, we propose a non-myopic receding-horizon optimization (RHO) strategy designed to maximize cable imaging quality while considering AUV maneuvering constraints. This strategy identifies the optimal heading decision sequence over a future horizon, ensuring stable and efficient cable tracking. We also employ a long short-term memory (LSTM) network to predict future cable states, further minimizing AUV motion instability during abrupt path changes. Given the computational limitations of AUVs, we have developed an efficient decision-making framework that can execute resource-intensive algorithms in real time. Finally, the robustness and effectiveness of the proposed algorithm were validated through comparative experiments. The results demonstrate that the proposed method outperforms existing methods in key metrics such as cable-tracking accuracy and AUV motion stability. This ensures that the AUV can acquire high-quality acoustic images of the submarine cable in an optimal state, enhancing the continuity and reliability of cable-tracking tasks.

Underwater Target Detection Using Side-Scan Sonar Images Based on Upsampling and Downsampling

Side-scan sonar (SSS) images present unique challenges to computer vision due to their lower resolution, smaller targets, and fewer features. Although the mainstream backbone networks have shown promising results on traditional vision tasks, they utilize traditional convolution to reduce the dimensionality of feature maps, which may cause information loss for small targets and decrease performance in SSS images. To address this problem, based on the yolov8 network, we proposed a new underwater target detection model based on upsampling and downsampling. Firstly, we introduced a new general downsampling module called shallow robust feature downsampling (SRFD) and a receptive field convolution (RFCAConv) in the backbone network. Thereby multiple feature maps extracted by different downsampling techniques can be fused to create a more robust feature map with a complementary set of features. Additionally, an ultra-lightweight and efficient dynamic upsampling module (Dysample) is introduced to improve the accuracy of the feature pyramid network (FPN) in fusing different levels of features. On the underwater shipwreck dataset, our improved model’s mAP50 increased by 4.4% compared to the baseline model.

Method for Augmenting Side-Scan Sonar Seafloor Sediment Image Dataset Based on BCEL1-CBAM-INGAN.

In this paper, a method for augmenting samples of side-scan sonar seafloor sediment images based on CBAM-BCEL1-INGAN is proposed, aiming to address the difficulties in acquiring and labeling datasets, as well as the insufficient diversity and quantity of data samples. Firstly, a Convolutional Block Attention Module (CBAM) is integrated into the residual blocks of the INGAN generator to enhance the learning of specific attributes and improve the quality of the generated images. Secondly, a BCEL1 loss function (combining binary cross-entropy and L1 loss functions) is introduced into the discriminator, enabling it to focus on both global image consistency and finer distinctions for better generation results. Finally, augmented samples are input into an AlexNet classifier to verify their authenticity. Experimental results demonstrate the excellent performance of the method in generating images of coarse sand, gravel, and bedrock, as evidenced by significant improvements in the Frechet Inception Distance (FID) and Inception Score (IS). The introduction of the CBAM and BCEL1 loss function notably enhances the quality and details of the generated images. Moreover, classification experiments using the AlexNet classifier show an increase in the recognition rate from 90.5% using only INGAN-generated images of bedrock to 97.3% using images augmented using our method, marking a 6.8% improvement. Additionally, the classification accuracy of bedrock-type matrices is improved by 5.2% when images enhanced using the method presented in this paper are added to the training set, which is 2.7% higher than that of the simple method amplification. This validates the effectiveness of our method in the task of generating seafloor sediment images, partially alleviating the scarcity of side-scan sonar seafloor sediment image data.

Side-Scan Sonar Image Augmentation Method Based on CC-WGAN

The utilization of deep learning algorithms for side-scan sonar target detection is impeded by the restricted quantity and representativeness of side-scan sonar (SSS) samples. To address this issue, this paper proposes a method for image augmentation using a CC-WGAN network. First, the generator incorporates the Convolutional Block Attention Module (CBAM) to enhance the assimilation of global information and local features in the input images. This integration also improves stability and avoids mode collapse problems associated with the original Generative Adversarial Network. Subsequently, the CBAM is incorporated into the discriminator to facilitate a better understanding of the relevance and significance of input data, thereby enhancing the model’s generalization ability. Finally, based on this model, existing few-sample SSS images are augmented, and we utilize the augmented images for discrimination and detection with YOLOv5. The experimental results show that following training with the SSS dataset that is augmented by this network, the accuracy of target detection increased by 7.6%, validating the feasibility of our proposed method. This method presents a novel solution to the problem of low model accuracy in underwater target detection with side-scan sonar due to limited samples.

Adversarial enhancement generation method for side-scan sonar images based on DDPM–YOLO

As marine exploration advances, efficient and accurate seabed target identification becomes increasingly critical. However, traditional methods and current technologies face challenges such as scarce samples and complex imaging conditions when dealing with side-scan sonar images. Given the scarcity of sample augmentation methods for side-scan sonar, this paper iteratively trains the Denoising Diffusion Probabilistic Models(DDPM), integrating the DDPM diffusion model and the downstream You Only Look Once(YOLO) retrieval task into a mutually reinforcing framework, proposing an adversarial enhancement generation method based on the DDPM and YOLO detection models. Experiments demonstrate that the DDPM model generated through this adversarial enhancement generation method can improve the accuracy of downstream YOLO target detection tasks by 7%. The images generated by this model also perform optimally on the Fréchet Inception Distance(FID), Maximum Mean Discrepancy(MMD), and Learned Perceptual Image Patch Similarity(LPIPS) metrics, thereby proving that our method can enhance the quality of generated images from the side-scan sonar diffusion model and offer a new avenue for improving the construction of underwater target detection models.

LSTM-based DEM generation in riverine environment

In the broad field of sensors and 3D information retrieval, bathymetric reconstruction from side-scan sonar imaging is associated with unique technical hurdles. Neural Networks have recently led to promising new solutions in this field, but the available methods tend to be complex and data-intensive in a way typically making their use in a riverine environment impossible. Throughout our work, we have focused on simplifying the problem-handling and treating compatibility with a riverine environment as priority. In our work, Long Short-Term Memory proved to be effective in a surprisingly simple form. Combined with traditional post-processing techniques in the GIS environment, like median filtered focal statistics, our workflow ultimately results in ∼0.259 m median of error on the evaluation dataset of the Dráva River.

DS-SIAUG: A Self-Training Approach Using a Disrupted Student Model for Enhanced Side-Scan Sonar Image Augmentation.

Side-scan sonar is a principal technique for subsea target detection, where the quantity of sonar images of seabed targets significantly influences the accuracy of intelligent target recognition. To expand the number of representative side-scan sonar target image samples, a novel augmentation method employing self-training with a Disrupted Student model is designed (DS-SIAUG). The process begins by inputting a dataset of side-scan sonar target images, followed by augmenting the samples through an adversarial network consisting of the DDPM (Denoising Diffusion Probabilistic Model) and the YOLO (You Only Look Once) detection model. Subsequently, the Disrupted Student model is used to filter out representative target images. These selected images are then reused as a new dataset to repeat the adversarial filtering process. Experimental results indicate that using the Disrupted Student model for selection achieves a target recognition accuracy comparable to manual selection, improving the accuracy of intelligent target recognition by approximately 5% over direct adversarial network augmentation.

SI-GAT: Enhancing Side-Scan Sonar Image Classification Based on Graph Structure

SI-GAT: Enhancing Side-Scan Sonar Image Classification Based on Graph Structure

Multi-Scale Marine Object Detection in Side-Scan Sonar Images Based on BES-YOLO.

Aiming at the problem of low accuracy of multi-scale seafloor target detection in side-scan sonar images with high noise and complex background texture, a model for multi-scale target detection using the BES-YOLO network is proposed. First, an efficient multi-scale attention (EMA) mechanism is used in the backbone of the YOLOv8 network, and a bi-directional feature pyramid network (Bifpn) is introduced to merge the information of different scales, finally, a Shape_IoU loss function is introduced to continuously optimize the model and improve its accuracy. Before training, the dataset is preprocessed using 2D discrete wavelet decomposition and reconstruction to enhance the robustness of the network. The experimental results show that 92.4% of the mean average accuracy at IoU of 0.5 (mAP@0.5) and 67.7% of the mean average accuracy at IoU of 0.5 to 0.95 (mAP@0.5:0.95) are achieved using the BES-YOLO network, which is an increase of 5.3% and 4.4% compared to the YOLOv8n model. The research results can effectively improve the detection accuracy and efficiency of multi-scale targets in side-scan sonar images, which can be applied to AUVs and other underwater platforms to implement intelligent detection of undersea targets.

Underwater Side-Scan Sonar Target Detection: YOLOv7 Model Combined with Attention Mechanism and Scaling Factor

Side-scan sonar plays a crucial role in underwater exploration, and the autonomous detection of side-scan sonar images is vital for detecting unknown underwater environments. However, due to the complexity of the underwater environment, the presence of a few highlighted areas on the targets, blurred feature details, and difficulty in collecting data from side-scan sonar, achieving high-precision autonomous target recognition in side-scan sonar images is challenging. This article addresses this problem by improving the You Only Look Once v7 (YOLOv7) model to achieve high-precision object detection in side-scan sonar images. Firstly, given that side-scan sonar images contain large areas of irrelevant information, this paper introduces the Swin-Transformer for dynamic attention and global modeling, which enhances the model’s focus on the target regions. Secondly, the Convolutional Block Attention Module (CBAM) is utilized to further improve feature representation and enhance the neural network model’s accuracy. Lastly, to address the uncertainty of geometric features in side-scan sonar target features, this paper innovatively incorporates a feature scaling factor into the YOLOv7 model. The experiment initially verified the necessity of attention mechanisms in the public dataset. Subsequently, experiments on our side-scan sonar (SSS) image dataset show that the improved YOLOv7 model has 87.9% and 49.23% in its average accuracy (mAP0.5) and (mAP0.5:0.95), respectively. These results are 9.28% and 8.41% higher than the YOLOv7 model. The improved YOLOv7 algorithm proposed in this paper has great potential for object detection and the recognition of side-scan sonar images.

Image stitching and target perception for Autonomous Underwater Vehicle-collected side-scan sonar images

Image stitching and target perception for Autonomous Underwater Vehicle-collected side-scan sonar images

SIGAN: A Multi-Scale Generative Adversarial Network for Underwater Sonar Image Super-Resolution

Super-resolution (SR) is a technique that restores image details based on existing information, enhancing the resolution of images to prevent quality degradation. Despite significant achievements in deep-learning-based SR models, their application in underwater sonar scenarios is limited due to the lack of underwater sonar datasets and the difficulty in recovering texture details. To address these challenges, we propose a multi-scale generative adversarial network (SIGAN) for super-resolution reconstruction of underwater sonar images. The generator is built on a residual dense network (RDN), which extracts rich local features through densely connected convolutional layers. Additionally, a Convolutional Block Attention Module (CBAM) is incorporated to capture detailed texture information by focusing on different scales and channels. The discriminator employs a multi-scale discriminative structure, enhancing the detail perception of both generated and high-resolution (HR) images. Considering the increased noise in super-resolved sonar images, our loss function emphasizes the PSNR metric and incorporates the L2 loss function to improve the quality of the output images. Meanwhile, we constructed a dataset for side-scan sonar experiments (DNASI-I). We compared our method with the current state-of-the-art super-resolution image reconstruction methods on the public dataset KLSG-II and our self-built dataset DNASI-I. The experimental results show that at a scale factor of 4, the average PSNR value of our method was 3.5 higher than that of other methods, and the accuracy of target detection using the super-resolution reconstructed images can be improved to 91.4%. Through subjective qualitative comparison and objective quantitative analysis, we demonstrated the effectiveness and superiority of the proposed SIGAN in the super-resolution reconstruction of side-scan sonar images.

АНАЛІЗ МЕТОДІВ ЗНАХОДЖЕННЯ АНОМАЛІЙ У ЗОБРАЖЕННЯХ

Abstract. Automatic anomaly detection is of great importance in industry, remote sensing, and medicine. It is important to be able to automatically process large amounts of data to detect, for example, chemical objects in multispectral and hyperspectral satellite images, sea mines in side-scan sonar images, or defects in industrial monitoring applications. Automatic detection of anomalous structures on arbitrary images refers to the task of finding inappropriate patterns relative to the normal state of the image. This is a difficult task in computer vision, since there is no clear and straightforward definition of what is normal or not normal for a given arbitrary image. The practical importance is manifested in the development of algorithms and models that can automatically detect unusual or anomalous patterns in images. An analysis of methods for finding anomalies in images from the point of view of the possibility of application to arbitrary images has been carried out. The classification of anomaly detection methods according to the criteria of the involved approaches and models used for modeling the background is presented. Methods that use machine learning, such as one-class support vector method and variational autoencoder, nearest neighbor-based anomaly detection, clustering-based anomaly detection, statistical anomaly detection, spectral anomaly detection, anomaly detection using information theory are discussed. The main attention is paid to the methods classified according to the background modeling approach. Five categories of background modeling methods based on probability density function, global and local homogeneity, sparsity, and self-similarity are considered. For anomaly detection applications, it is recommended to use methods in which the background model best describes the expected anomaly-free background, as this generally results in the best performance. On the basis of research, it was established that an effective universal model for detecting anomalies in arbitrary images should: use only a self-similar or sparse background model; process the residual image as a stochastic process to detect anomalies, such as anomalies in color noise; preprocess the residual image before detecting the anomaly.

Shuffle-RDSNet: a method for side-scan sonar image classification with residual dual-path shrinkage network

Shuffle-RDSNet: a method for side-scan sonar image classification with residual dual-path shrinkage network

A Multi-Scale Fusion Strategy for Side Scan Sonar Image Correction to Improve Low Contrast and Noise Interference

Side scan sonar images have great application prospects in underwater surveys, target detection, and engineering activities. However, the acquired sonar images exhibit low illumination, scattered noise, distorted outlines, and unclear edge textures due to the complicated undersea environment and intrinsic device flaws. Hence, this paper proposes a multi-scale fusion strategy for side scan sonar (SSS) image correction to improve the low contrast and noise interference. Initially, an SSS image was decomposed into low and high frequency sub-bands via the non-subsampled shearlet transform (NSST). Then, modified multi-scale retinex (MMSR) was employed to enhance the contrast of the low frequency sub-band. Next, sparse dictionary learning (SDL) was utilized to eliminate high frequency noise. Finally, the process of NSST reconstruction was completed by fusing the emerging low and high frequency sub-band images to generate a new sonar image. The experimental results demonstrate that the target features, underwater terrain, and edge contours could be clearly displayed in the image corrected by the multi-scale fusion strategy when compared to eight correction techniques: BPDHE, MSRCR, NPE, ALTM, LIME, FE, WT, and TVRLRA. Effective control was achieved over the speckle noise of the sonar image. Furthermore, the AG, STD, and E values illustrated the delicacy and contrast of the corrected images processed by the proposed strategy. The PSNR value revealed that the proposed strategy outperformed the advanced TVRLRA technology in terms of filtering performance by at least 8.8%. It can provide sonar imagery that is appropriate for various circumstances.

Object Detection in Side Scan Sonar Images using Local Binary Pattern features

Object Detection in Side Scan Sonar Images using Local Binary Pattern features

Side-Scan Sonar Image Matching Method Based on Topology Representation

In the realm of underwater environment detection, achieving information matching stands as a pivotal step, forming an indispensable component for collaborative detection and research in areas such as distributed mapping. Nevertheless, the progress in studying the matching of underwater side-scan sonar images has been hindered by challenges including low image quality, intricate features, and susceptibility to distortion in commonly used side-scan sonar images. This article presents a comprehensive overview of the advancements in underwater sonar image processing. Building upon the novel SchemaNet image topological structure extraction model, we introduce a feature matching model grounded in side-scan sonar images. The proposed approach employs a semantic segmentation network as a teacher model to distill the DeiT model during training, extracting the attention matrix of intermediate layer outputs. This emulates SchemaNet’s transformation method, enabling the acquisition of high-dimensional topological structure features from the image. Subsequently, utilizing a real side-scan sonar dataset and augmenting data, we formulate a matching dataset and train the model using a graph neural network. The resulting model demonstrates effective performance in side-scan sonar image matching tasks. These research findings bear significance for underwater detection and target recognition and can offer valuable insights and references for image processing in diverse domains.

Assessing the importance of sediment characterization on seabed embedment predictions of cylindrical objects

The prediction of object embedment into the seabed is important for the deployment and efficacy of submerged sensors and sensor networks, as well as for other applications such as unexploded ordnance risk assessment, subaquatic search and rescue efforts, or anchors and moorings. The embedment of objects into the seabed during placement depends on the strength parameters of the seabed sediment. However, a detailed characterization of seabed strength can be complex and costly since quality seabed samples and/or in situ testing are required. This study evaluates the importance of a detailed geotechnical seabed sediment characterization over seabed classification from geoacoustic surveying or accepting unknown seabed conditions for the prediction of seabed embedment of cylindrical objects. Monte Carlo simulation was applied to model the uncertainty of sediment strength parameters. The results suggest that prior knowledge of the general sediment type (e.g., fine-grained versus coarse-grained as determined from side scan sonar imaging) significantly improves the precision of the embedment predictions over unknown conditions, reducing the coefficient of variation (CV) in prediction of percent embedment (PE) by about 50 % to 65 %. Performing a detailed geotechnical testing of sediment strength represented a further improvement by 15 % to 35 % over the knowledge of sediment type. Both the mean and standard deviation of PE in fine-grained sediments are at least about 3 times the values for coarse-grained sediments indicating a wider distribution of PE with deeper embedment for cohesive fine-grained soils. The embedment behavior is a function of the method of deployment (free fall with a certain speed versus placement with no speed) in stiff cohesive sediments but is less relevant in soft sediments since PE values are always greater than 100 % (i.e., full embedment). Similarly, in cohesionless sediments, PE is always less than 50 %.