Side-scan Sonar Research Articles

Automated Recognition of Submerged Body-like Objects in Sonar Images Using Convolutional Neural Networks

The Police Robot for Inspection and Mapping of Underwater Evidence (PRIME) is an uncrewed surface vehicle (USV) currently being developed for underwater search and recovery teams to assist in crime scene investigation. The USV maps underwater scenes using sidescan sonar (SSS). Test exercises use a clothed mannequin lying on the seafloor as a target object to evaluate system performance. A robust, automated method for detecting human body-shaped objects is required to maximise operational functionality. The use of a convolutional neural network (CNN) for automatic target recognition (ATR) is proposed. SSS image data acquired from four different locations during previous missions were used to build a dataset consisting of two classes, i.e., a binary classification problem. The target object class consisted of 166 196 × 196 pixel image snippets of the underwater mannequin, whereas the non-target class consisted of 13,054 examples. Due to the large class imbalance in the dataset, CNN models were trained with six different imbalance ratios. Two different pre-trained models (ResNet-50 and Xception) were compared, and trained via transfer learning. This paper presents results from the CNNs and details the training methods used. Larger datasets are shown to improve CNN performance despite class imbalance, achieving average F1 scores of 97% in image classification. Average F1 scores for target vs background classification with unseen data are only 47% but the end result is enhanced by combining multiple weak classification results in an ensemble average. The combined output, represented as a georeferenced heatmap, accurately indicates the target object location with a high detection confidence and one false positive of low confidence. The CNN approach shows improved object detection performance when compared to the currently used ATR method.

New insights on gravity flow dynamics during submarine canyon flushing events

Millions of tons of material are flushed through submarine canyons during infrequent high-magnitude events, transporting coastal sediment to the deep ocean. However, observations related to individual canyon flushing events are challenging due to the destructive nature and infrequency of flushing events. The impacts of one of the largest gravity flows in the past decade were documented in Kaikōura Canyon, Aotearoa−New Zealand, where >1 km3 of sediment was mobilized by the 2016 CE Kaikōura earthquake (Mw 7.8). We present new high-resolution (<1 m) multibeam data collected with an autonomous underwater vehicle (AUV) along the Kaikōura Canyon axis, together with side-scan sonar and seafloor video imagery. These data sets reveal a wide range of erosional and depositional features that were not previously identified. Eroded bedrock and deep erosional structures are found in the upper canyon, including linear grooves, and rockfall debris (>5-m-diameter boulders). This erosional area transitions downcanyon to coarse-grained depositional bedforms, including cyclic steps and gravel waves (average wavelengths of 250 m and wave heights of ∼20 m), covering the mid- and lower canyon. Our observations provide high-resolution field-based evidence of (1) flow transformation, from a debris flow to a high-density turbidity current; and (2) variations of flow dynamics within turbidity currents both across- and downcanyon, during an infrequent, high-energy canyon flushing event. This research offers new insights into the processes that create and shape nearshore bedrock-incising submarine canyons.

Possibility of diagnostics of layered media with interferometric side-view sonar

A method is considered and an algorithm is developed that makes it possible to identify the layered structure of the propagation medium of a probing signal based on strip survey data from an interferometric side-scan sonar (ISSS) with antennas located in a vertical plane. Using the example of mathematical modeling of phase-difference measurements of ISSS for multilayer scattering planes, the capabilities of the proposed algorithm to determine their spatial position are demonstrated in the wave propagation medium. An analysis of the accuracy of calculating the position of scattering layers at heights (depths) and with different slopes was performed. The effectiveness of the method and algorithm for diagnosing the structure of layered media has been confirmed. The effectiveness of the method was tested on experimental data obtained using ISSS.

Autonomous Underwater Vehicle Navigation Enhancement by Optimized Side-Scan Sonar Registration and Improved Post-Processing Model Based on Factor Graph Optimization

Autonomous Underwater Vehicles (AUVs) equipped with Side-Scan Sonar (SSS) play a critical role in seabed mapping, where precise navigation data are essential for mosaicking sonar images to delineate the seafloor’s topography and feature locations. However, the accuracy of AUV navigation, based on Strapdown Inertial Navigation System (SINS)/Doppler Velocity Log (DVL) systems, tends to degrade over long-term mapping, which compromises the quality of sonar image mosaics. This study addresses the challenge by introducing a post-processing navigation method for AUV SSS surveys, utilizing Factor Graph Optimization (FGO). Specifically, the method utilizes an improved Fourier-based image registration algorithm to generate more robust relative position measurements. Then, through the integration of these measurements with data from SINS, DVL, and surface Global Navigation Satellite System (GNSS) within the FGO framework, the approach notably enhances the accuracy of the complete trajectory for AUV missions. Finally, the proposed method has been validated through both the simulation and AUV marine experiments.

Prediction-Based Submarine Cable-Tracking Strategy for Autonomous Underwater Vehicles with Side-Scan Sonar

This study investigates the tracking of underwater cables using autonomous underwater vehicles (AUVs) equipped with side-scan sonar (SSS). AUV motion stability is crucial for effective SSS imaging, which is essential for continuous cable tracking. Traditional methods that derive AUV guidance rates directly from measured cable states often cause unnecessary jitter when imaging, complicating accurate detection. To address this, we propose a non-myopic receding-horizon optimization (RHO) strategy designed to maximize cable imaging quality while considering AUV maneuvering constraints. This strategy identifies the optimal heading decision sequence over a future horizon, ensuring stable and efficient cable tracking. We also employ a long short-term memory (LSTM) network to predict future cable states, further minimizing AUV motion instability during abrupt path changes. Given the computational limitations of AUVs, we have developed an efficient decision-making framework that can execute resource-intensive algorithms in real time. Finally, the robustness and effectiveness of the proposed algorithm were validated through comparative experiments. The results demonstrate that the proposed method outperforms existing methods in key metrics such as cable-tracking accuracy and AUV motion stability. This ensures that the AUV can acquire high-quality acoustic images of the submarine cable in an optimal state, enhancing the continuity and reliability of cable-tracking tasks.

Underwater Target Detection Using Side-Scan Sonar Images Based on Upsampling and Downsampling

Side-scan sonar (SSS) images present unique challenges to computer vision due to their lower resolution, smaller targets, and fewer features. Although the mainstream backbone networks have shown promising results on traditional vision tasks, they utilize traditional convolution to reduce the dimensionality of feature maps, which may cause information loss for small targets and decrease performance in SSS images. To address this problem, based on the yolov8 network, we proposed a new underwater target detection model based on upsampling and downsampling. Firstly, we introduced a new general downsampling module called shallow robust feature downsampling (SRFD) and a receptive field convolution (RFCAConv) in the backbone network. Thereby multiple feature maps extracted by different downsampling techniques can be fused to create a more robust feature map with a complementary set of features. Additionally, an ultra-lightweight and efficient dynamic upsampling module (Dysample) is introduced to improve the accuracy of the feature pyramid network (FPN) in fusing different levels of features. On the underwater shipwreck dataset, our improved model’s mAP50 increased by 4.4% compared to the baseline model.

Application of the Coastal and Marine Ecological Classification Standard (CMECS) to Create Benthic Geologic Habitat Maps for Portions of Acadia National Park, Maine, USA

The Coastal and Marine Ecological Classification Standard (CMECS) was applied to four portions of Acadia National Park, USA, focusing on intertidal rocky and tidal flat habitats. Side-scan sonar coupled with multi-phase echo sounder bathymetry are the primary data sources used to map the seafloor, coupled with underwater video imagery and surface grab samples for grain size and macrofaunal analysis. The CMECS Substrate, Geoform, and Biotic components were effective in describing the study areas. However, integrating the CMECS components to define Biotopes was more challenging due to the limited number of grab samples available and because the dominant species within a given map unit is largely inconsistent. While Biotopes ultimately could not be defined in this study, working within the CMECS framework to create statistically significant biotopes revealed the complexity of these study areas that may otherwise have been overlooked. This study demonstrates the effectiveness of the CMECS classification, including the framework’s ability to be flexible in communicating information.

Storm-Driven Tree Exposure and Geomorphic Change: Predicting the Distribution of Preserved Late Pleistocene Tree Stumps on the Outer Alabama Continental Shelf

The Alabama Underwater or Drowned Forest is a well-preserved Late Pleistocene (dated to 72–56 ± 8 ka, 2σ) terrestrial landform on the northern Gulf of Mexico continental shelf that provides geomorphic and ecosystem information rarely preserved during the glacial intervals. Stumps of bald cypress (Taxodium distichum (L.) Rich.) trees were exposed in ∼18 m of water following Hurricane Ivan in 2004. This research investigates geomorphic changes to the Mississippi-Alabama-Florida (MAFLA) sand sheet, which presents as shore-oblique Holocene sand ridges, and the exposure and burial of tree stumps following the passage of Hurricane Sally in 2020 using repeat sidescan and bathymetric surveys (2015–2016 and 2021). Using two newly identified tree exposure areas and their geological properties, this research also hypothesized a new location where tree stumps may be outcropping. The bathymetry indicates regions with up to ∼1 m of erosion and deposition over the five years between the two surveys. Similarly, the sidescan sonar indicates changes in the location and numbers of exposed tree stumps as well as between 47,000 and 62,500 tons of sediment erosion within the study area following Hurricane Sally. The 2015 and 2016 data found 25 tree contacts whereas the 2021 survey found 76 tree contacts and only 5 of them occurred in both surveys suggesting the tree exposures are dynamic and presumably changing with the passing of large tropical cyclones. Additionally, the hypothesized exposure location had 26 newly identified tree stump contacts within a mixed texture unit along the sand-mud boundary, confirming our understanding of the geomorphic characteristics leading to the exposure of the buried forest. This research will expand the potential areas for investigations into Late Pleistocene ecosystems and landforms and their associated climatic and ecologic conditions in the Gulf of Mexico as well as in other passive continental margins.

Benthic modification and biotic associations at natural and artificial habitats excavated by Epinephelus morio and Lutjanus campechanus

Pockmarks are abundant seafloor features worldwide and, in West Florida Shelf waters <110 m deep, are thought to be sites of sediment excavation primarily by red grouper Epinephelus morio, although red snapper Lutjanus campechanus also excavate sediment. During 2014-2017, side-scan sonar (445 kHz) was used to locate and deploy stereo-baited remote underwater video arrays within view of 202 such excavations in waters 17-110 m deep on the West Florida Shelf off the Florida Panhandle and Peninsula. Three excavation habitat classes included 73 isolated excavations on open sand, 74 associated with low-relief hard bottom, and 55 associated with artificial reefs. Physical characteristics of excavations varied between regions, among habitats, and with depth; mean diameter (±1 SE) was 9.9 ± 0.3 m (range: 3-24.6 m). Excavations not around artificial reefs contained 6.9 ± 0.5 m2 (0-27.7 m2) of exposed rock, and epibenthic growth covered 33 ± 2% of the interiors. Members of 99 fish genera were identified. Fish abundance was greatest at isolated excavations which showed similar evenness to excavated artificial reefs; diversity was higher at excavated low-relief hard bottom. L. campechanus was much more common in Panhandle waters, especially at excavated artificial reefs which had subsided 0.8 ± 0.1 vertical meters below the seafloor (i.e. 48 ± 4% of the structure). These biotic and abiotic characteristics of excavations highlight the importance of E. morio’s ecosystem-engineering services and provide new insight into the contributions of L. campechanus in creating or maintaining excavations at natural and anthropogenic habitats.

Side-Scan Sonar Image Augmentation Method Based on CC-WGAN

The utilization of deep learning algorithms for side-scan sonar target detection is impeded by the restricted quantity and representativeness of side-scan sonar (SSS) samples. To address this issue, this paper proposes a method for image augmentation using a CC-WGAN network. First, the generator incorporates the Convolutional Block Attention Module (CBAM) to enhance the assimilation of global information and local features in the input images. This integration also improves stability and avoids mode collapse problems associated with the original Generative Adversarial Network. Subsequently, the CBAM is incorporated into the discriminator to facilitate a better understanding of the relevance and significance of input data, thereby enhancing the model’s generalization ability. Finally, based on this model, existing few-sample SSS images are augmented, and we utilize the augmented images for discrimination and detection with YOLOv5. The experimental results show that following training with the SSS dataset that is augmented by this network, the accuracy of target detection increased by 7.6%, validating the feasibility of our proposed method. This method presents a novel solution to the problem of low model accuracy in underwater target detection with side-scan sonar due to limited samples.

Adversarial enhancement generation method for side-scan sonar images based on DDPM–YOLO

As marine exploration advances, efficient and accurate seabed target identification becomes increasingly critical. However, traditional methods and current technologies face challenges such as scarce samples and complex imaging conditions when dealing with side-scan sonar images. Given the scarcity of sample augmentation methods for side-scan sonar, this paper iteratively trains the Denoising Diffusion Probabilistic Models(DDPM), integrating the DDPM diffusion model and the downstream You Only Look Once(YOLO) retrieval task into a mutually reinforcing framework, proposing an adversarial enhancement generation method based on the DDPM and YOLO detection models. Experiments demonstrate that the DDPM model generated through this adversarial enhancement generation method can improve the accuracy of downstream YOLO target detection tasks by 7%. The images generated by this model also perform optimally on the Fréchet Inception Distance(FID), Maximum Mean Discrepancy(MMD), and Learned Perceptual Image Patch Similarity(LPIPS) metrics, thereby proving that our method can enhance the quality of generated images from the side-scan sonar diffusion model and offer a new avenue for improving the construction of underwater target detection models.

LSTM-based DEM generation in riverine environment

In the broad field of sensors and 3D information retrieval, bathymetric reconstruction from side-scan sonar imaging is associated with unique technical hurdles. Neural Networks have recently led to promising new solutions in this field, but the available methods tend to be complex and data-intensive in a way typically making their use in a riverine environment impossible. Throughout our work, we have focused on simplifying the problem-handling and treating compatibility with a riverine environment as priority. In our work, Long Short-Term Memory proved to be effective in a surprisingly simple form. Combined with traditional post-processing techniques in the GIS environment, like median filtered focal statistics, our workflow ultimately results in ∼0.259 m median of error on the evaluation dataset of the Dráva River.

An Interpretable Multi-Model Machine Learning Approach for Spatial Mapping of Deep-Sea Polymetallic Nodule Occurrences

AbstractHigh-resolution mapping of deep-sea polymetallic nodules is needed (a) to understand the reasons behind their patchy distribution, (b) to associate nodule coverage with benthic fauna occurrences, and (c) to enable an accurate resource estimation and mining path planning. This study used an autonomous underwater vehicle to map 37 km2 of a geomorphologically complex site in the Eastern Clarion–Clipperton Fracture Zone. A multibeam echosounder system (MBES) at 400 kHz and a side scan sonar at 230 kHz were used to investigate the nodule backscatter response. More than 30,000 seafloor images were analyzed to obtain the nodule coverage and train five machine learning (ML) algorithms: generalized linear models, generalized additive models, support vector machines, random forests (RFs) and neural networks (NNs). All models ML yielded similar maps of nodule coverage with differences occurring in the range of predicted values, particularly at parts with irregular topography. RFs had the best fit and NNs had the worst spatial transferability. Attention was given to the interpretability of model outputs using variable importance ranking across all models, partial dependence plots and domain knowledge. The nodule coverage is higher on relatively flat seafloor ( < 3°) with eastward-facing slopes. The most important predictor was the MBES backscatter, particularly from incident angles between 25 and 55°. Bathymetry, slope, and slope orientation were important geomorphological predictors. For the first time, at a water depth of 4500 m, orthophoto-mosaics and image-derived digital elevation models with 2-mm and 5-mm spatial resolutions supported the geomorphological analysis, interpretation of polymetallic nodules occurrences, and backscatter response.

DS-SIAUG: A Self-Training Approach Using a Disrupted Student Model for Enhanced Side-Scan Sonar Image Augmentation.

Side-scan sonar is a principal technique for subsea target detection, where the quantity of sonar images of seabed targets significantly influences the accuracy of intelligent target recognition. To expand the number of representative side-scan sonar target image samples, a novel augmentation method employing self-training with a Disrupted Student model is designed (DS-SIAUG). The process begins by inputting a dataset of side-scan sonar target images, followed by augmenting the samples through an adversarial network consisting of the DDPM (Denoising Diffusion Probabilistic Model) and the YOLO (You Only Look Once) detection model. Subsequently, the Disrupted Student model is used to filter out representative target images. These selected images are then reused as a new dataset to repeat the adversarial filtering process. Experimental results indicate that using the Disrupted Student model for selection achieves a target recognition accuracy comparable to manual selection, improving the accuracy of intelligent target recognition by approximately 5% over direct adversarial network augmentation.

SI-GAT: Enhancing Side-Scan Sonar Image Classification Based on Graph Structure

SI-GAT: Enhancing Side-Scan Sonar Image Classification Based on Graph Structure

CORAL-Catamaran for Underwater Exploration: Development of a Multipurpose Unmanned Surface Vessel for Environmental Studies.

CORAL (Catamaran fOr UndeRwAter expLoration) is a compact, unmanned catamaran-type vehicle designed and developed to assist the scientific community in exploring marine areas such as inshore regions that are not easily accessible by traditional vessels. This vehicle can operate in different modalities: completely autonomous, semi-autonomous, or remotely assisted by the operator, thus accommodating various investigative scenarios. CORAL is characterized by compact dimensions, a very low draft and a total electric propulsion system. The vehicle is equipped with a single echo-sounder, a 450 kHz Side Scan Sonar, an Inertial Navigation System assisted by a GPS receiver and a pair of high-definition cameras for recording both above and below the water surface. Here, we present results from two investigations: the first conducted in the tourist harbour in Pozzuoli Gulf and the second in the Riomaggiore-Manarola marine area within the Cinque Terre territory (Italy). Both surveys yielded promising results regarding the potentiality of CORAL to collect fine-scale submarine elements such as anthropic objects, sedimentary features, and seagrass meadow spots. These capabilities characterize the CORAL system as a highly efficient investigation tool for depicting shallow bedforms, reconstructing coastal dynamics and erosion processes and monitoring the evolution of biological habitats.

Automated River Substrate Mapping From Sonar Imagery With Machine Learning

AbstractKnowledge of the variation and distribution of substrates at large spatial extents in aquatic systems, particularly rivers, is severely lacking, impeding species conservation and ecosystem restoration efforts. Recreation‐grade side‐scan sonar (SSS) instruments have demonstrated their unparalleled value as a low‐cost scientific instrument capable of efficient and rapid imaging of the benthic environment. However, existing methods for generating georeferenced data sets from these instruments, especially substrate maps, remain a barrier of adoption for scientific inquiry due to the high degree of human intervention and required expertise. To address this shortcoming, we introduced PING‐Mapper, an open‐source and freely available Python‐based software for automatically generating geospatial benthic data sets from popular Humminbird® instruments reproducibly. The previously released Version 1.0 of the software provided automated workflows for exporting georeferenced sonar imagery. This study extends functionality with version 2.0 by incorporating semantic segmentation with deep neural networks to reproducibly map substrates at large spatial extents. We present a novel approach for generating label‐ready sonar data sets, creating label‐image training sets, and model training with transfer learning; all with open‐source tools. The six‐class substrate model achieves an overall accuracy of 78% and the best performing class achieves 91%. Grouping substrates into three classes further improves the overall accuracy (87%) and best performing class accuracy (94%). Additional workflows enable masking sonar shadows, calculating independent bed picks and correcting attenuation effects in the imagery to improve interpretability. This software provides an improved mechanism for automatically mapping substrate distribution from recreation‐grade SSS systems, thereby lowering the barrier for inclusion in wider aquatic research.

Underwater Side-Scan Sonar Target Detection: YOLOv7 Model Combined with Attention Mechanism and Scaling Factor

Side-scan sonar plays a crucial role in underwater exploration, and the autonomous detection of side-scan sonar images is vital for detecting unknown underwater environments. However, due to the complexity of the underwater environment, the presence of a few highlighted areas on the targets, blurred feature details, and difficulty in collecting data from side-scan sonar, achieving high-precision autonomous target recognition in side-scan sonar images is challenging. This article addresses this problem by improving the You Only Look Once v7 (YOLOv7) model to achieve high-precision object detection in side-scan sonar images. Firstly, given that side-scan sonar images contain large areas of irrelevant information, this paper introduces the Swin-Transformer for dynamic attention and global modeling, which enhances the model’s focus on the target regions. Secondly, the Convolutional Block Attention Module (CBAM) is utilized to further improve feature representation and enhance the neural network model’s accuracy. Lastly, to address the uncertainty of geometric features in side-scan sonar target features, this paper innovatively incorporates a feature scaling factor into the YOLOv7 model. The experiment initially verified the necessity of attention mechanisms in the public dataset. Subsequently, experiments on our side-scan sonar (SSS) image dataset show that the improved YOLOv7 model has 87.9% and 49.23% in its average accuracy (mAP0.5) and (mAP0.5:0.95), respectively. These results are 9.28% and 8.41% higher than the YOLOv7 model. The improved YOLOv7 algorithm proposed in this paper has great potential for object detection and the recognition of side-scan sonar images.

Side‐Scan Sonar as a Tool For Measuring Fish Populations: Current State of the Science and Future Directions

Side‐scan sonar (SSS) is a powerful tool that can be used to address many key questions in fisheries science. In principle, SSS uses dual transducers to transmit a narrow‐beam, wide‐angle acoustic signal as the survey vessel transits an area. The intensity of reflected sound is recorded to generate an image mosaic comprised of benthic substrates and targets in the water column, including organisms such as fish. Although SSS has been around for decades, recent advancements have opened new opportunities to leverage this technology to directly measure fish populations. In this paper, we review the current state of the science and identify opportunities to further refine SSS for fisheries applications.

SIGAN: A Multi-Scale Generative Adversarial Network for Underwater Sonar Image Super-Resolution

Super-resolution (SR) is a technique that restores image details based on existing information, enhancing the resolution of images to prevent quality degradation. Despite significant achievements in deep-learning-based SR models, their application in underwater sonar scenarios is limited due to the lack of underwater sonar datasets and the difficulty in recovering texture details. To address these challenges, we propose a multi-scale generative adversarial network (SIGAN) for super-resolution reconstruction of underwater sonar images. The generator is built on a residual dense network (RDN), which extracts rich local features through densely connected convolutional layers. Additionally, a Convolutional Block Attention Module (CBAM) is incorporated to capture detailed texture information by focusing on different scales and channels. The discriminator employs a multi-scale discriminative structure, enhancing the detail perception of both generated and high-resolution (HR) images. Considering the increased noise in super-resolved sonar images, our loss function emphasizes the PSNR metric and incorporates the L2 loss function to improve the quality of the output images. Meanwhile, we constructed a dataset for side-scan sonar experiments (DNASI-I). We compared our method with the current state-of-the-art super-resolution image reconstruction methods on the public dataset KLSG-II and our self-built dataset DNASI-I. The experimental results show that at a scale factor of 4, the average PSNR value of our method was 3.5 higher than that of other methods, and the accuracy of target detection using the super-resolution reconstructed images can be improved to 91.4%. Through subjective qualitative comparison and objective quantitative analysis, we demonstrated the effectiveness and superiority of the proposed SIGAN in the super-resolution reconstruction of side-scan sonar images.