Sonar Images Research Articles

Sample Augmentation Method for Side-Scan Sonar Underwater Target Images Based on CBL-sinGAN

The scarcity and difficulty in acquiring Side-scan sonar target images limit the application of deep learning algorithms in Side-scan sonar target detection. At present, there are few amplification methods for Side-scan sonar images, and the amplification image quality is not ideal, which is not suitable for the characteristics of Side-scan sonar images. Addressing the current shortage of sample augmentation methods for Side-scan sonar, this paper proposes a method for augmenting single underwater target images using the CBL-sinGAN network. Firstly, considering the low resolution and monochromatic nature of Side-scan sonar images while balancing training efficiency and image diversity, a sinGAN network is introduced and designed as an eight-layer pyramid structure. Secondly, the Convolutional Block Attention Module (CBAM) is integrated into the network generator to enhance target learning in images while reducing information diffusion. Finally, an L1 loss function is introduced in the network discriminator to ensure training stability and improve the realism of generated images. Experimental results show that the accuracy of shipwreck target detection increased by 4.9% after training with the Side-scan sonar sample dataset augmented by the proposed network. This method effectively retains the style of the images while achieving diversity augmentation of small-sample underwater target images, providing a new approach to improving the construction of underwater target detection models.

AI Explainability and Acceptance: A Case Study for Underwater Mine Hunting

In critical operational context such as Mine Warfare, Automatic Target Recognition (ATR) algorithms are still hardly accepted. The complexity of their decision-making hampers understanding of predictions despite performances approaching human expert ones. Much research has been done in Explainability Artificial Intelligence (XAI) field to avoid this “black box” effect. This field of research attempts to provide explanations for the decision-making of complex networks to promote their acceptability. Most of the explanation methods applied on image classifier networks provide heat maps. These maps highlight pixels according to their importance in decision-making. In this work, we first implement different XAI methods for the automatic classification of Synthetic Aperture Sonar (SAS) images by convolutional neural networks (CNN). These different methods are based on a post hoc approach. We study and compare the different heat maps obtained. Second, we evaluate the benefits and the usefulness of explainability in an operational framework for collaboration. To do this, different user tests are carried out with different levels of assistance, ranging from classification for an unaided operator to classification with explained ATR. These tests allow us to study whether heat maps are useful in this context. The results obtained show that the heat maps explanation has a disputed utility according to the operators. Heat map presence does not increase the quality of the classifications. On the contrary, it even increases the response time. Nevertheless, half of operators see a certain usefulness in heat maps explanation.

Research on the classification of seabed sediments sonar images based on MoCo self-supervised learning

The discrimination of seafloor substrate type is an extremely important part of seafloor science, and the substrate information is of great significance for the development of marine science and the protection of the marine environment. Current sonar equipment can efficiently generate seafloor images and present seafloor information visually, so the seafloor substrate classification technology based on sonar images has become a hot research topic. Convolutional neural network, as one of the most important classification algorithms in seabed substrate sonar image classification, has excellent performance in most cases. However, the size of the convolutional kernel of convolutional neural network limits the global feature extraction ability, and the ability to discriminate global features in sonar images is weak. In addition, seabed substrate sonar images have labelled data acquisition difficulty and high cost, and acoustic seabed substrate classification in practice generally belongs to small sample classification scenarios. Aiming at the above problems, this thesis selects Swin Transformer, which has strong global feature extraction ability, as the classifier, and uses MoCo self-supervised learning to pre-train the unlabeled data in order to achieve better results.

Understanding model fidelity for training synthetic aperture sonar image classifiers

Convolutional neural networks (CNNs) are increasingly employed for classification tasks in automated target recognition (ATR) algorithms for synthetic aperture sonar (SAS) images. Training ATR algorithms, however, requires many unique observations of the targets of interest. Data available for training is often limited, and acquiring additional training data through experimentation is prohibitively expensive. Increasingly, training with simulated data is being considered as an alternative, but the fidelity required of the models that generate this data is not yet known. SAS imagery typically contains significant complexity from countless physical mechanisms. Simulating complex scenes requires multiple models to account for these different effects, but some effects may not require accurate modeling for the purposes of training an ATR algorithm. This presentation will describe a study to investigate the fidelity of models required so that simulated data may be used interchangeably with experimental data for training CNNs. Using in-air experimentation, a high-fidelity data set was developed with multiple degrees of complexity. A high-frequency sonar signal model was then used to generate complementary simulated data. This approach allows for specific physical features in the data to be individually isolated, enabling detailed exploration of the relationships between model fidelity and CNN architecture.

Angular dependence of acoustic scattering statistics for rocky outcrops

The use of automated detection algorithms in undersea remote sensing and target detection allows for decreased timelines and risk to human assets. However, understanding the performance of these systems requires a thorough understanding of the physical processes that affect the acoustic scattering statistics of the sonar image. Many studies have reported on the scattering statistics of the seafloor generally, and recent work has quantified the dependence of scintillation index on range for sandy seafloors. To date, no studies have examined the angular dependence of acoustic scattering statistics for rocky bottoms, which have more complex spatial texture. This paper analyzes synthetic aperture sonar (SAS) images of rocky outcrops imaged near Bergen Norway to quantify the grazing angle dependency of scattering statistics for rocky seafloors. Outcrops are categorized by texture and analyzed using scintillation index and relative scattering strength with respect to both true grazing angle and a flat seafloor assumption. Comparisons between results utilizing calibrated (beampattern removed) and non-calibrated images will be presented to display the complexity of the grazing angle dependence of scattering statistics for rocky seafloors. SAS imagery provided by the Norwegian Defence Research Establishment. [Funding provided by the Office of Naval Research]

Research on Sonar Motion Compensation Method Based on Acoustic Calibration System

The influence of sonar motion error restricts the improvement of sonar imaging resolution. High-precision sonar motion compensation is an important guarantee for high-quality imaging. In this paper, a sonar motion compensation method based on acoustic calibration system is proposed, and the imaging results are compared with those based on the traditional combined inertial guidance measurement method. Experimental results show that the sonar motion compensation method based on acoustic calibration system plays a crucial role in imaging, which proves that acoustic calibration method is an effective means to improve the resolution of sonar imaging.

A model-experiment validation of near-range phase center breakdown

Spatial coherence of a scattered field has helped to improve sensor navigation techniques and field imagery. Under the phase center approximation (PCA), a bistatic sensor geometry is approximated as a monostatic phase center. In the far field, the approximation is valid, and signals with common phase centers are highly coherent. The approximation fails when a non-trivial spatiotemporal delay appears in the sensor’s near-field, and signals that share phase centers are less coherent. A model by Brown and Blanford shows this near-field degradation of PCA, using comparisons between the van Cittert-Zernike Theorem and a point-based scattering model. Experimental validation of the model was performed using an in-air sonar mounted on a linear actuator. The sonar collected returns from a shallow bed filled with plastic beads, which approximates rough interfaces typical of underwater environments. Multiple trials were conducted using a variety of transmitters, receiver placements, and ping intervals. These geometries were simulated using the point-based scattering model and the two sets of data were compared in order to validate the model. Results from this model-data comparison, along with future applications for sonar imaging and navigation, will be discussed.

Bistatic spatial coherence for micronavigation of a volumetric synthetic aperture sonar

Combining multiple passes of a volumetric synthetic aperture sonar (SAS) may open new possibilities for sonar imaging, if the passes can be precisely aligned. Previous work suggests that coherence-based micronavigation is a potential approach to achieve this alignment. However, existing coherence-based techniques require assumptions that break down in many volumetric imaging geometries. This breakdown leads to a loss of coherence that has not been widely studied, and consequently, has hindered the application of micronavigation to volumetric SAS systems. This work will explore the impact of sensing range and bistatic separation on spatial coherence in downward-looking, volumetric sonar applications where the far-field assumption no longer holds. As signals transmitted from downward-looking sonars may penetrate into the sediment floor, this analysis will consider both interface and volumetric scattering. First, using both simulated and experimental sonar data, we will quantify the loss in coherence that occurs in these near-field geometries. Then, methods to regain lost coherence will be examined in an effort to enable spatial coherence-based navigation for a volumetric SAS. Finally, the work’s relevancy for ongoing efforts in acoustic navigation and repeat-pass imaging will be discussed.

Learning the Ego-Motion of an Underwater Imaging Sonar: A Comparative Experimental Evaluation of Novel CNN and RCNN Approaches

Learning the Ego-Motion of an Underwater Imaging Sonar: A Comparative Experimental Evaluation of Novel CNN and RCNN Approaches

Semiparametric constant false alarm rate method for radar and sonar images

AbstractThis work proposed a novel constant false alarm rate detection method based on Gaussian mixture model (GMM). The main contribution contains two aspects. First, the Gabor wavelet is used to determine the GMM component number. Second, the expression of probability of false alarm is derived based on the GMM parameter.

A Novel Technological Approach to Recover Aquatic Research Equipment from Depth

Researchers are increasingly using biologging equipment (e.g., telemetry receivers, temperature loggers) to characterize the ecology of aquatic ecosystems. This equipment is commonly deployed at a wide range of water depths and greatly expands our capacity to remotely monitor aquatic ecosystems; however, equipment retrieval can be a major challenge. Here, we describe a technological solution to this challenge that uses a combination of live imaging sonar and a remotely operated vehicle to efficiently locate and recover equipment across a wide range of conditions (e.g., turbid water, range of water depths). We provide details on our specific equipment setup (total cost < Can$15,000) used for the recovery of acoustic fish tracking receivers moored to the benthos of Stoney Lake, Ontario, Canada. There are some limitations to this approach, which are discussed. With technological advances and increases in affordability of commercially available products, this approach may be widely applicable to recover biologgers from depth.

A blending method for forward-looking sonar mosaicing handling intra- and inter-frame artifacts

Forward-looking sonar (FLS) has been gaining attractions in the realm of near-bottom, close-range underwater inspections, owing to its high-resolution and rapid framerate capabilities. Although automatic target recognition (ATR) algorithms have been tentatively employed for object detection tasks, the necessity for human oversight remains vital, particularly in sensitive areas. A comprehensive FLS mosaic encapsulating all relevant information is highly desired to aid experts in managing an extensive array of perception data. Yet, prior research has often presumed that FLS operates under an ideal system configuration, assuming optimal sonar imaging setups and the availability of precise positioning data. Without these assumptions, intra-frame and inter-frame artifacts would emerge, deteriorating the quality of the resultant mosaic by obscuring essential information. In this study, we propose an innovative blending method specifically for FLS mosaicing aiming preserve important information. We first develop a long–short time sliding window (LST-SW) to rectify the local statistics of raw sonar images. These statistics then serve to create a global variance map (GVM). This step helps to emphasize the useful information contained in images in the blending phase by classifying the informative and featureless pixels, thereby enhancing the quality of final mosaic. This approach is substantiated through data collected in real-world scenarios. The results show that our method can preserve more details in FLS mosaics for human inspection purposes in practice.

Exploration and identification of ancient Roman shipwreck located at Quseir, Red Sea

Recent marine geophysical investigations in front of the Myos Hormos Port (also known as Quseir al-Qadim) on the Red Sea coast of Egypt indicated the existence of an ancient Roman shipwreck, among other features on the seafloor. The study showed that side scan sonar imaging is a powerful technique for classifying the seafloor and detecting submerged targets among other seafloor features with resembling acoustic signatures. Integrating bathymetric data with sonar imageries displayed the seafloor topography in front of the ancient port where reef habitats were dominant. The Autonomous Underwater Vehicle was efficient in mapping closed and near-shore areas like bays. This vehicle provided high-resolution bathymetric and sonar data for the semi-closed bay area located at the entrance of the ancient port. Acoustic measurements were done for the sunken shipwreck using sonar software, and a structural model was established for the ∼ 32 m long sunken ship. The model suggested the presence of a pile of shipload gathered around the shipwreck. Results can contribute to the development of Egypt's tourism industry by promoting underwater cultural heritage preservation, as the discovery of an ancient Roman shipwreck is a significant finding that can attract tourists worldwide.

Open-set recognition with long-tail sonar images

Current sonar image recognition methods excel in closed-set and balanced scenarios, but real underwater data often follow an open-set and long-tailed distribution, leading to misclassifications, especially among tail classes. Although open-set long-tail recognition (OLTR) tasks have received attention in natural images in recent years, there has been a lack of systematic research in sonar images. To address this gap, we present the first comprehensive study and analysis of open-set long-tail recognition in sonar images (Sonar-OLTR). In this paper, we establish a Sonar-OLTR benchmark by introducing the Nankai Sonar Image Dataset (NKSID), a new collection of 2617 real-world forward-looking sonar images. We investigate the challenges posed by long-tail distributions in existing open-set recognition (OSR) evaluation metrics for sonar images and propose two improved evaluation metrics. Using this benchmark, we conduct a thorough examination of state-of-the-art OSR, long-tail recognition, OLTR, and out-of-distribution detection algorithms. Additionally, we propose a straightforward yet effective integrated Sonar-OLTR approach as a new baseline. This method introduces a Push the right Logit Up and the wrong logit Down (PLUD) loss to increase feature space margins between known and unknown classes, as well as head and tail classes within known classes. Extensive experimental evaluation based on the benchmark demonstrates the performance and speed advantages of PLUD, providing insights for future Sonar-OLTR research. The code and dataset are publicly available at https://github.com/Jorwnpay/Sonar-OLTR.

Sonar Image Target Detection Based on Simulated Stain-like Noise and Shadow Enhancement in Optical Images under Zero-Shot Learning

There are many challenges in using side-scan sonar (SSS) images to detect objects. The challenge of object detection and recognition in sonar data is greater than in optical images due to the sparsity of detectable targets. The complexity of real-world underwater scanning presents additional difficulties, as different angles produce sonar images of varying characteristics. This heterogeneity makes it difficult for algorithms to accurately identify and detect sonar objects. To solve these problems, this paper presents a novel method for sonar image target detection based on a transformer and YOLOv7. Thus, two data augmentation techniques are introduced to improve the performance of the detection system. The first technique applies stain-like noise to the training optical image data to simulate the real sonar image environment. The second technique adds multiple shadows to the optical image and 3D data targets to represent the direction of the target in the sonar image. The proposed method is evaluated on a public sonar image dataset, and the experimental results demonstrate that the proposed method outperforms the state-of-the-art methods in terms of accuracy and speed. The experimental results show that our method achieves better precision.

A High-Resolution Imaging Method for Multiple-Input Multiple-Output Sonar Based on Deterministic Compressed Sensing.

Differences between conventional sonar and Multiple-Input Multiple-Output (MIMO) sonar systems arise in achieving high angular and range resolution. MIMO sonar uses Matched Filtering (MF) with well-correlated transmitted signals to enhance spatial resolution by obtaining virtual arrays. However, imperfect correlation characteristics yield high sidelobe values, which hinder accurate target localization in underwater imagery. To address this, a Compressed Sensing (CS) method is proposed by reconstructing echo signals to suppress correlation noise between orthogonal waveforms. A shifted dictionary matrix and a deterministic Discrete Fourier Transform (DFT) measurement matrix are used to multiply received echo signals to yield compressed measurements. A sparse recovery algorithm is applied to optimize signal reconstruction before joint transmit-receive beamforming forms a 2D sonar image in the angle-range domain. Numerical simulations and lake experimental results confirm the effectiveness of the proposed method, by obtaining a lower sidelobe sonar image under sub-Nyquist sampling rates as compared with other approaches.

Three-dimentional reconstruction of underwater side-scan sonar images based on shape-from-shading and monocular depth fusion

Modern marine research requires high-precision three-dimensional (3D) underwater data. Underwater environments experience severe visible light attenuation, which leads to inferior imaging compared with air. In contrast, sound waves are less affected underwater; hence side-scan sonar is used for underwater 3D reconstruction. Typically, the shape-from-shading algorithm (SfS) is widely used to reconstruct surface normal or heights from side-scan sonar images. However, this approach has challenges because of global information loss and noise. To address these issues, this study introduces a surface-normal fusion method. Specifically, we propose a frequency separation SfS algorithm using a discrete cosine transform, which provides a surface-normal map with less noise. We then fuse the surface-normal map with a novel depth estimation network to achieve high-precision 3D reconstruction of underwater side-scan sonar images. We conducted experiments on synthetic, NYU-depth-v2, and real side-scan sonar datasets to demonstrate the effectiveness of the proposed method.

Advanced Underwater Measurement System for ROVs: Integrating Sonar and Stereo Vision for Enhanced Subsea Infrastructure Maintenance

In the realm of ocean engineering and maintenance of subsea structures, accurate underwater distance quantification plays a crucial role. However, the precision of such measurements is often compromised in underwater environments due to backward scattering and feature degradation, adversely affecting the accuracy of visual techniques. Addressing this challenge, our study introduces a groundbreaking method for underwater object measurement, innovatively combining image sonar with stereo vision. This approach aims to supplement the gaps in underwater visual feature detection with sonar data while leveraging the distance information from sonar for enhanced visual matching. Our methodology seamlessly integrates sonar data into the Semi-Global Block Matching (SGBM) algorithm used in stereo vision. This integration involves introducing a novel sonar-based cost term and refining the cost aggregation process, thereby both elevating the precision in depth estimations and enriching the texture details within the depth maps. This represents a substantial enhancement over existing methodologies, particularly in the texture augmentation of depth maps tailored for subaquatic environments. Through extensive comparative analyses, our approach demonstrates a substantial reduction in measurement errors by 1.6%, showing significant promise in challenging underwater scenarios. The adaptability and accuracy of our algorithm in generating detailed depth maps make it particularly relevant for underwater infrastructure maintenance, exploration, and inspection.

AUV SLAM method based on SO-CFAR and ADT feature extraction.

Due to the exceptional detection capabilities, the forward-looking sonar could be adopted in simultaneous localization and mapping (SLAM) for autonomous underwater vehicle (AUVs). This paper primarily investigates the application of the factor graph optimization SLAM algorithm based on feature maps in AUV. It achieves this by combining the smallest of constant false alarm rate (SO-CFAR) and adaptive threshold (ADT) to filter noise from the forward-looking sonar and extract feature point clouds. Furthermore, a weighted iterative closest point (WICP) algorithm is employed for feature point registration, which is extracted from the sonar image. The experimental result based on field data demonstrates that the proposed method, with an 8.52% improvement in root mean square error (RMSE) compared with dead reckoning (DR).

Anti-Interference Bottom Detection Method of Multibeam Echosounders Based on Deep Learning Models

Multibeam echosounders, as the most commonly used bathymetric equipment, have been widely applied in acquiring seabed topography and underwater sonar images. However, when interference occurs in the water column, traditional bottom detection methods may fail, resulting in discontinuities in the bathymetry and distortion in the sonar images. To solve this problem, we propose an anti-interference bottom detection method based on deep learning models. First, the variation differences of backscatter strengths at different incidence angles and the failure conditions of traditional methods were analyzed. Second, the details of our deep learning models are explained. And these models were trained using samples in the specular reflection, scatter reflection, and high-incidence angle regions, respectively. Third, the bottom detection procedures of the along-track and across-track water column data using the trained models are provided. In the experiments, multibeam data with strong interferences in the water column were selected. The bottom detection results of the along-track water column data at incidence angles of 0°, 35°, and 60° and the across-track ping data validated the effectiveness of our method. By comparison, our method acquired the correct bottom position when the traditional methods had inaccurate or even no detection results. Our method can be used to supplement existing methods and effectively improve bathymetry robustness under interference conditions.