Synthetic Aperture Sonar Images Research Articles

AI Explainability and Acceptance: A Case Study for Underwater Mine Hunting

In critical operational context such as Mine Warfare, Automatic Target Recognition (ATR) algorithms are still hardly accepted. The complexity of their decision-making hampers understanding of predictions despite performances approaching human expert ones. Much research has been done in Explainability Artificial Intelligence (XAI) field to avoid this “black box” effect. This field of research attempts to provide explanations for the decision-making of complex networks to promote their acceptability. Most of the explanation methods applied on image classifier networks provide heat maps. These maps highlight pixels according to their importance in decision-making. In this work, we first implement different XAI methods for the automatic classification of Synthetic Aperture Sonar (SAS) images by convolutional neural networks (CNN). These different methods are based on a post hoc approach. We study and compare the different heat maps obtained. Second, we evaluate the benefits and the usefulness of explainability in an operational framework for collaboration. To do this, different user tests are carried out with different levels of assistance, ranging from classification for an unaided operator to classification with explained ATR. These tests allow us to study whether heat maps are useful in this context. The results obtained show that the heat maps explanation has a disputed utility according to the operators. Heat map presence does not increase the quality of the classifications. On the contrary, it even increases the response time. Nevertheless, half of operators see a certain usefulness in heat maps explanation.

Combining range-general and range-specific techniques for late-time focused image reconstructiona).

Synthetic aperture sonar (SAS) is an acoustic method for detecting objects in an environment. Conventional SAS image reconstruction techniques invert a forward model based on geometric scattering and straight-line propagation. Acoustic features that do not fit this model, such as multiple scattering and late-time returns, appear out of focus. This paper describes an image reconstruction technique that selectively applies range-general and range-specific methods to improve the focus of late-time returns while maintaining image quality away from the focal plane. The technique is demonstrated on experimental data and compared with a range-specific algorithm.

A Statistical Evaluation of the Connection between Underwater Optical and Acoustic Images

The use of Synthetic Aperture Sonar (SAS) in autonomous underwater vehicle (AUV) surveys has found applications in archaeological searches, underwater mine detection and wildlife monitoring. However, the easy confusability of natural objects with the target object leads to high false positive rates. To improve detection, the combination of SAS and optical images has recently attracted attention. While SAS data provides a large-scale survey, optical information can help contextualize it. This combination creates the need to match multimodal, optical–acoustic image pairs. The two images are not aligned, and are taken from different angles of view and at different times. As a result, challenges such as the different resolution, scaling and posture of the two sensors need to be overcome. In this research, motivated by the information gain when using both modalities, we turn to statistical exploration for feature analysis to investigate the relationship between the two modalities. In particular, we propose an entropic method for recognizing matching multimodal images of the same object and investigate the probabilistic dependency between the images of the two modalities based on their conditional probabilities. The results on a real dataset of SAS and optical images of the same and different objects on the seafloor confirm our assumption that the conditional probability of SAS images is different from the marginal probability given an optical image, and show a favorable trade-off between detection and false alarm rate that is higher than current benchmarks. For reproducibility, we share our database.

Deep learning‐based time delay estimation for motion compensation in synthetic aperture sonars

AbstractAccurate and robust time delay estimation is crucial for synthetic aperture sonar (SAS) imaging. A two‐step time delay estimation method based on displaced phase centre antenna (DPCA) micronavigation has been widely applied in motion estimation and compensation of SASs. However, the existing methods for time delay estimation are not sufficiently robust, which reduces the performance of SAS motion estimation. Deep learning is currently one of the cutting‐edge techniques and has brought about a remarkable progress in the field of underwater acoustic signal processing. In this study, a deep learning‐based time delay estimation method is introduced to SAS motion estimation and compensation. The subband processing is first applied to obtain ambiguous time delays between adjacent pings from phases of SAS echoes. Then, a lightweight neural network is utilised to construct phase unwrapping. The model of the employed neural network is trained with simulation data and applied to real SAS data. The results of time delay estimation and motion compensation demonstrate that the proposed neural network‐based method has much better performance than the two‐step and joint‐subband methods.

Exploitation of SAS interferometric phase statistics for coherent change detection

AbstractCoherent change detection (CCD) takes advantage of the interferometric processing of synthetic aperture sonar (SAS) images and enables the detection of small scale changes to the scene between image pairs. This paper builds upon interferometric phase statistics developed for synthetic aperture radar and presents a method to differentiate between possible sources of decorrelation. A single‐pass pair of SAS images is categorized into four zones based on acoustic amplitude and coherence properties. Results suggest that regions of low coherence caused by a scene change can be differentiated from regions of low coherence due to other sources of decorrelation. Phase statistics could provide a technique to reduce the false alarm rate in CCD. Further studies are required to validate the approach for repeat‐pass SAS imagery and consider the effects of temporal decorrelation and misregistration errors on the phase distributions.

A wide-beam NCS algorithm for multi-receiver SAS based on azimuth spectrum superposition

The existing multi-receiver synthetic aperture sonar (SAS) imaging algorithms are suitable for narrow-beam width, which will lead to a decrease in imaging quality under wide-beam condition and are not in line with the development needs of SAS. We propose a non-linear chirp scaling algorithm (NCSA) for wide beam multi-receiver SAS. Firstly, the point target reference spectrum (PTRS) of each receiver is obtained by the Lagrange inversion theorem (LIT), and then the under-sampled signal in the azimuth frequency domain is obtained through azimuth spectrum extension; Then, considering the cubic term of range frequency in the PTRS and the linear variation of equivalent frequency modulation slope with range, each receiver is imaged using the NCSA, and coherent superposition is performed in the azimuth frequency domain to eliminate spectrum aliasing caused by azimuth spectrum extension; Finally, the azimuth inverse transform is performed on the superimposed signal to obtain the focusing imaging. Computer simulation experiments and field data verify that this method is superior to the existing SAS imaging algorithm, improving the quality of wide-beam imaging, avoiding the interpolation operation of the traditional range-Doppler algorithm, and saving computation cost.

Tank test results of synthetic aperture sonar image reconsturction using two frequency bands

Synthetic aperture experiments have been done in water tanks using two different frequency bands. Their center frequencies are around 25 kHz and 300 kHz, respectively. The lower frequency system has eight equally spaced receiver channels with one wavelength distance and the higher frequency one has sixteen receiver channels with about seven wavelength spacing. Five different types of structures were selected as sonar targets. They are sphere, cylinder, octahedron-shape structure, star-shape structure, and an in-plane grid target. The sphere and cylinder were floating in the water column while the others were located on the tank bottoms. As the transmitting signals linear frequency modulation was applied. Sonar images were reconstructed by synthetic aperture processing and the deviated motion from the perfect rectilinear trajectory due to the vibration of the vertical support was compensated by using motion sensor data and receiver signals. Synthetic aperture images were experimentally generated using the two different frequency bands and compared. [This research was supported by KIMST funded by the Agency of Korea Coast Guard (KIMST-20210547).]

Underwater object classification combining SAS and transferred optical-to-SAS Imagery

Combining synthetic aperture sonar (SAS) imagery with optical images for underwater object classification has the potential to overcome challenges such as water clarity, the stability of the optical image analysis platform, and strong reflections from the seabed for sonar-based classification. In this work, we propose this type of multi-modal combination to discriminate between man-made targets and objects such as rocks or litter. We offer a novel classification algorithm that overcomes the problem of intensity and object formation differences between the two modalities. To this end, we develop a novel set of geometrical shape descriptors that takes into account the geometrical relation between the object’s shadow and highlight. Results from 7,052 pairs of SAS and optical images collected during several sea experiments show improved classification performance compared to the state-of-the-art for better discrimination between different types of underwater objects. For reproducability, we share our database.

Neural Volumetric Reconstruction for Coherent Synthetic Aperture Sonar

Synthetic aperture sonar (SAS) measures a scene from multiple views in order to increase the resolution of reconstructed imagery. Image reconstruction methods for SAS coherently combine measurements to focus acoustic energy onto the scene. However, image formation is typically under-constrained due to a limited number of measurements and bandlimited hardware, which limits the capabilities of existing reconstruction methods. To help meet these challenges, we design an analysis-by-synthesis optimization that leverages recent advances in neural rendering to perform coherent SAS imaging. Our optimization enables us to incorporate physics-based constraints and scene priors into the image formation process. We validate our method on simulation and experimental results captured in both air and water. We demonstrate both quantitatively and qualitatively that our method typically produces superior reconstructions than existing approaches. We share code and data for reproducibility.

A simulation method on target strength and circular SAS imaging of X-rudder UUV including multiple acoustic scattering

Target strength (TS) and circular synthetic aperture sonar (CSAS) images provide essential information for active acoustic detection and recognition of non-cooperative unmanned undersea vehicles (UUVs), which pose a significant threat to underwater preset facilities. To access them, we propose an iterative physical acoustics (IPA)-based method to simulate the multiple acoustic scattered fields on rigid surfaces in high-frequency cases. It uses the Helmholtz integral equation with an appropriate Green's function in terms of the Neumann series, and then incorporates the ideas of triangulation and iteration into a numerical implementation. Then two approximate analytic formulae with precise physical meanings are derived to predict the TS and CSAS images of concave targets, respectively. There are no restrictions on the surface's curvature and the order of multiple scattering. The method is validated against the finite element method (FEM) for acoustic scattering from a sphere segment and against an experiment involving an X-rudder UUV's stern. On this basis, we simulate and analyze the TS and CSAS images of an X-rudder UUV. In addition, the influence of the angle of adjacent rudders on the multiple scattering characteristics is discussed. Results show that this method can potentially predict accurate UUV features, especially the multiple scattered features.

Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation

Deep learning has enabled significant improvements in semantic image segmentation, especially in underwater imaging domains such as side scan sonar (SSS). In this work, we apply deep learning to synthetic aperture sonar (SAS) imagery, which has an advantage over traditional SSS in which SAS produces coherent high- and constant-resolution imagery. Despite the successes of deep learning, one drawback is the need for abundant labeled training data to enable success. Such abundant labeled data are not always available as in the case of SAS where collections are expensive and obtaining quality ground-truth labels may require diver intervention. To overcome these challenges, we propose a domain-specific deep learning network architecture utilizing a unique property to complex-valued SAS imagery: the ability to resolve angle-of-arrival (AoA) of acoustic returns through <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -space processing. By sweeping through consecutive incrementally advanced AoA bandpass filters (a process sometimes referred to as multi-look processing), this technique generates a sequence of images emphasizing angle-dependent seafloor scattering and motion from biologics along the seafloor or in the water column. Our proposal, which we call multi-look sequence processing network (MLSP-Net), is a domain-enriched deep neural network architecture that models the multi-look image sequence using a recurrent neural network (RNN) to extract robust features suitable for semantic segmentation of the seafloor without the need for abundant training data. Unlike previous segmentation works in SAS, our model ingests a complex-valued SAS image and affords the ability to learn the AoA filters in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -space as part of the training procedure. We show the results on a challenging real-world SAS database, and despite the lack of abundant training data, our proposed method shows superior results over state-of-the-art techniques.

Texture-Based Seafloor Characterization Using Gaussian Process Classification

Synthetic aperture sonar (SAS) allows for high-resolution imagery of the seafloor with a quality that approaches that of an optical camera. Identifying seabed types within SAS images can prove very useful in a setting where autonomous underwater vehicles (AUVs) are used, since information about the bottom type can be used to determine the behavior of the AUV. In this article, we present a new algorithm for the automatic detection of different seabed types based on SAS image texture. Using a Gaussian process classification (GPC) model, we were able to predict the seabed type, as well as an uncertainty estimate on this prediction. The GPC-based model is able to learn the most informative textures from a training data set, and by construction is robust against outliers or data points that are very different from those in the training set. An additional benefit of the model is that it can be easily configured to hit the desired tradeoff between computational complexity and accuracy.

Target Detection and Segmentation in Circular-Scan Synthetic Aperture Sonar Images Using Semisupervised Convolutional Encoder–Decoders

We propose a framework for saliency-based, multi-target detection and segmentation of circular-scan, synthetic-aperture-sonar (CSAS) imagery. Our framework relies on a multi-branch, convolutional encoder-decoder network (MB-CEDN). The encoder portion of the MB-CEDN extracts visual contrast features from CSAS images. These features are fed into dual decoders that perform pixel-level segmentation to mask targets. Each decoder provides different perspectives as to what constitutes a salient target. These opinions are aggregated and cascaded into a deep-parsing network to refine the segmentation. We evaluate our framework using real-world CSAS imagery consisting of five broad target classes. We compare against existing approaches from the computer-vision literature. We show that our framework outperforms supervised, deep-saliency networks designed for natural imagery. It greatly outperforms unsupervised saliency approaches developed for natural imagery. This illustrates that natural-image-based models may need to be altered to be effective for this imaging-sonar modality.

INVESTIGATION ON THE CONSTRUCTION OF THE RECEIVED SIGNAL MODEL FOR MULTI-RECEIVER SYNTHETIC APERTURE SONAR IN VIETNAM’S SEA

In this article,the received signal models are constructed for multi-receiver synthetic aperture sonar (SAS) using CW pulses and LFM pulses when considering the change of acoustic velocity in seawater and the Doppler effect. With a more complete description of the physical processes than the traditional models and the mathematical representation of signals, the proposed model can provide data sets for the research and thedevelopment of SAS imaging algorithms to save time and cost in actual data collection. The simulation results illustrate the characteristics of the signals generated from the proposed model with real data of acoustic velocity in Vietnam’s sea.

Modeling the effect of random roughness on synthetic aperture sonar image statistics.

A model has been developed to predict the effect of random seafloor roughness on synthetic aperture sonar (SAS) image statistics, based on the composite roughness approximation-a physical scattering model. The continuous variation in scattering strength produced by a random slope field is treated as an intensity scaling on the image speckle produced by the coherent SAS imaging process. Changes in image statistics caused by roughness are quantified in terms of the scintillation index (SI). Factors influencing the SI include the seafloor slope variance, geo-acoustic properties of the seafloor, the probability density function describing the speckle, and the signal-to-noise ratio. Example model-data comparisons are shown for SAS images taken at three different sites using three different high-frequency SAS systems. Agreement between the modeled and measured SI show that it is possible to link range-dependent image statistics to measurable geo-acoustic properties, providing the foundation necessary for solving problems related to the detection of targets using high-frequency imaging sonars, including performance prediction or adaptation of automated detection algorithms. Additionally, this work illustrates the possible use of SAS systems for remote sensing of roughness parameters such as root mean square slope or height.

Challenges of Labelling Unknown Seabed Munition Dumpsites from Acoustic and Optical Surveys: A Case Study at Skagerrak

The disposal of unexploded ordnance (UXOs) at sea is a global problem. The mapping and remediation of historic UXOs can be assisted by autonomous underwater vehicles (AUVs) carrying sensor payloads such as synthetic aperture sonar (SAS) and optical cameras. AUVs can image large areas of the seafloor in high resolution, motivating an automated approach to UXO detection. Modern methods commonly use supervised machine learning which requires labelled examples from which to learn. This work investigates the often-overlooked labelling process and resulting dataset using an example historic UXO dumpsite at Skagerrak. A counterintuitive finding of this work is that optical images cannot be relied on for ground truth as a significant number of UXOs visible in SAS images are not in optical images, presumed buried. Given the lack of ground truth, we use an ordinal labelling scheme to incorporate a measure of labeller uncertainty. We validate this labelling regime by quantifying label accuracy compared to optical labels with high confidence. Using this approach, we explore different taxonomies and conclude that grouping objects into shells, bombs, debris, and natural gave the best trade-off between accuracy and discrimination.

Three-Dimensional Imaging of Circular Array Synthetic Aperture Sonar for Unmanned Surface Vehicle

Synthetic aperture sonar (SAS) and interferometric synthetic aperture sonar (InSAS) have a range layover phenomenon during underwater observation, the AUV-mounted circular synthetic aperture sonar (CSAS) system, that insonifies targets using multiple circular scans that vary in height and can eliminate the layover phenomenon. However, this observation method is time-consuming and difficult to compensate. To solve this problem, the circular array synthetic aperture sonar (CASAS) based on the equivalent phase center was established for unmanned surface vehicles. Corresponding to the echo signal model of circular array synthetic aperture sonar, a novel three-dimensional imaging algorithm was derived. Firstly, the echo datacube was processed by signal calibration with near-field to far-field transformation and grid interpolation. Then, the sparse recover method was adopted to achieve the scattering coefficient in the height direction by sparse Bayesian learning. Thirdly, the Fourier slice theorem was adopted to obtain the 2D image of the ground plane. After the reconstruction of all height slice cells was accomplished, the final 3D image was obtained. Numerical simulations and experiments using the USV-mounted CASAS system were performed. The imaging results verify the effectiveness of the 3D imaging algorithm for the proposed model and validate the feasibility of CASAS applied in underwater target imaging and detection.

Frequency‐domain multireceiver synthetic aperture sonar imagery with Chebyshev polynomials

Based on Chebyshev polynomials, the two-way slant range is first approximated by the fourth-order polynomial approximation. With the series reversion method, the point target reference spectrum (PTRS) which is further used for the development of imaging algorithm in this letter is obtained. The presented method processes the echoed signal corresponding to each transmitter/receiver pair. And then a high-resolution synthetic aperture sonar (SAS) image is obtained by coherently superposing all coarse-resolution images. Comparing the peak sidelobe ratio (PSLR) and integration sidelobe ratio (ISLR), the performance of the presented method is superior to that of the traditional method. Simulations are further validated by the presented method.

Modeling the effects of internal waves on synthetic aperture sonar resolution

Synthetic aperture sonar (SAS) beamforming is performed under the assumption of a constant sound speed within the water column or by utilizing a measured sound speed profile. Water column features, such as internal waves or internal wave boluses that cause a sound speed structure different than that assumed, can break the constant sound speed assumption resulting in image defocusing and loss of resolution. The resultant loss of resolution can be quantified using the point spread function (PSF). An ellipse fitting technique was applied to SAS images collected by the Norwegian Defence Research Establishment (FFI) using the HISAS system. In conjunction with the analysis conducted on real data, a ray tracing method was applied to model the broadening effects on the PSF for sound speed differences caused by internal wave boluses. A wavenumber beamforming algorithm was also applied to artificial echo data, in which sound speed errors were introduced. It is hoped that the two models can be used in conjunction with SAS data to determine the sound speed within an internal wave feature from the observed loss in resolution.

Remote Sensing of the Tautra Ridge: An Overview of the World’s Shallowest Cold-Water Coral Reefs

On the Tautra Ridge – a 39-100 m deep morainic sill located in the middle of the Trondheimsfjord, Norway – some of the world’s shallowest known occurrences of the scleractinian cold-water coral (CWC) Desmophyllum pertusum can be found. The earliest D. pertusum records from the Tautra Ridge date back to the 18th century, and since then, the location has provided easy access to physical coral specimens for numerous scientific studies. In 2013, the ridge was declared a marine protected area by the Norwegian Government due to its unique CWC reefs. However, few attempts have to our knowledge yet been made to characterize the distribution, extent and condition of these reefs extensively. The aim of the current study was therefore to add geospatial context to the Tautra CWC reef complex. In the study, data from multibeam echo sounding, synthetic aperture sonar imaging and underwater hyperspectral imaging are used to assess CWC reef occurrences from multiple perspectives. The study demonstrates how complementary remote sensing techniques can be used to increase knowledge generation during seafloor mapping efforts. Ultimately, predictive modeling based on seafloor geomorphometry is used to estimate both distribution and areal coverage of D. pertusum reefs along the majority of the Tautra Ridge. Our findings suggest that D. pertusum reef distribution on the Tautra Ridge is affected by several geomorphometric seafloor properties, and that the total reef extent in the area likely is close to 0.64 km2. Better description of current patterns across the Tautra Ridge will improve our understanding of the interaction between hydrography and geomorphology at the Tautra CWC reef complex in the future.