Synthetic Aperture Sonar Imagery Research Articles

Understanding model fidelity for training synthetic aperture sonar image classifiers

Convolutional neural networks (CNNs) are increasingly employed for classification tasks in automated target recognition (ATR) algorithms for synthetic aperture sonar (SAS) images. Training ATR algorithms, however, requires many unique observations of the targets of interest. Data available for training is often limited, and acquiring additional training data through experimentation is prohibitively expensive. Increasingly, training with simulated data is being considered as an alternative, but the fidelity required of the models that generate this data is not yet known. SAS imagery typically contains significant complexity from countless physical mechanisms. Simulating complex scenes requires multiple models to account for these different effects, but some effects may not require accurate modeling for the purposes of training an ATR algorithm. This presentation will describe a study to investigate the fidelity of models required so that simulated data may be used interchangeably with experimental data for training CNNs. Using in-air experimentation, a high-fidelity data set was developed with multiple degrees of complexity. A high-frequency sonar signal model was then used to generate complementary simulated data. This approach allows for specific physical features in the data to be individually isolated, enabling detailed exploration of the relationships between model fidelity and CNN architecture.

Angular dependence of acoustic scattering statistics for rocky outcrops

The use of automated detection algorithms in undersea remote sensing and target detection allows for decreased timelines and risk to human assets. However, understanding the performance of these systems requires a thorough understanding of the physical processes that affect the acoustic scattering statistics of the sonar image. Many studies have reported on the scattering statistics of the seafloor generally, and recent work has quantified the dependence of scintillation index on range for sandy seafloors. To date, no studies have examined the angular dependence of acoustic scattering statistics for rocky bottoms, which have more complex spatial texture. This paper analyzes synthetic aperture sonar (SAS) images of rocky outcrops imaged near Bergen Norway to quantify the grazing angle dependency of scattering statistics for rocky seafloors. Outcrops are categorized by texture and analyzed using scintillation index and relative scattering strength with respect to both true grazing angle and a flat seafloor assumption. Comparisons between results utilizing calibrated (beampattern removed) and non-calibrated images will be presented to display the complexity of the grazing angle dependence of scattering statistics for rocky seafloors. SAS imagery provided by the Norwegian Defence Research Establishment. [Funding provided by the Office of Naval Research]

Deep Adaptive Phase Learning: Enhancing Synthetic Aperture Sonar Imagery Through Learned Coherent Autofocus

Deep Adaptive Phase Learning: Enhancing Synthetic Aperture Sonar Imagery Through Learned Coherent Autofocus

Exploitation of SAS interferometric phase statistics for coherent change detection

AbstractCoherent change detection (CCD) takes advantage of the interferometric processing of synthetic aperture sonar (SAS) images and enables the detection of small scale changes to the scene between image pairs. This paper builds upon interferometric phase statistics developed for synthetic aperture radar and presents a method to differentiate between possible sources of decorrelation. A single‐pass pair of SAS images is categorized into four zones based on acoustic amplitude and coherence properties. Results suggest that regions of low coherence caused by a scene change can be differentiated from regions of low coherence due to other sources of decorrelation. Phase statistics could provide a technique to reduce the false alarm rate in CCD. Further studies are required to validate the approach for repeat‐pass SAS imagery and consider the effects of temporal decorrelation and misregistration errors on the phase distributions.

Uncrewed undersea vehicle embedded autonomous target recognition in synthetic aperture sonar imagery

The opacity of the ocean to electromagnetic signals makes communication with Uncrewed Undersea Vehicles (UUVs) extremely difficult, especially for situations where the vehicle is operating far from its center of command and control, and makes mid-operation data exfiltration for external processing untenable. This leaves operator intervention and control limited necessitating the need for advanced and adaptive autonomy that can create situational awareness from mission data provided by vehicle sensors. Long range high-resolution imagery generated by synthetic aperture sonar (SAS) is a high fidelity data product that can be further processed by both machine learning techniques and traditional image processing techniques to create computer aided detection and classification (CAD/CAC) capabilities to feed that developing autonomy need. This work examines two separate detection and classification pipelines that have been used in parallel to create contact sets that are actionable for onboard decision processes.

Underwater object classification combining SAS and transferred optical-to-SAS Imagery

Combining synthetic aperture sonar (SAS) imagery with optical images for underwater object classification has the potential to overcome challenges such as water clarity, the stability of the optical image analysis platform, and strong reflections from the seabed for sonar-based classification. In this work, we propose this type of multi-modal combination to discriminate between man-made targets and objects such as rocks or litter. We offer a novel classification algorithm that overcomes the problem of intensity and object formation differences between the two modalities. To this end, we develop a novel set of geometrical shape descriptors that takes into account the geometrical relation between the object’s shadow and highlight. Results from 7,052 pairs of SAS and optical images collected during several sea experiments show improved classification performance compared to the state-of-the-art for better discrimination between different types of underwater objects. For reproducability, we share our database.

Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation

Deep learning has enabled significant improvements in semantic image segmentation, especially in underwater imaging domains such as side scan sonar (SSS). In this work, we apply deep learning to synthetic aperture sonar (SAS) imagery, which has an advantage over traditional SSS in which SAS produces coherent high- and constant-resolution imagery. Despite the successes of deep learning, one drawback is the need for abundant labeled training data to enable success. Such abundant labeled data are not always available as in the case of SAS where collections are expensive and obtaining quality ground-truth labels may require diver intervention. To overcome these challenges, we propose a domain-specific deep learning network architecture utilizing a unique property to complex-valued SAS imagery: the ability to resolve angle-of-arrival (AoA) of acoustic returns through <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -space processing. By sweeping through consecutive incrementally advanced AoA bandpass filters (a process sometimes referred to as multi-look processing), this technique generates a sequence of images emphasizing angle-dependent seafloor scattering and motion from biologics along the seafloor or in the water column. Our proposal, which we call multi-look sequence processing network (MLSP-Net), is a domain-enriched deep neural network architecture that models the multi-look image sequence using a recurrent neural network (RNN) to extract robust features suitable for semantic segmentation of the seafloor without the need for abundant training data. Unlike previous segmentation works in SAS, our model ingests a complex-valued SAS image and affords the ability to learn the AoA filters in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -space as part of the training procedure. We show the results on a challenging real-world SAS database, and despite the lack of abundant training data, our proposed method shows superior results over state-of-the-art techniques.

Image-to-Height Domain Translation for Synthetic Aperture Sonar

Synthetic aperture sonar (SAS) intensity statistics are dependent upon the sensing geometry at the time of capture. Estimating bathymetry from acoustic surveys is challenging. While several methods have been proposed to estimate seabed relief via intensity, we develop the first large-scale study that relies on deep learning models. In this work, we pose bathymetric estimation from SAS surveys as a domain translation problem of translating intensity to height. Since no dataset of coregistered seabed relief maps and sonar imagery previously existed to learn this domain translation, we produce the first large simulated dataset containing coregistered pairs of seabed relief and intensity maps from two unique sonar data simulation techniques. We apply four types of models, with varying complexity, to translate intensity imagery to seabed relief: a shape-from-shading (SFS) approach, a Gaussian Markov random field (GMRF) approach, a conditional Generative Adversarial Network (cGAN), and UNet architectures. Each model is applied to datasets containing sand ripples, rocky, mixed, and flat sea bottoms. Methods are compared in reference to the coregistered simulated datasets using L1 error. Additionally, we provide results on simulated and real SAS imagery. Our results indicate that the proposed UNet architectures outperform an SFS, a GMRF, and a pix2pix cGAN model.

Survey on deep learning based computer vision for sonar imagery

Research on the automatic analysis of sonar images has focused on classical, i.e. non deep learning based, approaches for a long time. Over the past 15 years, however, the application of deep learning in this research field has constantly grown. This paper gives a broad overview of past and current research involving deep learning for feature extraction, classification, detection and segmentation of sidescan and synthetic aperture sonar imagery. Most research in this field has been directed towards the investigation of convolutional neural networks (CNN) for feature extraction and classification tasks, with the result that even small CNNs with up to four layers outperform conventional methods.The purpose of this work is twofold. On one hand, due to the quick development of deep learning it serves as an introduction for researchers, either just starting their work in this specific field or working on classical methods for the past years, and helps them to learn about the recent achievements. On the other hand, our main goal is to guide further research in this field by identifying main research gaps to bridge. We propose to leverage the research in this field by combining available data into an open source dataset as well as carrying out comparative studies on developed deep learning methods.

Frequency‐domain multireceiver synthetic aperture sonar imagery with Chebyshev polynomials

Based on Chebyshev polynomials, the two-way slant range is first approximated by the fourth-order polynomial approximation. With the series reversion method, the point target reference spectrum (PTRS) which is further used for the development of imaging algorithm in this letter is obtained. The presented method processes the echoed signal corresponding to each transmitter/receiver pair. And then a high-resolution synthetic aperture sonar (SAS) image is obtained by coherently superposing all coarse-resolution images. Comparing the peak sidelobe ratio (PSLR) and integration sidelobe ratio (ISLR), the performance of the presented method is superior to that of the traditional method. Simulations are further validated by the presented method.

Model validation for simulated synthetic aperture sonar time series data

Synthetic aperture sonar (SAS) systems have grown in popularity due to their ability to create high-resolution sonar imagery with a high area coverage rate. Researchers continue to find new uses for SAS data products. To support the development of new uses and algorithms, it is important to be able to accurately simulate SAS systems with synthetic time series data. This simulated data can be used for many purposes, including estimating system signal-to-noise ratio and performance in a variety of environments, testing signal processing algorithms, and training algorithms to recognize objects in SAS imagery. For these processing tasks to be successful, the simulator must accurately represent physical phenomena in the oceanic underwater environment, which can be extremely complex. This presentation will outline a process for validation and verification of a high-frequency SAS time series simulator, and discuss several useful tests for validating model fidelity. Two fundamental concepts will be explored in depth: the sonar equation and spatial coherence theory. The sonar equation is important because it accounts for correct implementation of multiple lower level models such as geometry, beam patterns, propagation, and seafloor scattering strength. Spatial coherence is important because it is relied on to form coherent imagery.

Iterative, Deep Synthetic Aperture Sonar Image Segmentation

Synthetic aperture sonar (SAS) systems produce high-resolution images of the seabed environment. Moreover, deep learning has demonstrated superior ability in finding robust features for automating imagery analysis. However, the success of deep learning is conditioned on having lots of labeled training data, but obtaining generous pixel-level annotations of SAS imagery is often practically infeasible. This challenge has thus far limited the adoption of deep learning methods for SAS segmentation. Algorithms exist to segment SAS imagery in an unsupervised manner, but they lack the benefit of state-of-the-art learning methods and the results present significant room for improvement. In view of the above, we propose a new iterative algorithm for unsupervised SAS image segmentation combining superpixel formation, deep learning, and traditional clustering methods. We call our method Iterative Deep Unsupervised Segmentation (IDUS). IDUS is an unsupervised learning framework that can be divided into four main steps: 1) A deep network estimates class assignments. 2) Low-level image features from the deep network are clustered into superpixels. 3) Superpixels are clustered into class assignments (which we call pseudo-labels) using $k$-means. 4) Resulting pseudo-labels are used for loss backpropagation of the deep network prediction. These four steps are performed iteratively until convergence. A comparison of IDUS to current state-of-the-art methods on a realistic benchmark dataset for SAS image segmentation demonstrates the benefits of our proposal even as the IDUS incurs a much lower computational burden during inference (actual labeling of a test image). Finally, we also develop a semi-supervised (SS) extension of IDUS called IDSS and demonstrate experimentally that it can further enhance performance while outperforming supervised alternatives that exploit the same labeled training imagery.

Coherence-Induced Bias Reduction in Synthetic Aperture Sonar Along-Track Micronavigation

Subwavelength motion estimation is vital for the production of focused synthetic aperture sonar (SAS) imagery. The required precision is obtainable from the sonar data itself through a process termed micronavigation. Along-track micronavigation is achieved by a similar technique to that used in correlation velocity logs (CVLs), where sparse estimates of the spatial coherence function are interpolated to estimate the location of the peak coherence and hence estimate the interping vehicle motion. However, along-track micronavigation estimates made using this technique are biased, which limits the utility of these measurements for long-term navigation of autonomous underwater vehicles (AUVs). Three sources of along-track motion estimation bias are considered in this article. First, imperfect temporal registration between the signals results in coherence estimates that are negatively biased as a function of the temporal offset. Second, the sparse estimates of the spatial coherence function are obtained by cross-correlation of complex baseband signals, a process which is known to result in positively biased coherence estimates, especially when the true coherence is low. Finally, mismatches between the underlying spatial coherence function and the interpolation kernel used to estimate the peak coherence location also result in along-track micronavigation bias. In this article, we describe and evaluate three methods for reducing along-track micronavigation bias. We introduce a temporal registration of the signals before coherence estimation, which reduces the impact of negative coherence bias due to temporal offsets. The remaining coherence estimation bias is reduced by combining multiple coherence estimates in a Bayesian coherence estimator. Additionally, an improved interpolation kernel is derived with a significantly improved fit compared to the current gold standard Gaussian interpolation kernel. The improvements in along-track micronavigation accuracy are demonstrated using two simulated data sets, which both allow comparison with ground truth. The first involves direct simulation of the spatial coherence from a given interping geometry using the pulse-echo formulation of the van Cittert–Zernike theorem, while the second involves simulation of raw sonar echo data using a point-scatterer model. Using these simulations, a reduction in along-track micronavigation bias of 48.5%–99.5% is demonstrated, with reductions in along-track micronavigation error standard deviation of up to 34%. This improvement expands the potential for SAS-equipped AUVs to reduce their long-term navigation drift, facilitating longer underwater transits, improved target localization, and reduced track misalignment in repeat-pass operations.

Environmentally adaptive automated recognition of underwater mines with synthetic aperture sonar imagery.

This work demonstrates that automated mine countermeasure (MCM) tasks are greatly facilitated by characterizing the seafloor environment in which the sensors operate as a first step within a comprehensive strategy for how to exploit information from available sensors, multiple detector types, measured features, and target classifiers, depending on the specific seabed characteristics present within the high-frequency synthetic aperture sonar (SAS) imagery used to perform MCM tasks. This approach is able to adapt as environmental characteristics change and includes the ability to recognize novel seabed types. Classifiers are then adaptively retrained through active learning in these unfamiliar seabed types, resulting in improved mitigation of challenging environmental clutter as it is encountered. Further, a segmentation constrained network algorithm is introduced to enable enhanced generalization abilities for recognizing mine-like objects from underrepresented environments within the training data. Additionally, a fusion approach is presented that allows the combination of multiple detectors, feature types spanning both measured expert features and deep learning, and an ensemble of classifiers for the particular seabed mixture proportions measured around each detected target. The environmentally adaptive approach is demonstrated to provide the best overall performance for automated mine-like object recognition.

Structural Prior Driven Regularized Deep Learning for Sonar Image Classification

Deep learning has been recently shown to improve performance in the domain of synthetic aperture sonar (SAS) image classification. Given the constant resolution with a range of SAS, it is no surprise that deep learning techniques perform so well. Despite deep learning’s recent success, there are still compelling open challenges in reducing the high false alarm rate and enabling success when training imagery is limited, which is a practical challenge that distinguishes the SAS classification problem from standard image classification set-ups where training imagery may be abundant. We address these challenges by exploiting prior knowledge that humans use to grasp the scene. These include unconscious elimination of the image speckle and localization of objects in the scene. We introduce a new deep learning architecture that incorporates these priors with the goal of improving automatic target recognition (ATR) from SAS imagery. Our proposal—called SPDRDL, structural prior driven regularized deep learning—incorporates the previously mentioned priors in a multitask convolutional neural network (CNN) and requires no additional training data when compared to traditional SAS ATR methods. Two structural priors are enforced via regularization terms in the learning of the network: 1) structural similarity prior—enhanced imagery (often through despeckling) aids human interpretation and is semantically similar to the original imagery and 2) structural scene context priors—learned features ideally encapsulate target centering information; hence learning may be enhanced via a regularization that encourages fidelity against known ground truth target shifts (relative target position from scene center). Experiments on a challenging real-world data set reveal that SPDRDL outperforms state-of-the-art deep learning and other competing methods for SAS image classification.

Alternative representations and object classification of circular synthetic aperture in-air acoustic data

Synthetic aperture sonar imagery is typically generated using data collected with unmanned underwater vehicles. The prohibitive cost of collecting underwater data and the need for well-controlled factors such as collection geometry and object configuration has provided the motivation for devising a benchtop in-air circular acoustic data collection framework. This set-up makes it practically feasible to explore a multitude of parameters that are not as feasible with underwater measurement scenarios, including waveform type, object shapes and material. It is also practically feasible to explore various representations of the collected acoustic data that help better emphasize different aspects of the information embedded in the acoustic signal, which various machine learning algorithms can utilize. Signal processing and feature organization are critical to improving performance of machine learning algorithms. For example, geometric scattering response of objects is well-represented in spatial imagery with sharp contrast of pixel intensity between the object and surrounding environment, while spatial spectrum of the complex SAS image better represents the aspect-dependent spectral response of the object that help discriminate objects of the same shape, but with different material. We will discuss the relationship between the choice of representation and discriminatory information with illustrative classification problems.

Enhanced automated recognition of underwater mine-like objects through environmentally adaptive fusion of detectors, features, and classifiers

The complexity of the natural underwater environment creates a challenging arena in which to find underwater mines. In this work, we demonstrate that automated mine-like object detection tasks are greatly facilitated by a comprehensive fusion process. Our approach begins with characterization of the seafloor based on textures within synthetic aperture sonar (SAS) imagery and uses this to exploit information from the available sensors, multiple detector types, measured features, and target classifiers, to facilitate mine-like object recognition. Our approach is able to adapt as environmental characteristics change, including the ability to recognize novel seabed types. We then adaptively retrain classifiers through active learning in these novel seabed types resulting in improved mitigation of challenging environmental clutter as it is encountered, and develop a segmentation constrained network (SCN) algorithm which enables increased generalization abilities for recognizing mine-like objects in both under-represented and novel, unseen environments in available training data. Additionally, we present a fusion approach that allows us to combine multiple detectors, feature types spanning both measured expert features and deep learning, and an ensemble of classifiers, for the particular seabed mixture proportions measured around each detected target. [Work supported by the Office of Naval Research.]

On the Use of Tiny Convolutional Neural Networks for Human-Expert-Level Classification Performance in Sonar Imagery

Efficient convolutional neural networks (CNNs) are designed and trained for an underwater target classification task with synthetic aperture sonar (SAS) imagery collected at sea. The main contribution is demonstrating that classification performance that matches, and even surpasses, the level achievable by a human domain expert obtained from tiny CNNs with three to six orders of magnitude fewer parameters than have traditionally been used in the literature. In doing so, this work represents the first large-scale classification study in the sonar domain to establish a favorable comparison between automated algorithm performance and human ability. Extensive experimental results on challenging real-world SAS image data sets collected in diverse environments and conditions demonstrate that the CNNs possess strong generalization ability. These findings should significantly impact the manner in which CNNs are utilized in the underwater remote-sensing community. To wit, the tiny CNNs proposed here provide a blueprint for achieving excellent classification performance even with limited computing power or limited data.

Characterization of internal waves in SAS imagery via ray tracing

Synthetic aperture sonar imagery often captures features that appear similar to sand waves but are actually pockets of denser water traveling as isolated waves along the seafloor. These pockets of cold water refract acoustic waves like a lens, causing intensity peaks and shadows that resemble medium to large scale sand waves. This work uses dynamic ray tracing to predict the intensity return as affected by refraction. First, we explore the nature of the intensity pattern created by internal waves of various shapes and sizes. Then, we use an optimization-based approach to solve the inverse problem: given an intensity pattern, determine the size, shape, and location of the internal wave that created it.

Experimental analysis of the effects of bistatic target scattering on synthetic aperture sonar imagery

Downward looking synthetic aperture sonar (SAS) systems, such as those used to detect surficial and buried unexploded ordinance, may use arrays with large spatial extent compared to the depth of the targets they are imaging. With such systems the targets are in the near field of the physical array formed by the projector and receiver. Beamforming algorithms, data representation schemes, and automated target recognition algorithms can benefit from considering the bistatic scattering patterns of targets in this geometry. In downward looking SAS systems, resonant scattering behavior may be used to discriminate targets from clutter because scattering from the sediment-water interface may obscure the geometric scattering response of targets. An experimental analysis of the effects on SAS imagery due to bistatic collection geometries was conducted using an in-air laboratory setup with resonant targets. This data was used to develop a signal processing algorithm that may help improve target localization, detection, and identification by focusing re-radiated resonant energy from targets. Results from the application of this algorithm are compared for an approximately monostatic and a truly bistatic SAS collection geometry using an aluminum pipe target.