Sonar Images Research Articles

АНАЛІЗ МЕТОДІВ ЗНАХОДЖЕННЯ АНОМАЛІЙ У ЗОБРАЖЕННЯХ

Abstract. Automatic anomaly detection is of great importance in industry, remote sensing, and medicine. It is important to be able to automatically process large amounts of data to detect, for example, chemical objects in multispectral and hyperspectral satellite images, sea mines in side-scan sonar images, or defects in industrial monitoring applications. Automatic detection of anomalous structures on arbitrary images refers to the task of finding inappropriate patterns relative to the normal state of the image. This is a difficult task in computer vision, since there is no clear and straightforward definition of what is normal or not normal for a given arbitrary image. The practical importance is manifested in the development of algorithms and models that can automatically detect unusual or anomalous patterns in images. An analysis of methods for finding anomalies in images from the point of view of the possibility of application to arbitrary images has been carried out. The classification of anomaly detection methods according to the criteria of the involved approaches and models used for modeling the background is presented. Methods that use machine learning, such as one-class support vector method and variational autoencoder, nearest neighbor-based anomaly detection, clustering-based anomaly detection, statistical anomaly detection, spectral anomaly detection, anomaly detection using information theory are discussed. The main attention is paid to the methods classified according to the background modeling approach. Five categories of background modeling methods based on probability density function, global and local homogeneity, sparsity, and self-similarity are considered. For anomaly detection applications, it is recommended to use methods in which the background model best describes the expected anomaly-free background, as this generally results in the best performance. On the basis of research, it was established that an effective universal model for detecting anomalies in arbitrary images should: use only a self-similar or sparse background model; process the residual image as a stochastic process to detect anomalies, such as anomalies in color noise; preprocess the residual image before detecting the anomaly.

Shuffle-RDSNet: a method for side-scan sonar image classification with residual dual-path shrinkage network

Shuffle-RDSNet: a method for side-scan sonar image classification with residual dual-path shrinkage network

Экстраполяционные алгоритмы улучшения качества гидролокационных изображений

Рассмотрены вопросы улучшения качества изображений морского дна по данным измерений гидролокатора бокового обзора, оснащенного антеннами с различной шириной диаграммы направленности. В рамках кинетической модели, описывающей процесс импульсного зондирования в полупространстве с диффузными условиями отражения на границе, для повышения точности восстановления коэффициента донного рассеяния предложены экстраполяционные алгоритмы первого и второго порядков относительно малого параметра, характеризующего ширину диаграммы направленности. Проведен численный анализ алгоритмов в зависимости от порядка экстраполяции и точности измерений исходных данных. Purpose. Mathematical modelling of the processes of acoustic sounding of the marine bottom using side-scan sonar mounted on autonomous unmanned underwater vehicles. Development of new extrapolation algorithms for processing the measured signal to improve the quality of seabed images. Methodology. As a mathematical model, the nonstationary radiative transfer equation in an unbounded region with diffuse reѕection conditions on the boundaries is considered. An inverse problem is formulated, which includes ҥnding for the coefficient of bottom scattering under some redeҥnition conditions. Using some simplifying restrictions, the problem is reduced to an integral equation of the ҥrst kind, which has an explicit solution in the narrow beam approximation. For an approximate solution of the obtained equation, it is proposed to use the data of measurements by antennas with radiation patterns of different widths. Then the solution of the radiation transfer equation is extrapolated to the region of small values of the parameter characterizing the width of the radiation pattern. An explicit formula is used to ҥnd the bottom scattering coefficient for the extrapolated solution of the direct problem. Extrapolation algorithms of the ҥrst and second order are considered. Findings. A series of numerical experiments revealed that the developed algorithms are very effective if the condition on the accuracy of measurements are satisҥed. The limitations associated with the accuracy of the original data, especially affect the quality of images when using higher order extrapolation formulas.

A Multi-Scale Fusion Strategy for Side Scan Sonar Image Correction to Improve Low Contrast and Noise Interference

Side scan sonar images have great application prospects in underwater surveys, target detection, and engineering activities. However, the acquired sonar images exhibit low illumination, scattered noise, distorted outlines, and unclear edge textures due to the complicated undersea environment and intrinsic device flaws. Hence, this paper proposes a multi-scale fusion strategy for side scan sonar (SSS) image correction to improve the low contrast and noise interference. Initially, an SSS image was decomposed into low and high frequency sub-bands via the non-subsampled shearlet transform (NSST). Then, modified multi-scale retinex (MMSR) was employed to enhance the contrast of the low frequency sub-band. Next, sparse dictionary learning (SDL) was utilized to eliminate high frequency noise. Finally, the process of NSST reconstruction was completed by fusing the emerging low and high frequency sub-band images to generate a new sonar image. The experimental results demonstrate that the target features, underwater terrain, and edge contours could be clearly displayed in the image corrected by the multi-scale fusion strategy when compared to eight correction techniques: BPDHE, MSRCR, NPE, ALTM, LIME, FE, WT, and TVRLRA. Effective control was achieved over the speckle noise of the sonar image. Furthermore, the AG, STD, and E values illustrated the delicacy and contrast of the corrected images processed by the proposed strategy. The PSNR value revealed that the proposed strategy outperformed the advanced TVRLRA technology in terms of filtering performance by at least 8.8%. It can provide sonar imagery that is appropriate for various circumstances.

Object Detection in Side Scan Sonar Images using Local Binary Pattern features

Object Detection in Side Scan Sonar Images using Local Binary Pattern features

Side-Scan Sonar Image Matching Method Based on Topology Representation

In the realm of underwater environment detection, achieving information matching stands as a pivotal step, forming an indispensable component for collaborative detection and research in areas such as distributed mapping. Nevertheless, the progress in studying the matching of underwater side-scan sonar images has been hindered by challenges including low image quality, intricate features, and susceptibility to distortion in commonly used side-scan sonar images. This article presents a comprehensive overview of the advancements in underwater sonar image processing. Building upon the novel SchemaNet image topological structure extraction model, we introduce a feature matching model grounded in side-scan sonar images. The proposed approach employs a semantic segmentation network as a teacher model to distill the DeiT model during training, extracting the attention matrix of intermediate layer outputs. This emulates SchemaNet’s transformation method, enabling the acquisition of high-dimensional topological structure features from the image. Subsequently, utilizing a real side-scan sonar dataset and augmenting data, we formulate a matching dataset and train the model using a graph neural network. The resulting model demonstrates effective performance in side-scan sonar image matching tasks. These research findings bear significance for underwater detection and target recognition and can offer valuable insights and references for image processing in diverse domains.

NAS-MFF: NAS-Guided Multiscale Feature Fusion Network With Pareto Optimization for Sonar Images Classification

NAS-MFF: NAS-Guided Multiscale Feature Fusion Network With Pareto Optimization for Sonar Images Classification

Biomimetic Sonar Innovation Inspired from Dolphins: A Comprehensive Review

Abstract: The sonar of dolphins has evolved over millions of years and has attained outstanding performance levels. Using the exceptional performance of the dolphins’ sonar as an impetus, bio-inspired wideband acoustic sensing approaches for underwater target recognition and tracking are under development. In this study, we have sight seen what they expect to extract as a gain from such a wideband sonar. The systems wideband sensors are grounded on bottlenose dolphins’ sonar, encapsulating a frequency band from about 30 to 150 kHz and having a frequency reliant on beamwidth substantially larger than that of conventional imaging sonars. The system can be made fairly compact and apt for mounting on diverse platforms including small-scale autonomous underwater automobiles to permit the sonar to function in an analogous way to that used by dolphins. In this paper, we have highlighted the mechanism and applications of the sonar innovation in various domains along with the ongoing developments with regard to it.

Assessing the importance of sediment characterization on seabed embedment predictions of cylindrical objects

The prediction of object embedment into the seabed is important for the deployment and efficacy of submerged sensors and sensor networks, as well as for other applications such as unexploded ordnance risk assessment, subaquatic search and rescue efforts, or anchors and moorings. The embedment of objects into the seabed during placement depends on the strength parameters of the seabed sediment. However, a detailed characterization of seabed strength can be complex and costly since quality seabed samples and/or in situ testing are required. This study evaluates the importance of a detailed geotechnical seabed sediment characterization over seabed classification from geoacoustic surveying or accepting unknown seabed conditions for the prediction of seabed embedment of cylindrical objects. Monte Carlo simulation was applied to model the uncertainty of sediment strength parameters. The results suggest that prior knowledge of the general sediment type (e.g., fine-grained versus coarse-grained as determined from side scan sonar imaging) significantly improves the precision of the embedment predictions over unknown conditions, reducing the coefficient of variation (CV) in prediction of percent embedment (PE) by about 50 % to 65 %. Performing a detailed geotechnical testing of sediment strength represented a further improvement by 15 % to 35 % over the knowledge of sediment type. Both the mean and standard deviation of PE in fine-grained sediments are at least about 3 times the values for coarse-grained sediments indicating a wider distribution of PE with deeper embedment for cohesive fine-grained soils. The embedment behavior is a function of the method of deployment (free fall with a certain speed versus placement with no speed) in stiff cohesive sediments but is less relevant in soft sediments since PE values are always greater than 100 % (i.e., full embedment). Similarly, in cohesionless sediments, PE is always less than 50 %.

Density and abundance estimation of West Indian manatee, Trichechus manatus, between the states of Ceará and Piauí, Northeast Brazil, using active acoustics

The West Indian manatee is one of the most threatened species in Brazil. The species has currently a patchy distribution from the state of Alagoas to the state of Amapá. The difficulty of observing manatees, especially in estuarine waters, is a challenge for conservation. Therefore, it is necessary to use new methodologies and technologies to solve manatee detection problems in their natural habitats. The goal of this study was to use an active acoustic method of detection to estimate manatee density and abundance in the estuarine complex of the Timonha and Ubatuba rivers, between the states of Ceará and Piauí, northeast Brazil. Data collection was conducted from a wooden boat with an outboard motor using a side-scan sonar. Sonar images were collected along line transects in three regions within the study area. Manatee abundance was calculated using distance sampling (Distance 6.0 software). There were 1,396 transection lines that totaled 863.6 km traveled in the study area. The observed general encounter rate was 0.089 group detected per kilometer traveled. The best detection model was the hazard-rate with no adjustment terms, resulting in a detection probability of 33.7%, and an estimated density for the entire study area of 9.19 manatees per km2. Abundance of manatees in the estuary was estimated to be 37 animals (CV% = 30.2, 95% CI = 21 - 66). The methodology using side-scan sonar proposed here was successful in the detection of manatees in the study area and in defining parameters for its use to estimate the population size using the distance sampling method. This study provides an estimate of manatee density and abundance in a key estuarine system along the northeast coast of Brazil, which can serve as a baseline for future studies and aid in the development of conservation strategies for the species. This is the first time this methodological approach has been used for manatee detection and abundance estimation in Brazil. We recommend the use of side-scan sonar in future West Indian manatee research.

Estimating spawning green sturgeon (Acipenser medirostris Ayres, 1854) abundance in the Sacramento River using side-scan sonar and N-mixture models

Current estimates of the threatened southern distinct population segment of the North American green sturgeon ( Acipenser medirostris) combine a plot-sampling density estimator with Dual frequency IDentification SONar (DIDSON) and adaptive resolution imaging sonar (ARIS) sonar data. From 2020 to 2022, we annually collected images of all known green sturgeon aggregations and compared the established method to an N-mixture model using side-scan sonar images. We compared 18 different N-mixture model combinations and chose an overdispersed Poisson model that produced estimated abundances of 742, 1286, and 1208 for 2020–2022, respectively. These numbers are ∼2 times greater than the previous method and, if sustained, would fulfill a key criterion for green sturgeon recovery. N-mixture models are known to be sensitive to violations of assumptions, such as the highly dispersed data from our study that caused serious issues, and we recommend practitioners make judicious use of overdispersion and goodness-of-fit tests and be able to identify parameter confounding between detectability and abundance estimates. For our green sturgeon, we recommend simpler population estimates and to focus future energy on reducing variability in the data collection process.

GCT-YOLOv5: a lightweight and efficient object detection model of real-time side-scan sonar image

GCT-YOLOv5: a lightweight and efficient object detection model of real-time side-scan sonar image

The accuracy of length measurements made using imaging SONAR is inversely proportional to the beamwidth

Abstract Multibeam imaging SONARs have been used for a range of measurement applications, such as measurements of fish lengths. This study aimed to quantify the accuracy of imaging SONAR systems, that varied in frequency and beam geometry, to measure the length of synthetic targets positioned perpendicularly. Blueprint Oculus imaging SONAR systems, with four different (centre) frequencies (750 kHz, 1.2 MHz, 2.1 MHz and 3 MHz), were used to measure the length of three targets of nominal lengths: 10 cm, 20 cm and at ranges between 1 and 15.5 m. The effect of beam geometry on measurement error was then examined using regression analysis. This study found that there was an overestimation of the actual length of the target for all measurements that was inversely proportional to the horizontal beamwidth. The measurement error can be reduced by normalising for beamwidth. However, the variation of measurements (i.e. the precision) was found to also increase with range, which was attributed to the increasing beam separation. It is important that the effect of beamwidth on the accuracy of target length measurements, by imaging SONARs is acknowledged in future studies. One approach to mitigating this problem, is to limit the range at which length measurements are made to a beamwidth that produces an estimated level of error that is acceptable for that study.

3D Detection and Tracking of a Moving Object by an Autonomous Underwater Vehicle With a Multibeam Imaging Sonar: Toward Continuous Observation of Marine Life

3D Detection and Tracking of a Moving Object by an Autonomous Underwater Vehicle With a Multibeam Imaging Sonar: Toward Continuous Observation of Marine Life

Erosional and Depositional Features along the Axis of a Canyon in the Northern South China Sea and Their Implications: Insights from High-Resolution AUV-Based Geophysical Data

Autonomous Underwater Vehicle (AUV)-based multibeam bathymetry, sub-bottom profiles, and side-scan sonar images were collected in 2009 and 2010 to map the geomorphic features along the axial zone of a canyon (referred to as C4) within the canyon system developed on the northern slope of the South China Sea. These data significantly improved the spatial resolution of acoustic data, leading to a better understanding of the sedimentary processes within the modern canyon system. The bathymetric data reveal that sections across the canyon axis exhibit either asymmetrical or symmetrical characteristics, which differ from the overall asymmetrical pattern of the entire canyon. This suggests that the overall asymmetrical pattern of the canyon is not primarily due to axial incision. Various morphological elements were identified along the canyon axis, including failure scars, undulating features, knickpoints, flat terraces, furrows, and mass transport deposits (MTDs). Landslides, predominantly located in the upper canyon, were formed after at least 5000 years BP. Between the beginning of the canyon and a water depth of approximately 1300 m, there are alternating flat terraces and knickpoints. The large knickpoints’ low slope gradients are likely formed by the presence of undulating features. The internal configurations of undulating features suggest that they are depositional structures rather than sediment deformation. The formation of small-scale furrows below a depth of 1200 m may be associated with occasional gravity flows down the canyon. It is suggested that the canyon was generally inactive during the Holocene but experienced sporadic processes of sediment erosion, transport, and re-deposition in the axial zone that were triggered by landslide events occasionally in the upper canyon.

Advanced deep learning framework for underwater object detection with multibeam forward-looking sonar

Underwater object detection (UOD) is an essential activity in maintaining and monitoring underwater infrastructure, playing an important role in their efficient and low-risk asset management. In underwater environments, sonar, recognized for overcoming the limitations of optical imaging in low-light and turbid conditions, has increasingly gained popularity for UOD. However, due to the low resolution and limited foreground-background contrast in sonar images, existing sonar-based object detection algorithms still face challenges regarding precision and transferability. To solve these challenges, this article proposes an advanced deep learning framework for UOD that uses the data from multibeam forward-looking sonar. The framework is adapted from the network architecture of YOLOv7, one of the state-of-the-art vision-based object detection algorithms, by incorporating unique optimizations in three key aspects: data preprocessing, feature fusion, and loss functions. These improvements are extensively tested on a dedicated public dataset, showing superior object classification performance compared to the selected existing sonar-based methods. Through experiments conducted on an underwater remotely operated vehicle, the proposed framework validates significant enhancements in target classification, localization, and transfer learning capabilities. Since the engineering structures have similar geometric shapes to the objects tested in this study, the proposed framework presents potential applicability to underwater structural inspection and monitoring, and autonomous asset management.

Improved YOLOv7 model for underwater sonar image object detection

Underwater object detection technology for sonar images is widely employed and in high demand for civil and military purposes. However, due to the inhomogeneity of the seawater medium, which causes attenuation and distortion of the acoustic signal, achieving ideal performance for sonar object detection approaches is challenging. Furthermore, the process of acoustic wave transmission is further complicated by floating objects and particles, resulting in increased multipath effects. This study proposes an object detection method named YOLOv7C, which is based on deep learning to address these challenges. An attention mechanism is incorporated into the backbone network of the model to improve its ability to handle complex backgrounds in sonar images and effectively extract features. In addition, to facilitate high-order interaction between key features, the Neck part of the network integrates Multi-GnBlock blocks. Model redundancy pruning is used to substantially reduce the model size while maintaining high detection accuracy, thereby enhancing real-time performance. The proposed YOLOv7C achieves a 1.9% increase in the mean average precision, reduces the model memory by 47.50% after pruning, and enhances the detection speed by nearly 2.5 times compared with the YOLOv7C. These findings indicate that the success rate of multi-object detection is significantly enhanced by the attention mechanism and the new module. Additionally, the model can be controlled within a reasonable size by employing appropriate pruning methods.

Multi-scale constraints and perturbation consistency for semi-supervised sonar image segmentation

Multi-scale constraints and perturbation consistency for semi-supervised sonar image segmentation

Sample Augmentation Method for Side-Scan Sonar Underwater Target Images Based on CBL-sinGAN

The scarcity and difficulty in acquiring Side-scan sonar target images limit the application of deep learning algorithms in Side-scan sonar target detection. At present, there are few amplification methods for Side-scan sonar images, and the amplification image quality is not ideal, which is not suitable for the characteristics of Side-scan sonar images. Addressing the current shortage of sample augmentation methods for Side-scan sonar, this paper proposes a method for augmenting single underwater target images using the CBL-sinGAN network. Firstly, considering the low resolution and monochromatic nature of Side-scan sonar images while balancing training efficiency and image diversity, a sinGAN network is introduced and designed as an eight-layer pyramid structure. Secondly, the Convolutional Block Attention Module (CBAM) is integrated into the network generator to enhance target learning in images while reducing information diffusion. Finally, an L1 loss function is introduced in the network discriminator to ensure training stability and improve the realism of generated images. Experimental results show that the accuracy of shipwreck target detection increased by 4.9% after training with the Side-scan sonar sample dataset augmented by the proposed network. This method effectively retains the style of the images while achieving diversity augmentation of small-sample underwater target images, providing a new approach to improving the construction of underwater target detection models.

AI Explainability and Acceptance: A Case Study for Underwater Mine Hunting

In critical operational context such as Mine Warfare, Automatic Target Recognition (ATR) algorithms are still hardly accepted. The complexity of their decision-making hampers understanding of predictions despite performances approaching human expert ones. Much research has been done in Explainability Artificial Intelligence (XAI) field to avoid this “black box” effect. This field of research attempts to provide explanations for the decision-making of complex networks to promote their acceptability. Most of the explanation methods applied on image classifier networks provide heat maps. These maps highlight pixels according to their importance in decision-making. In this work, we first implement different XAI methods for the automatic classification of Synthetic Aperture Sonar (SAS) images by convolutional neural networks (CNN). These different methods are based on a post hoc approach. We study and compare the different heat maps obtained. Second, we evaluate the benefits and the usefulness of explainability in an operational framework for collaboration. To do this, different user tests are carried out with different levels of assistance, ranging from classification for an unaided operator to classification with explained ATR. These tests allow us to study whether heat maps are useful in this context. The results obtained show that the heat maps explanation has a disputed utility according to the operators. Heat map presence does not increase the quality of the classifications. On the contrary, it even increases the response time. Nevertheless, half of operators see a certain usefulness in heat maps explanation.