Sonar Image Segmentation Research Articles

Multi-scale constraints and perturbation consistency for semi-supervised sonar image segmentation

Multi-scale constraints and perturbation consistency for semi-supervised sonar image segmentation

Correction to: Different treatments of pixels in unlabeled images for semi-supervised sonar image segmentation

Correction to: Different treatments of pixels in unlabeled images for semi-supervised sonar image segmentation

Seg2Sonar: A Full-Class Sample Synthesis Method Applied to Underwater Sonar Image Target Detection, Recognition, and Segmentation Tasks

Seg2Sonar: A Full-Class Sample Synthesis Method Applied to Underwater Sonar Image Target Detection, Recognition, and Segmentation Tasks

An Effective Strategy of Object Instance Segmentation in Sonar Images

Instance segmentation is a task that involves pixel‐level classification and segmentation of each object instance in images. Various CNN‐based methods have achieved promising results in natural image instance segmentation. However, the noise interference, low resolution, and blurred edges bring more significant challenges for sonar image instance segmentation. To solve these problems, we propose the Effective Strategy for Sonar Images Instance Segmentation (ESSIIS). We introduce ASception, a new network combining Atrous Spatial Pyramid Pooling (ASPP) and Extreme Inception (Xception). By integrating this with ResNet and transforming traditional convolutions into deformable convolutions, we further improve the ability of the network to extract features from sonar images. Additionally, we incorporate a bidirectional feature fusion module to enhance information fusion. Finally, we evaluate the detection accuracy and segmentation accuracy of the proposed method on the public sonar image dataset and the self‐constructed dataset. ESSIIS attains a detection accuracy of 0.981 and a segmentation accuracy of 0.951 on SCTD, further impressively achieving 0.986 in both metrics when appraised on our dataset. The evaluation results demonstrate that the proposed method is more accurate, robust, and considerable for sonar image detection and segmentation.

RPFNet: Recurrent Pyramid Frequency Feature Fusion Network for Instance Segmentation in Side-Scan Sonar Images

Side-scan sonar (SSS) is an essential acoustic sensor device for obtaining underwater information. The instance segmentation of sonar images can effectively locate and detect underwater objects. Although various CNN-based frameworks have achieved promising results in natural image instance segmentation, the noise interference, highlight shadow, and blurred edge in sonar images bring more significant challenges for sonar image instance segmentation. To solve these problems, we propose a novel recurrent pyramid frequency feature fusion network (RPFNet), which mainly consists of the pyramid frequency feature fusion network (PFN), recurrent residual attention mechanism (RRAM), mask prediction module, and semantic segmentation module. By enhancing and fusing different frequency features of SSS images, PFN can effectively extract fine-grained features and reduce background information interference. The RRAM uses residual structure and attention mechanism to enhance the correlation of different frequency features and improve nonlinear feature representation ability. The mask prediction and semantic segmentation modules are used to discriminate object instance categories and generate corresponding semantic segmentation results. We extensively evaluate the proposed framework on the real-scenes sonar image dataset. The evaluation results demonstrate that the proposed method outperforms the state-of-the-art methods in various evaluation metrics. Code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/darkseid-arch/SonarRPFNet</uri> .

A Level Set Method With Heterogeneity Filter for Side-Scan Sonar Image Segmentation

A Level Set Method With Heterogeneity Filter for Side-Scan Sonar Image Segmentation

SonarNet: Hybrid CNN-Transformer-HOG Framework and Multifeature Fusion Mechanism for Forward-Looking Sonar Image Segmentation

SonarNet: Hybrid CNN-Transformer-HOG Framework and Multifeature Fusion Mechanism for Forward-Looking Sonar Image Segmentation

Filtering level-set model based on saliency and gradient information for sonar image segmentation

Filtering level-set model based on saliency and gradient information for sonar image segmentation

Different treatments of pixels in unlabeled images for semi- supervised sonar image segmentation

Different treatments of pixels in unlabeled images for semi- supervised sonar image segmentation

An underwater small target boundary segmentation method in forward-looking sonar images

Underwater target boundary segmentation is integral to forward-looking sonar image processing. However, small target boundary segmentation is always challenging due to the low resolution, low signal-to-noise ratio, and inhomogeneity of intensity of forward-looking sonar images. This paper proposes an improved level set segmentation method to accurately obtain the contour of small targets in the forward-looking sonar images: morphological reconstruction combined with the level set method (MRLSM). Compared with the classical level set and the level set method combined with the morphological method, MRLSM improves the accuracy and stability for forward-looking sonar image segmentation. Furthermore, the fuzzy C-means, Markov random field, and MRLSM are applied to the numerical simulations and experimental data for comparison. The deep learning methods are also used to compare the performances. The results demonstrated that the proposed method is more accurate, robust, and considerable for underwater small target boundary segmentation.

Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation

Deep learning has enabled significant improvements in semantic image segmentation, especially in underwater imaging domains such as side scan sonar (SSS). In this work, we apply deep learning to synthetic aperture sonar (SAS) imagery, which has an advantage over traditional SSS in which SAS produces coherent high- and constant-resolution imagery. Despite the successes of deep learning, one drawback is the need for abundant labeled training data to enable success. Such abundant labeled data are not always available as in the case of SAS where collections are expensive and obtaining quality ground-truth labels may require diver intervention. To overcome these challenges, we propose a domain-specific deep learning network architecture utilizing a unique property to complex-valued SAS imagery: the ability to resolve angle-of-arrival (AoA) of acoustic returns through <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -space processing. By sweeping through consecutive incrementally advanced AoA bandpass filters (a process sometimes referred to as multi-look processing), this technique generates a sequence of images emphasizing angle-dependent seafloor scattering and motion from biologics along the seafloor or in the water column. Our proposal, which we call multi-look sequence processing network (MLSP-Net), is a domain-enriched deep neural network architecture that models the multi-look image sequence using a recurrent neural network (RNN) to extract robust features suitable for semantic segmentation of the seafloor without the need for abundant training data. Unlike previous segmentation works in SAS, our model ingests a complex-valued SAS image and affords the ability to learn the AoA filters in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -space as part of the training procedure. We show the results on a challenging real-world SAS database, and despite the lack of abundant training data, our proposed method shows superior results over state-of-the-art techniques.

Feature Pyramid U-Net with Attention for Semantic Segmentation of Forward-Looking Sonar Images.

Forward-looking sonar is a technique widely used for underwater detection. However, most sonar images have underwater noise and low resolution due to their acoustic properties. In recent years, the semantic segmentation model U-Net has shown excellent segmentation performance, and it has great potential in forward-looking sonar image segmentation. However, forward-looking sonar images are affected by noise, which prevents the existing U-Net model from segmenting small objects effectively. Therefore, this study presents a forward-looking sonar semantic segmentation model called Feature Pyramid U-Net with Attention (FPUA). This model uses residual blocks to improve the training depth of the network. To improve the segmentation accuracy of the network for small objects, a feature pyramid module combined with an attention structure is introduced. This improves the model's ability to learn deep semantic and shallow detail information. First, the proposed model is compared against other deep learning models and on two datasets, of which one was collected in a tank environment and the other was collected in a real marine environment. To further test the validity of the model, a real forward-looking sonar system was devised and employed in the lake trials. The results show that the proposed model performs better than the other models for small-object and few-sample classes and that it is competitive in semantic segmentation of forward-looking sonar images.

Semantic Segmentation of Side-Scan Sonar Images with Few Samples

Underwater sensing and detection still rely heavily on acoustic equipment, known as sonar. As an imaging sonar, side-scan sonar can present a specific underwater situation in images, so the application scenario is comprehensive. However, the definition of side scan sonar is low; many objects are in the picture, and the scale is enormous. Therefore, the traditional image segmentation method is not practical. In addition, data acquisition is challenging, and the sample size is insufficient. To solve these problems, we design a semantic segmentation model of side-scan sonar images based on a convolutional neural network, which is used to realize the semantic segmentation of side-scan sonar images with few training samples. The model uses a large convolution kernel to extract large-scale features, adds a parallel channel using a small convolution kernel to obtain multi-scale features, and uses SE-block to focus on the weight of different channels. Finally, we verify the effect of the model on the self-collected side-scan sonar dataset. Experimental results show that, compared with the traditional lightweight semantic segmentation network, the model’s performance is improved, and the number of parameters is relatively small, which is easy to transplant to AUV.

DSA-SOLO: Double Split Attention SOLO for Side-Scan Sonar Target Segmentation

Side-scan sonar systems play an important role in tasks such as marine terrain exploration and underwater target identification. Target segmentation of side-scan sonar images is an effective method of underwater target detection. However, the principle of side-scan sonar systems leads to high noise interference, weak boundary information, and difficult target feature extraction of sonar images. To solve these problems, we propose a Double Split Attention (DSA) SOLO. Specially, we present an efficient attention module called DSA which fuses spatial attention and channel attention together effectively. DSA first splits feature maps into two parts along channel dimensions before processing them in parallel. Next, DSA utilizes C-S Unit and S-C Unit to describe relevant features in the spatial and channel dimensions, respectively. After that, the results of the two parts are aggregated to improve feature representation. We embedded the proposed DSA module after the FPN network of SOLOv2, and this approach improves the instance segmentation accuracy to a great extent. Experimental results show that our proposed DSA-SOLO on SCTD dataset achieves 78.4% mAP.5, which is 5.1% higher than SOLOv2.

Data augmentation using image translation for underwater sonar image segmentation

In underwater environment, the study of object recognition is an important basis for implementing an underwater unmanned vessel. For this purpose, abundant experimental data to train deep learning model is required. However, it is very difficult to obtain these data because the underwater experiment itself is very limited in terms of preparation time and resources. In this study, the image transformation model, Pix2Pix is utilized to generate data similar to experimental one obtained by our ROV named SPARUS between the pool and reservoir. These generated data are applied to train the other deep learning model, FCN for a pixel segmentation of images. The original sonar image and its mask image have to be prepared for all training data to train the image segmentation model and it takes a lot of effort to do it what if all training data are supposed to be real sonar images. Fortunately, this burden can be released here, for the pairs of mask image and synthesized sonar image are already consisted in the image transformation step. The validity of the proposed procedures is verified from the performance of the image segmentation result. In this study, when only real sonar images are used for training, the mean accuracy is 0.7525 and the mean IoU is 0.7275. When the both synthetic and real data is used for training, the mean accuracy is 0.81 and the mean IoU is 0.7225. Comparing the results, the performance of mean accuracy increase to 6%, performance of the mean IoU is similar value.

Side-Scan Sonar Image Segmentation Based on Multi-Channel CNN for AUV Navigation.

The AUV (Autonomous Underwater Vehicle) navigation process relies on the interaction of a variety of sensors. The side-scan sonar can collect underwater images and obtain semantic underwater environment information after processing, which will help improve the ability of AUV autonomous navigation. However, there is no practical method to utilize the semantic information of side scan sonar image. A new convolutional neural network model is proposed to solve this problem in this paper. The model is a standard codec structure, which extracts multi-channel features from the input image and then fuses them to reduce parameters and strengthen the weight of feature channels. Then, a larger convolution kernel is used to extract the features of large-scale sonar images more effectively. Finally, a parallel compensation link with a small-scale convolution kernel is added and spliced with features extracted from a large convolution kernel in the decoding part to obtain features of different scales. We use this model to conduct experiments on self-collected sonar data sets, which were uploaded on github. The experimental results show that ACC and MIoU reach 0.87 and 0.71, better than other classical small-order semantic segmentation networks. Furthermore, the 347.52 g FOLP and the number of parameters around 13 m also ensure the computing speed and portability of the network. The result can extract the semantic information of the side-scan sonar image and assist with AUV autonomous navigation and mapping.

Forward looking sonar image segmentation based on empirical mode decomposition

Due to the low imaging quality and serious noise pollution of forward-looking sonar images, traditional image segmentation methods based on edge information or statistical information are difficult to obtain high precision and robust segmentation results. Therefore, it is very complicated to segment forward-looking sonar images. Based on the analysis of the characteristics of forward-looking sonar, a new image segmentation method for forward-looking sonar is proposed.Firstly, 2d maximum entropy segmentation principle combined with chicken flock optimization algorithm is used to remove the background of the forward-looking sonar image.Then, based on 2d empirical mode decomposition, appropriate eigenmode functions are selected to denoise and enhance the forward-looking sonar image.Finally, the reconstructed forward-looking sonar image is segmented by enhanced fuzzy C-means clustering, and the segmented forward-looking sonar image result is obtained. This method can not only suppress noise interference effectively, but also protect the details of image edge for segmentation.

Active Mask-Box Scoring R-CNN for Sonar Image Instance Segmentation

Instance segmentation of sonar images is an effective method for underwater target recognition. However, the mismatch among positioning accuracy found by boxIoU and classification confidence, which is used as NMS score in current instance segmentation models; and the high annotation cost of sonar images, are two major problems in the task. To tackle these problems, in this paper, we present a novel instance segmentation method called Mask-Box Scoring R-CNN and embedded it in our proposed deep active learning framework. For the mismatch problem between boxIoU and NMS score, Mask-Box Scoring R-CNN uses a boxIoU head to predict the quality of the bounding boxes. We amend the non-maximum suppression (NMS) score predicted by BoxIoU to preserve high-quality bounding boxes in inference flow. To deal with the annotating problem, we propose a triplets-measure-based active learning (TBAL) method and a balanced-sampling method applicable for deep learning. The TBAL method evaluates the amount of information of unlabeled samples from the aspects of classification confidence, positioning accuracy, and mask quality. The balanced-sampling method selects hard samples from the dataset to train the model to improve performance. The experimental results show that Mask-Box Scoring R-CNN achieves improvements of 1% in boxAP and 1.3% boxAP on our sonar image dataset compared with Mask Scoring R-CNN and Mask R-CNN, respectively. The active learning framework with TBAL and balanced sampling can achieve a competitive performance with less labeled samples than other frameworks, which can better facilitate underwater target recognition.

Spectral Normalized CycleGAN with Application in Semisupervised Semantic Segmentation of Sonar Images.

The effectiveness of CycleGAN is demonstrated to outperform recent approaches for semisupervised semantic segmentation on public segmentation benchmarks. In contrast to analog images, however, the acoustic images are unbalanced and often exhibit speckle noise. As a consequence, CycleGAN is prone to mode-collapse and cannot retain target details when applied directly to the sonar image dataset. To address this problem, a spectral normalized CycleGAN network is presented, which applies spectral normalization to both generators and discriminators to stabilize the training of GANs. Without using a pretrained model, the experimental results demonstrate that our simple yet effective method helps to achieve reasonably accurate sonar targets segmentation results.

Side-Scan Sonar Image Segmentation Based on Multi-Channel Fusion Convolution Neural Networks

Side-scan sonar is an important application in the field of ocean exploration. Accurate segmentation of target regions in side-scan sonar images is a challenging issue due to the low-resolution and strong noise interference. To accurately and faster segment the different categories target in sonar image, a novel convolutional neural networks (CNNs) model is proposed in this study. Firstly, the deep separable residual module is used for target regions multi-scale feature extraction and suppression noise feature information interference, and the multi-channel feature fusion method is used to enhance feature information transfer of convolution layers. Secondly, the adaptive supervised function is used for pixel-wise classification of different categories targets. Finally, to improve model generalization ability and robustness, the adaptive transfer learning method is introduced in the model training process. We have performed extensive experiments on side-scan sonar image with different targets and scales. The experimental results show that the detection accuracy of the proposed method reaches 95.73%, which is outperforms other state-of-the-art methods on the side-scan sonar image segmentation tasks. Moreover, the method has fewer computational parameters, facilitating future deployment it to underwater mobile detection devices.