Acoustic Target Classification Research Articles

Broadband backscattering by Atlantic herring (Clupea harengus L.) differs when measured from a research vessel vs. a silent uncrewed surface vehicle

Abstract Broadband frequency-modulated signals are believed to improve acoustic spectral-based target classification. Efficient use of uncrewed surface vehicles (USV) for fisheries science applications, with no possibility for biological sampling, is believed to be facilitated by use of broadband signals with methods for target classification. If the broadband frequency response used to train automated target classifiers are obtained from conventional research vessels (RVs), due to potential vessel avoidance, the swimming angle distribution may be different than for USVs. This may have consequences for target classification if the model is trained with RV data. The aim of this study was to assess whether the frequency response differs between platforms due to avoidance. Broadband acoustic data were collected with a conventional RV and a small USV. The broadband frequency response of Norwegian spring spawning herring obtained with the USV and RV was found to be significantly different for shallow herring layers in the 200 kHz band. This indicates that broadband frequency response has potential as a tool for real-time monitoring of behaviour reactions to vessels and to provide insight into fish behaviour in general. When using broadband frequency response for target classification, the potential platform-dependent broadband frequency response should be considered.

Addressing class imbalance in deep learning for acoustic target classification

Abstract Acoustic surveys provide important data for fisheries management. During the surveys, ship-mounted echo sounders send acoustic signals into the water and measure the strength of the reflection, so-called backscatter. Acoustic target classification (ATC) aims to identify backscatter signals by categorizing them into specific groups, e.g. sandeel, mackerel, and background (as bottom and plankton). Convolutional neural networks typically perform well for ATC but fail in cases where the background class is similar to the foreground class. In this study, we discuss how to address the challenge of class imbalance in the sampling of training and validation data for deep convolutional neural networks. The proposed strategy seeks to equally sample areas containing all different classes while prioritizing background data that have similar characteristics to the foreground class. We investigate the performance of the proposed sampling methodology for ATC using a previously published deep convolutional neural network architecture on sandeel data. Our results demonstrate that utilizing this approach enables accurate target classification even when dealing with imbalanced data. This is particularly relevant for pixel-wise semantic segmentation tasks conducted on extensive datasets. The proposed methodology utilizes state-of-the-art deep learning techniques and ensures a systematic approach to data balancing, avoiding ad hoc methods.

Differential treatment for time and frequency dimensions in mel-spectrograms: An efficient 3D Spectrogram network for underwater acoustic target classification

Underwater acoustic target classification (UATC) has traditionally relied on time–frequency (TF) analysis, with the mel-spectrogram being widely used due to its 2D image-like format and size efficiency. However, many previous UATC approaches have not adequately acknowledged the distinction between natural images and mel-spectrograms. In this paper, we propose a novel approach to transform mel-spectrograms into 3D data and introduce an efficient 3D Spectrogram Network (3DSNet) that treats the time and frequency dimensions separately. Our 3DSNet consists of three key components: the Time–Frequency Separate Convolution (TFSConv) module, Asymmetric Pooling (AsyPool) module, and Channel-Time Attention (CTA) module. We introduce the TFSConv module, which utilizes two convolution operators for the time and frequency dimensions to extract 3D T-F features. This module serves as a lightweight approximation of 3D convolution, effectively reducing approximately 85 percent of parameters. To maintain the inherent distinction between the frequency and time dimensions, we propose an AsyPool module that employs two downsampling strategies. Additionally, we introduce a CTA module to capture more informative and meaningful T-F features. We evaluate our 3DSNet on two publicly available underwater acoustic datasets, and the results demonstrate that our method achieves the optimal balance between performance and model parameters compared to other mainstream methods.

Array-Based Underwater Acoustic Target Classification with Spectrum Reconstruction Based on Joint Sparsity and Frequency Shift Invariant Feature

The target spectrum, which is commonly used in feature extraction for underwater acoustic target classification, can be improperly recovered via conventional beamformer (CBF) owing to its frequency-variant spatial response and lead to degraded classification performance. In this paper, we propose a target spectrum reconstruction method under a sparse Bayesian learning framework with joint sparsity priors that can not only achieve high-resolution target separation in the angular domain but also attain beamwidth constancy over a frequency range at no cost of reducing angular resolution. Experiments on real measured array data show the recovered spectrum via our proposed method can effectively suppress interference and preserve more detailed spectral structures than CBF. This indicates our method is more suitable for target classification because it has the capability of retaining more representative and discriminative characteristics. Moreover, due to target motion and the underwater channel effect, the frequency of prominent spectral line components can be shifted over time, which is harmful to classification performance. To overcome this problem, we proposed a frequency shift-invariant feature extraction method with the help of elaborately designed frequency shift-invariant filter banks. The classification experiments demonstrate that our proposed methods outperform traditional CBF and Mel-frequency features and can help improve underwater recognition performance.

Acoustic Classification of Juvenile Pacific Salmon (Oncorhynchus spp) and Pacific Herring (Clupea pallasii) Schools Using Random Forests

Acoustic surveys are the standard approach for evaluating many fish stocks around the world. The analysis of such survey data requires the accurate echo-classification of target species. This classification is often challenging as many organisms exhibit overlapping characteristics in terms of shape, acoustic amplitude, and behavior. In this study, a random forest approach was used to distinguish juvenile Pacific salmon (Oncorhynchus spp) from Pacific herring (Clupea pallasii) aggregations using the acoustic and morphological characteristics of their echo traces. The acoustic data was collected with an autonomous, multi-frequency echosounder deployed on the seafloor in the Discovery Islands, British Columbia from May to September 2015. The model was able to differentiate juvenile Pacific salmon from Pacific herring with a 98% accuracy. School depth and school mean volume backscattering strength were the most important predictors in determining the school classification. This study supports other publications suggesting that random forests represent a promising approach to acoustic target classification in fisheries science.

Evaluation of echosounder data preparation strategies for modern machine learning models

Fish stock assessment and management requires accurate estimates of fish abundance, which are typically derived from echosounder observations using acoustic target classification (ATC). Skilled operators are regularly assisted in classifying acoustic targets by software and there has been an increasing interest toward using machine learning to create improved tools. Recent studies have applied deep learning approaches to acoustic data, however, algorithm data-preparation strategies (influencing model output) are presently poorly understood and standardization is needed to enable collaborative research and management. For example, a common pre-processing technique is to resample backscatter data coming from echosounder measurements from the original resolution to a coarser resolution in the horizontal (time) and vertical (range) directions. Using data values derived from the volume backscattering coefficient obtained during the Norwegian sandeel survey, we investigate which resampling resolutions are suitable for ATC using a convolutional neural network trained to classify single values of backscatter data. This process is known as pixel-level semantic segmentation. Our results indicate that it is possible to downsample the data if important information related to acoustic characteristics is not smoothed out. We also show that the classification performance is improved when providing the network with contextual information relating to range. These findings will provide input to fisheries acoustic data standards and contribute to the on-going development of automated ATC methods.

Spatio-temporal contrastive learning for acoustic data

Large volumes of water column sonar data have been generated from acoustic surveys of living marine resources. They provide valuable information about marine ecosystems. To leverage them for acoustic target identification, scarcity of annotations is usually an issue. While some annotations have been generated by scientists via manual scrutiny or limited automation, they are limited to certain species. To fill in this gap, we propose a spatio-temporal contrastive learning approach for acoustic data. This unsupervised deep learning technique leverages both acoustic and non-acoustic (spatial and temporal) information for acoustic target classification. We firstly employ the Simple Linear Iterative Clustering (SLIC) algorithm to extract superpixels from acoustic data. Then the spatial and temporal similarity between two superpixels is computed, and the proposed spatio-temporal contrastive learning approach is applied to learn a semantically meaningful representation for superpixels. Finally, the learned representations of superpixels are used in acoustic target classification. We demonstrate that this approach outperforms previous methods with acoustic data collected by the NOAA National Marine Fisheries Service in the Gulf of Maine and Georges Bank region over 10 years. This dataset is archived at NOAA National Centers for Environmental Information and accessed for free through AWS and NOAA Big Data Program.

A Multi-Feature Compression and Fusion Strategy of Vertical Self-Contained Hydrophone Array

Applying passive sonar to classify underwater acoustic targets at different depths is a challenging task. Although the self-contained hydrophone array can ensure the normal operation of most units in various environments, it is arduous to achieve precise time synchronization between each hydrophone, which results in difficulties in data fusion between hydrophones. For a vertical sonar array composed of self-contained units, a deep learning-based data compression and multihydrophone fusion (DCMF) model is proposed to quickly extract acoustic propagation interference features, which are used for underwater acoustic target classification. Unlike the frequency-range domain striation features acquired by long-term accumulation, this paper exploits the depth difference between multiple hydrophones to obtain the frequency-depth domain joint striation features in a short time. The proposed DCMF conducts efficient feature compression and fusion via parallel stacked sparse autoencoders and a multi-input fusion network. The experimental results illustrate that the compressed features have strong robustness, a low mean square error with the simulation results, and shorter signal length requirements, which improves the classification efficiency and real-time performance of DCMF. In the case of the experimental dataset, DCMF is compared with several state-of-the-art multiscale fusion models, and the experiments indicate that DCMF has the best performance and smallest computational complexity.

Fisheries acoustics in Norway—Wide band data, autonomous platforms and deep learning

This presentation gives an overview of recent achievements in fisheries acoustics associated with the Center for Research Based Innovation in Marine Acoustic Abundance Estimation and Backscatter Classification (CRIMAC), Norway. We present a data processing pipeline from raw acoustic data to survey indices of abundance for fisheries assessments as a test bench. This includes the raw data as well as manually worked up labels for acoustic target classification (ATC). We also collect data from dedicated CRIMAC surveys and organize broadband data across a range of species, using different echo sounder setting, to find optimal settings for various standard surveys. We have trained deep neural networks on labelled acoustic data for ATC and tested this in the pipeline. Semi-supervised methods are also developed, requiring only 10% of the training data compared to the supervised models. IMR has a developed a strategy for phasing in unmanned platforms in our surveys. The process is stepwise, where we first augment existing surveys to maintain time series integrity; this approach has been tested on our sand eel and sprat surveys. We also embed our deep learning models in containers to facilitate simple deployment. This allows adaptive survey strategies, which is a step towards fully autonomous acoustic surveys.

Underwater acoustic target classification based on u-shaped network

Target recognition is a key task and a difficult technique in underwater acoustic signal processing. One of the most challenging problem is that the label information of the underwater acoustic samples is scarce or missing. To solve the problem, this paper presents a local skip connection u-shaped architecture network(U-Net)based on the convolutional neural network(CNN).To this end, the network architecture is designed cleverly to generate a contracting path and an expansive path to achieve the extraction of different scale features. More importantly, a local skip connection mechanism is proposed to optimize classification rates by reusing former feature maps in contracting path. The experimental results of the measured dataset demonstrate the recognition accuracy of the model is better than that of deep belief network(DBN) and generative adversarial network(GAN) networks.Further research on three kinds of network by visualization method shows that the proposed network can learn more effective feature information with limited samples.

Underwater Acoustic Target Classification Based on Dense Convolutional Neural Network

In oceanic remote sensing operations, underwater acoustic target recognition is always a difficult and extremely important task of sonar systems, especially in the condition of complex sound wave propagation characteristics. The expensively learning recognition model for big data analysis is typically an obstacle for most traditional machine learning (ML) algorithms, whereas the convolutional neural network (CNN), a type of deep neural network, can automatically extract features for accurate classification. In this study, we propose an approach using a dense CNN model for underwater target recognition. The network architecture is designed to cleverly reuse all former feature maps to optimize classification rates under various impaired conditions while satisfying low computational cost. In addition, instead of using time–frequency spectrogram images, the proposed scheme allows directly utilizing the original audio signal in the time domain as the network input data. Based on the experimental results evaluated on the real-world data set of passive sonar, our classification model achieves the overall accuracy of 98.85% at 0-dB signal-to-noise ratio (SNR) and outperforms traditional ML techniques, as well as other state-of-the-art CNN models.

Efficiency of Various Time-Frequency Representations in Deep Neural Network based Passive Sonar Target Classifiers

Passive acoustic target classification is an exceptionally challenging problem due to the complex phenomena associated with the channel and the relatively low Signal to Noise Ratio (SNR) manifested by the pervasive ambient noise field. Inspired by the overwhelming success of Deep Neural Networks (DNNs) in many such hard problems, a carefully crafted network specifically for target recognition application has been employed in this work. Although deep neural networks can learn characteristic features or representations directly from the raw observations, domain specific intermediate representations can mitigate the computational requirements as well as the sample complexity required to achieve an acceptable error rate in prediction. As the sonar target records are essentially a time series, spectro-temporal representations can make the intricate relationship between time and spectral components more explicit. In a passive sonar target recognition scenario, since most of the defining spectral components reside at the lower part of the spectrum, a nonlinear dilated spectral scale having an emphasis on low frequencies is highly desirable. This can be easily achieved using a filterbank based time-frequency decomposition, which allows more filters to be positioned at the desired frequency ranges of interest. In this work, a rigorous analysis of the performance of time-frequency representations initialized at various frequency scales, is conducted independently as well as in combination. A convolutional neural network based spectro-temporal feature learner has been utilized as the initial layers, while a deep stack of Long Short Term Memories (LSTMs) with residual connections has been used for learning the intricate temporal relationships hidden in the intermediate representations. From the experimental results it can be observed that a linear scale spectrogram achieves an accuracy of 92.4% and 90.2% respectively for validation and test sets in the single feature configuration, whereas the gammatone spectrogram is capable of attaining an accuracy in the order of 96.7% and 96.1% respectively for the same. In a multifeatured setup however, the accuracy reaches up to 97.3% and 96.6% respectively, which reveals that a combination of properly initialized intermediate representations can improve the classification performance significantly.

Acoustic classification in multifrequency echosounder data using deep convolutional neural networks

Abstract Acoustic target classification is the process of assigning observed acoustic backscattering intensity to an acoustic category. A deep learning strategy for acoustic target classification using a convolutional network is developed, consisting of an encoder and a decoder, which allow the network to use pixel information and more abstract features. The network can learn features directly from data, and the learned feature space may include both frequency response and school morphology. We tested the method on multifrequency data collected between 2007 and 2018 during the Norwegian sandeel survey. The network was able to distinguish between sandeel schools, schools of other species, and background pixels (including seabed) in new survey data with an F1 score of 0.87 when tested against manually labelled schools. The network separated schools of sandeel and schools of other species with an F1 score of 0.94. A traditional school classification algorithm obtained substantially lower F1 scores (0.77 and 0.82) when tested against the manually labelled schools. To train the network, it was necessary to develop sampling and preprocessing strategies to account for unbalanced classes, inaccurate annotations, and biases in the training data. This is a step towards a method to be applied across a range of acoustic trawl surveys.

An Acoustic-Based Feature Extraction Method for the Classification of Moving Vehicles in the Wild

The acoustic classification technology of moving vehicles is a significant application in wireless sensor networks (WSNs). However, the wild environment makes it more intractable for the acoustic target classification owing to the complicated interference noise. The widely used mel-frequency cepstral coefficients (MFCCs) are somewhile contaminated by the acoustic noise, resulting in degrading the classification performance in real environments. To increase the classification accuracy of the moving vehicles, this paper presents an acoustic feature extraction method, which integrates the nonlinear function with the MFCC method, termed as NMFCC. Through nonlinear transformation, the proposed method could amplify the acoustic features in a nonlinear way, and obtain more robust acoustic features, thereby achieving a superior classification performance. Comparing the NMFCC with the baseline feature extraction method, experimental results not only demonstrate the viability of the proposed algorithm but also show a satisfactory performance of the moving vehicle classification for diverse practical scenarios in the wild.

Depth Discrimination for Low-Frequency Sources Using a Horizontal Line Array of Acoustic Vector Sensors Based on Mode Extraction.

Depth discrimination is a key procedure in acoustic detection or target classification for low-frequency underwater sources. Conventional depth-discrimination methods use a vertical line array, which has disadvantage of poor mobility due to the size of the sensor array. In this paper, we propose a depth-discrimination method for low-frequency sources using a horizontal line array (HLA) of acoustic vector sensors based on mode extraction. First, we establish linear equations related to the modal amplitudes based on modal beamforming in the vector mode space. Second, we solve the linear equations by introducing the total least square algorithm and estimate modal amplitudes. Third, we select the power percentage of the low-order modes as the decision metric and construct testing hypotheses based on the modal amplitude estimation. Compared with a scalar sensor, a vector sensor improves the depth discrimination, because the mode weights are more appropriate for doing so. The presented linear equations and the solution algorithm allow the method to maintain good performance even using a relatively short HLA. The constructed testing hypotheses are highly robust against mismatched environments. Note that the method is not appropriate for the winter typical sound speed waveguide, because the characteristics of the modes differ from those in downward-refracting sound speed waveguide. Robustness analysis and simulation results validate the effectiveness of the proposed method.

Design of an Acoustic Target Classification System Based on Small-Aperture Microphone Array

The acoustic recognition module of the unattended ground sensor (UGS) system applied in wild environments is faced with the challenge of complicated noise interference. In this paper, a small-aperture microphone array (MA)-based acoustic target classification system, including the system hardware architecture and classification algorithm scheme, is designed as a node-level sensor for the application of UGS in noisy situation. Starting from the analysis of signature of the acoustic signal in wild environments and the merits of small-aperture array in noise reduction, a closely arranged microelectromechanical systems MA is designed to improve the signal quality. Considering the similarities between speaker discrimination and acoustic target recognition, a classification algorithm scheme, consisting of a simplified Mel-frequency cepstrum coefficients and the Gaussian mixture model, is developed to distinguish acoustic targets’ patterns. The proposed classification algorithm has been implemented on embedded system after being tested on training datasets. By combining the small-aperture array and low-complexity classification algorithm, the presented acoustic classification prototype system is portable and efficient. To demonstrate the efficiency of the design, the prototype system is verified in a practical situation with the employment of wheeled and tracked vehicles. Evaluation of the system performances in comparison with other state-of-the-art methods indicates that the proposed design is practical for the acoustic target classification and may be widely adopted by UGS.

A Robust Feature Extraction Algorithm for the Classification of Acoustic Targets in Wild Environments

The acoustic recognition technology of unattended ground sensor systems applied in wild environments is faced with the challenge of complicated and strong acoustic noise, especially wind noise. Moreover, the commonly used Mel-frequency cepstral coefficients (MFCCs) are sensitive to noise interference. To resolve the problem, a robust feature extraction method, called harmonic Mel-frequency cepstral coefficients (HMFCCs), is proposed for acoustic target classification. By combining an acoustic signal's harmonic model with the MFCC method, the HMFCC has the ability to emphasize the signals emitted by the principal acoustic components of the target. In the experiment conducted for this study, three data sets are sampled under the same conditions, except for wind power levels. Then the classifier, which is trained by one of the three data sets, is used to recognize the others data sets. According to the experimental results, the HMFCC-based classification accuracies of the three data sets are higher than those of other state-of-the-art methods, indicating that HMFCC is a kind of noise-insensitive feature.

Underwater acoustic target classification and auditory feature identification based on dissimilarity evaluation

The purpose of this study is to explore perceptual classification of underwater acoustic targets and auditory features used by human being. First, we design a paired comparison experiment. Then we use the CLASCAL algorithm to model the dissimilarity ratings as a perceptual space, and analyze the properties in three common dimensions, specialties, 3 subjects' latent classes and their roles in target perceptual classification. Finally, based on the gammatone filterbank, we find some auditory features that can effectively underlie 3 common dimensions and beat properties, so as to use them to construct a binary decision tree to classify some new samples; thus we can provide some guidance about how to use these features in practical applications.

Two kinds of timbre representations and its application into acoustic target classification

Being an important auditory attribute of sound, timbre exhibits great potential for classifying sound source and its suitable representation and parameterization are crucial for feature extraction. In this study, we express environmental sound's timbre in terms of verbal description and its projection in independent space, which are respectively referred to as Natural Timbre (NT) and Essential Timbre (ET). In this study, such two kinds of timbre expressions are applied to acoustic target recognition using synthesized steady-state underwater noise with two subjective rating experiments. First a semantic differential test is conducted and the NT-based target identification rates are achieved by forced clustering; then a paired comparison experiment is carried out to get the ET-based identification rates. Finally, the recognition performances for two kinds of timbre representations are compared and its advantages are discussed in association with feature extraction and acoustic target recognition.

음향 표적 식별을 위한 무선 센서 네트워크에서 웨이블릿 상수를 이용한 표적 특징 추출

Acoustic target classification in wireless sensor network is important research at environmental surveillance, invasion surveillance, multiple target separation. General sensor node signal processing methods concentrated on received signal energy based target detection and received raw signal compression. The former is not suited to target classification because of almost every target information are lost except target energy. The latter bring down life-time of sensor node owing to high computational complexity and transmission energy. In this paper, we introduce an feature extraction algorithm for acoustic target classification in wireless sensor network which has time and frequency information. The proposed method extracts time information and de-noised target classification information using wavelet decomposition step. This method reduces communication energy by 28% of original signal and computational complexity.