Audio Event Detection Research Articles

Fish feeding behavior quantification and automatic acoustic event detection: algorithmic optimization with deep learning

In recent years, the integration of sensor processing technologies with intelligent feeding control systems has become crucial for enhancing recirculating aquaculture systems (RAS). Intelligent feeding control significantly improves profitability and animal welfare in aquaculture by accurately monitoring feed intake and reducing waste. Acoustic methods have proven effective in automating feed monitoring, particularly in assessing feeding intensity. Automatic Audio Event Detection (AED) holds promise for identifying sound events associated with feeding activities, enabling precise monitoring of feeding behavior and estimation of feed consumption through frequency-based feeding signatures. Advancements in sensor technologies and computational capabilities enable real-time collection, processing, and analysis of acoustic data. However, handling high volumes of data with minimal latency and high accuracy poses challenges. To address these challenges, we propose a systematic approach: (1) Data preprocessing using wavelet transforms enhances efficiency and reduces computational overhead. (2) Data pruning focuses on informative features and reduces input data dimensionality, improving classification accuracy. (3) We adopt a weakly-labeled semi-supervised learning model with Convolutional Neural Networks (CNNs) to detect feeding events, improving classification performance. (4) Ensemble learning combines multiple classifiers to mitigate overfitting and enhance model generalization. Our methodology shows promise in accurately quantifying fish feeding behavior using acoustic signals.

Analysis of representation and generalization capabilities of pre-trained audio models in urban environments

In the last decade, urban noise pollution has become a significant environmental concern that can be mitigated with the help of audio detection algorithms for classifying different sources of noise and creating more informative noise maps. In this context, machine learning, particularly transfer learning, is an essential technology that enables accurate analysis of urban noise sources. However, the choice of the pre-trained model used to compute audio embeddings can significantly influence the performance of downstream classification tasks. This paper aims to compare the embeddings of various pre-trained models on different data collection campaigns in the context of the Sons al balcó project and quantify the robustness of audio representations. To achieve this, we develop metrics and statistically test the presence of distribution shifts in learned latent features. To evaluate the quality of the embeddings, we perform both qualitative and quantitative analysis using dimensionality reduction methods and assess the performance on downstream tasks using data from different collection campaigns. Results highlight major differences between general purpose and specific models. Our findings suggest the need for careful consideration during the choice of the pre-trained model to use in audio event detection applications.

Listen as you wish: Fusion of audio and text for cross-modal event detection in smart cities

In the era of smart cities, the advent of the Internet of Things technology has catalyzed the proliferation of multimodal sensor data, presenting new challenges in cross-modal event detection, particularly in audio event detection via textual queries. This paper focuses on the novel task of text-to-audio grounding (TAG), aiming to precisely localize sound segments that correspond to events described in textual queries within an untrimmed audio. This challenging new task requires multi-modal (acoustic and linguistic) information fusion as well as the reasoning for the cross-modal semantic matching between the given audio and textual query. Unlike conventional methods that often overlook the nuanced interactions between and within modalities, we introduce the Cross-modal Graph Interaction (CGI) model. This innovative approach leverages a language graph to model complex semantic relationships between query words, enhancing the understanding of textual queries. Additionally, a cross-modal attention mechanism generates snippet-specific query representations, facilitating fine-grained semantic matching between audio segments and textual descriptions. A cross-gating module further refines this process by emphasizing relevant features across modalities and suppressing irrelevant information, optimizing multimodal information fusion. Our comprehensive evaluation on the Audiogrounding benchmark dataset not only demonstrates the CGI model’s superior performance over existing methods, but also underscores the significance of sophisticated multimodal interaction in improving the efficacy of TAG in smart cities.

Multi-rate modulation encoding via unsupervised learning for audio event detection

Technologies in healthcare, smart homes, security, ecology, and entertainment all deploy audio event detection (AED) in order to detect sound events in an audio recording. Effective AED techniques rely heavily on supervised or semi-supervised models to capture the wide range of dynamics spanned by sound events in order to achieve temporally precise boundaries and accurate event classification. These methods require extensive collections of labeled or weakly labeled in-domain data, which is costly and labor-intensive. Importantly, these approaches do not fully leverage the inherent variability and range of dynamics across sound events, aspects that can be effectively identified through unsupervised methods. The present work proposes an approach based on multi-rate autoencoders that are pretrained in an unsupervised way to leverage unlabeled audio data and ultimately learn the rich temporal dynamics inherent in natural sound events. This approach utilizes parallel autoencoders that achieve decompositions of the modulation spectrum along different bands. In addition, we introduce a rate-selective temporal contrastive loss to align the training objective with event detection metrics. Optimizing the configuration of multi-rate encoders and the temporal contrastive loss leads to notable improvements in domestic sound event detection in the context of the DCASE challenge.

Greedy regression and differential convex-based deep learning for audio event classification

Audio Event Detection (AED) and classification of acoustic events has become a notable task for machines to interpret the auditory information around us. Nevertheless, it has been a difficult and cumbersome task to extract the most basic characteristics of acoustic events that encapsulate the fundamental elements of the audio events. Previous works on audio event classification utilized supervised pre-training as well as meta-learning approaches that happened to depend on labeled data therefore facing instability. Deep Learning is progressing in an increasingly mature direction, and the application of deep learning methods to detect acoustic event has become more and more sought after. The proposed hybrid method called Greedy Regression-based Convolutional Neural Network and Differential Convex Bidirectional Gated Recurrent Unit (GRCNN-DCBGRU) is introduced to learn a vector representation of an audio sequence for Audio Event Classification (AEC). Differential Convex Bidirectional Gated Recurrent Unit is analogous to long short-term memory and involves time-cyclic long-term dependencies with a lesser processing complexity. The model first extracts acoustic features from the sound event dataset through a Differential Convex Bidirectional Gated Recurrent Unit employing Gabor Filter bank features and then extracts the local static acoustic features through the Greedy Regression-based Convolutional Neural Network by utilizing Mel Frequency Cepstral Coefficients (MFCC). Finally, the Differential Convex Meta-Learning classifier is used for the final acoustic event classification. Extensive evaluation on large-size publicly available acoustic event database like Findsounds2016 will be performed in Python programming language to demonstrate the efficiency of the proposed method for the AEC task. To demonstrate the visualizations of individual modules and their influence on overall representation learning for AEC tasks, several parameters like audio detection time, audio detection accuracy, precision, and recall are measured.

A generic AI pipeline for identifying known species and discovering new species

To counterattack the current biodiversity crisis, we need innovative tools and technologies that can greatly improve species identification and discovery at scale. Here, we describe a data analysis pipeline to automatically identify biological sound categories in soundscape data and accelerate the discovery of new species. The pipeline consists of two components: (1) Unsupervised audio event detection and clustering that enables automatic detection and categorization of biological sounds in large audio datasets and (2) supervised deep learning-based signal detection that enables automated cluster labeling and detection of known sound categories in new audio. We evaluated the results of our unsupervised analysis tools applied to a labeled dataset of ∼18 h of recordings from Panama. Experts assigned a label to each sound event with a max frequency of 6 kHz. Our results show that clusters generally had high homogeneity and completeness and indicate that our cluster analysis tool can successfully detect and group like-sounds together, improving data processing and species identification. In addition, we were able to implement these analytical tools in a user-friendly data analysis platform that will provide scientists, wildlife managers, conservationists, and public and private environmental organizations with the information they need to make informed conservation and management decisions.

Attention mechanism combined with residual recurrent neural network for sound event detection and localization

In the task of sound event detection and localization (SEDL) in a complex environment, the acoustic signals of different events usually have nonlinear superposition, so the detection and localization effect is not good. Given this, this paper is based on the Residual-spatially and channel Squeeze-Excitation (Res-scSE) model. Combined with Multiple-scale Convolutional Recurrent Neural Network (M-CRNN), the Res-scSE-CRNN model is proposed. Firstly, to solve the problem of insufficient extraction of time-frequency feature in single-size convolution kernel, multi-scale feature fusion is carried out by using the feature hierarchy of the convolutional neural network to improve the accuracy of detection. Secondly, aiming at the problem of overlapping audio event localization accuracy is not high, with Res-scSE to replace common convolution module and add residual structure to strengthen the feature extraction, and combining with an attention mechanism to enhance neural network channels and spatial relationships, to improve the network to extract the characteristics of directivity, achieve the goal of the overlapped audio localization. In this paper, experiments are carried out in the open dataset DCASE2019, and evaluation indicators are used to analyze the effectiveness of the proposed model and baseline model in the detection and localization of audio events. The results show that compared with the M-CRNN model, the detection error rate of Res-scSE-CRNN model is reduced 4%, the F1-Score is increased 3.4%, the localization error is reduced by 22.8°, and the frame recall rate is increased 3%.

Deep learning technique based real-time audio event detection experiment in a distributed system architecture

The paper investigates a noise-robust and bandwidth-efficient audio event detection technique. In this work, audio features are extracted from audio data using a gammatone filter bank. The proposed denoising autoencoder-based fully connected deep neural network scheme is realized in a distributed architecture. Audio features are extracted using a Raspberry Pi edge computing device and those are fed to a local data server to detect audio events. Experiments have shown that feature extraction and event classification tasks using the proposed technique take 840 ms for a 2-second audio clip that promises real-time implementation. The prototype model shows improved classification probability of noisy audio events with an accuracy of 98.12% and an F1-score of 90.6% at a 10 dB signal-to-noise-ratio. The performance of the proposed technique is compared with state-of-the-art techniques using the Freiburg-106 dataset and improved noise-robust audio event detection performance is found.

Acoustic Event Detection Using Machine Learning

Nowadays Audio event detection is playing an important vital role in research area it has become the main part of machine learning which plays an important role in everyday life it consists of audio tagging, classified music, emotional speech, audio sounds. Convolutional neural networks are proposed and applied on sound event detection complications. This system detects sound events such has Laughter, crying sounds of humans, Singing of Birds, Firing, speaking sounds, speech, blast and boom sounds even including animals and birds’ sounds were also detected it can also include news broadcasting, each and every situation were included. Sometimes sounds might overlap at that time it becomes hard to detect the overlapped sound events so such problems can be solved by using CNN models.

Hand-crafted versus learned representations for audio event detection

Hand-crafted versus learned representations for audio event detection

ANALYSIS OF THE SOUND EVENT DETECTION METHODS AND SYSTEMS

Detection and recognition of loud sounds and characteristic noises can significantly increase the level of safety and ensure timely response to various emergency situations. Audio event detection is the first step in recognizing audio signals in a continuous audio input stream. This article presents a number of problems that are associated with the development of sound event detection systems, such as the deviation for each environment and each sound category, overlapping audio events, unreliable training data, etc. Both methods for detecting monophonic impulsive audio event and polyphonic sound event detection methods which are used in the state-of-the-art sound event detection systems are presented. Such systems are presented in Detection and Classification of Acoustic Scenes and Events (DCASE) challenges and workshops, which take place every year. Beside a majority of works focusing on the improving overall performance in terms of accuracy many other aspects have also been studied. Several systems presented at DCASE 2021 task 4 were considered, and based on their analysis, there was a conclusion about possible future for sound event detection systems. Also the actual directions in the development of modern audio analytics systems are presented, including the study and use of various architectures of neural networks, the use of several data augmentation techniques, such as universal sound separation, etc.

Audio event detection using deep learning model

Audio event detection using deep learning model

Audio event detection using deep learning model

Audio event detection using deep learning model

Device Invariant Deep Neural Networks for Pulmonary Audio Event Detection Across Mobile and Wearable Devices.

Mobile and wearable devices are being increasingly used for developing audio based machine learning models to infer pulmonary health, exacerbation and activity. A major challenge to widespread usage and deployment of such pulmonary health monitoring audio models is to maintain accuracy and robustness across a variety of commodity devices, due to the effect of device heterogeneity. Because of this phenomenon, pulmonary audio models developed with data from one type of device perform poorly when deployed on another type of device. In this work, we propose a framework incorporating feature normalization across individual frequency bins and combining task specific deep neural networks for model invariance across devices for pulmonary event detection. Our empirical and extensive experiments with data from 131 real pulmonary patients and healthy controls show that our framework can recover up to 163.6% of the accuracy lost due to device heterogeneity for four different pulmonary classification tasks across two broad classification scenarios with two common mobile and wearable devices: smartphone and smartwatch.Clinical relevance- The methods presented in this paper will enable efficient and easy portability of clinician recommended pulmonary audio event detection and analytic models across various mobile and wearable devices used by a patient.

Disrupting Audio Event Detection Deep Neural Networks with White Noise

Audio event detection (AED) systems can leverage the power of specialized algorithms for detecting the presence of a specific sound of interest within audio captured from the environment. More recent approaches rely on deep learning algorithms, such as convolutional neural networks and convolutional recurrent neural networks. Given these conditions, it is important to assess how vulnerable these systems can be to attacks. As such, we develop AED-suited convolutional neural networks and convolutional recurrent neural networks, and attack them next with white noise disturbances, conceived to be simple and straightforward to be implemented and employed, even by non-tech savvy attackers. We develop this work under a safety-oriented scenario (AED systems for safety-related sounds, such as gunshots), and we show that an attacker can use such disturbances to avoid detection by up to 100 percent success. Prior work has shown that attackers can mislead image classification tasks; however, this work focuses on attacks against AED systems by tampering with their audio rather than image components. This work brings awareness to the designers and manufacturers of AED systems, as these solutions are vulnerable, yet may be trusted by individuals and families.

Efficient multichannel detection of impulsive audio events for wireless networks

Impulsive audio events such as gunshots, explosions or glass shattering, are commonly associated with security threats, thus they are of particular interest for automated acoustic surveillance. Even though impulsive audio events are greatly influenced by their propagation path, little work has been done in multichannel detection, and most precedents available in the literature deal with single-channel detection systems. Unfortunately, the spatial dependence of impulsive sound recordings proves as a problem for robust performance under real conditions. It is possible, however, to take advantage of the spatial diversity provided by a wireless sensor network to counteract this problem. In this paper, we show how an ensemble of spatially diverse detectors can greatly improve the performance of the system. We propose an efficient multichannel detection system of impulsive audio events intended for low-cost Wireless Acoustic Sensor Networks. Our proposal is based on a low complexity classification algorithm and an efficient method to include temporal context into the feature vector. The obtained results show that the proposed detection system is capable of achieving a more than adequate performance without incurring large computational loads.

Soft computing based audio signal analysis for accident prediction

PurposeRoad accidents, an inadvertent mishap can be detected automatically and alerts sent instantly with the collaboration of image processing techniques and on-road video surveillance systems. However, to rely exclusively on visual information especially under adverse conditions like night times, dark areas and unfavourable weather conditions such as snowfall, rain, and fog which result in faint visibility lead to incertitude. The main goal of the proposed work is certainty of accident occurrence.Design/methodology/approachThe authors of this work propose a method for detecting road accidents by analyzing audio signals to identify hazardous situations such as tire skidding and car crashes. The motive of this project is to build a simple and complete audio event detection system using signal feature extraction methods to improve its detection accuracy. The experimental analysis is carried out on a publicly available real time data-set consisting of audio samples like car crashes and tire skidding. The Temporal features of the recorded audio signal like Energy Volume Zero Crossing Rate 28ZCR2529 and the Spectral features like Spectral Centroid Spectral Spread Spectral Roll of factor Spectral Flux the Psychoacoustic features Energy Sub Bands ratio and Gammatonegram are computed. The extracted features are pre-processed and trained and tested using Support Vector Machine (SVM) and K-nearest neighborhood (KNN) classification algorithms for exact prediction of the accident occurrence for various SNR ranges. The combination of Gammatonegram with Temporal and Spectral features of the validates to be superior compared to the existing detection techniques.FindingsTemporal, Spectral, Psychoacoustic features, gammetonegram of the recorded audio signal are extracted. A High level vector is generated based on centroid and the extracted features are classified with the help of machine learning algorithms like SVM, KNN and DT. The audio samples collected have varied SNR ranges and the accuracy of the classification algorithms is thoroughly tested.Practical implicationsDenoising of the audio samples for perfect feature extraction was a tedious chore.Originality/valueThe existing literature cites extraction of Temporal and Spectral features and then the application of classification algorithms. For perfect classification, the authors have chosen to construct a high level vector from all the four extracted Temporal, Spectral, Psycho acoustic and Gammetonegram features. The classification algorithms are employed on samples collected at varied SNR ranges.

Hardware-Optimized Reservoir Computing System for Edge Intelligence Applications

Edge artificial intelligence or edge intelligence is an ever-growing research area due to the current popularization of the Internet of Things. Unfortunately, incorporation of artificial intelligence (AI) in smart devices operating at the edge is a challenging task due to the power-hungry characteristics of deep learning implementations, such as convolutional neural networks (CNNs). As a feasible alternative, reservoir computing (RC) has attracted a lot of attention in the field of machine learning due to its promising performance in a wide range of applications. In this work, we propose a simple hardware-optimized circuit design of RC systems presenting high energy-efficiency capacities that fulfill the low power requirements of edge intelligence applications. As a proof of concept, we used the proposed design for the implementation of a low-power audio event detection (AED) application in FPGA. The measurements and simulation results obtained show that the proposed approach may provide significant accuracy with the advantage of presenting ultra-low-power characteristics (the energy efficiency estimated is below the microjoule per inference). These results make the proposed system optimal for edge intelligence applications in which energy efficiency and accuracy are the key issues.

Detection and classification of human-produced nonverbal audio events

Audio wearable devices, or hearables, are becoming an increasingly popular consumer product. Some of these hearables contain an in-ear microphone to capture audio signals inside the user’s occluded earcanal. Mainly, the microphone is used to pick up speech in noisy environments, but it can also capture other signals, such as nonverbal events that could be used to interact with the device or a computer. Teeth or tongue clicking could be used to interact with a device in a discreet manner, and coughing or throat-clearing sounds could be used to monitor the health of a user. In this paper, 10 human produced nonverbal audio events are detected and classified in real-time with a classifier using the Bag-of-Audio-Words algorithm. To build this algorithm, different clustering and classification methods are compared. Mel-Frequency Cepstral Coefficient features are used alongside Auditory-inspired Amplitude Modulation features and Per-Channel Energy Normalization features. To combine the different features, concatenation performance at the input level and at the histogram level is compared. The real-time detector is built using the detection by classification technique, classifying on a 400 ms window with 75% overlap. The detector is tested in a controlled noisy environment on 10 subjects. The classifier had a sensitivity of 81.5% while the detector using the same classifier had a sensitivity of 69.9% in a quiet environment.

Audio Event Detection Based on Attention CRNN

Audio Event Detection Based on Attention CRNN