Environmental Sound Events Research Articles

Signal latent subspace: A new representation for environmental sound classification

In this study, we propose Signal Latent Subspace (SLS), a flexible method that classifies environmental sound events using the subspace representations of latent features obtained from various neural network-based models. Our main goal is to leverage the high expressiveness of neural networks while retaining the advantages of subspace representation, such as its robustness to noise and ability to work under small sample size (SSS) conditions. We also propose an ensemble strategy native to the subspace representation, to achieve increased performance and reduce the generalization error. We do this through product Grassmann manifold (PGM), resulting in SLS-PGM. Each subspace constructed from latent features of a network can be seen as a point on a factor Grassmann manifold (GM) of a neural network; through PGM, it is possible to unify factor manifolds into a singular representation, and perform classification through a similarity metric on the manifold. We further improve SLS and SLS-PGM in two ways: (1) by using generalized difference subspace (GDS) projection to address the lack of between-class discrimination of subspace representation and (2) by leveraging finetuning regimes to better adapt neural network models to the ESC task. We evaluate our proposed methods, factoring various neural networks, on ESC-10, ESC-50 and UrbanSound environmental sound datasets, and provide extensive ablation experiments and notes for practical use.

Adaptive Noise Reduction Algorithm Based on SPP and NMF for Environmental Sound Event Recognition under Low-SNR Conditions

Nonnegative matrix factorization- (NMF-) based noise reduction methods can effectively improve the performance of environmental sound recognition. However, when the environmental sound overlaps highly with the noise, the spectral line loss and noise residue will occur in the low signal-to-noise ratio (SNR) condition. An adaptive noise reduction algorithm was proposed in this paper. First, noisy environmental sound is separated into estimated noise and environmental sound using NMF. Then, the estimated noise is used to calculate sound presence probability (SPP), which is adapted to decrease spectral line loss and achieve accurately estimated noise. Subsequently, the estimated noise combines with noisy environmental sound to obtain the estimated environmental sound. Finally, SPP is applied to reduce residual noise in the estimated environmental sound and reconstruct the environmental sound. The simulation results demonstrate that the proposed algorithm outperforms the traditional algorithms and NMF-based methods in terms of perceptual evaluation of speech quality (PESQ) and global SNR with increase of X% and X%, respectively. Moreover, the proposed method can effectively improve the environmental sound recognition rate. Particularly, the proposed method makes a 16.2% increase of F1-score in car horn recognition under the realistic acoustic condition.

Sound Event Classification Based on Frequency-Energy Feature Representation and Two-Stage Data Dimension Reduction

The classification of environmental sound events is of great significance for applications such as machine hearing and acoustic surveillance. Feature representation and feature vector dimension directly affect system performance. To better extract features and reduce computational burden, a novel frequency-energy feature representation and two-stage dimension reduction system were proposed. First, a frequency-energy diagram is generated. Based on this, the importance screening is done and only the energy bins of high importance are retained, which reduces the dimension of feature vector while extracting key information. Then the Bicubic interpolation method is used to further reduce the dimension. And the appropriate feature vector dimension is determined based on the change of information entropy. The proposed frequency-energy feature representation and two-stage dimension reduction system are evaluated with Real Word Computing Partnership sound scene database (RWCP-SSD), UrbanSound8K, and ESC-50 datasets, which demonstrate that the robustness is satisfactory under low signal-to-noise ratios (SNRs) and 15 noise types from NOISEX-92 database.

Multi-view representation for sound event recognition

The sound event recognition (SER) task is gaining lot of importance in emerging applications such as machine audition, audio surveillance, and environmental audio scene recognition. The recognition of sound events with noisy conditions in real-time surveillance applications is a difficult task. In this paper, we focus on learning patterns using multiple forms (views) of the given sound events. We propose two variants of the Multi-View Representation (MVR)-based approach for the SER task. The first variant combines the auditory image-based features and the cepstral features from sound signal. The second variant combines the statistical features extracted from the auditory images and the cepstral features of sound signal. In addition to these variants, Constant Q-transform and Variable Q-transform image-based features are also explored to study the other effective forms of multi-view representations. A discriminative model-based classifier is then used to recognize these representations as environmental sound events. The performance of the proposed MVR approaches is evaluated on three benchmark sound event datasets namely ESC-50, DCASE2016 Task 2, and DCASE2018 Task 2 for the SER task. The recognition accuracy of the proposed MVR approach is significantly better than the other approaches proposed in the recent literature.

Generative Model Driven Representation Learning in a Hybrid Framework for Environmental Audio Scene and Sound Event Recognition

The analysis of sound information is helpful for audio surveillance, multimedia information retrieval, audio tagging, and forensic applications. Environmental audio scene recognition (EASR) and sound event recognition (SER) for audio surveillance are challenging tasks due to the presence of multiple sound sources, background noises, and the existence of overlapping or polyphonic contexts. We focus on learning robust and compact representations for environmental audio scenes and sound events using mel-frequency cepstral coefficients as basic features, which have proved to be effective in speech and audio-related tasks. In this paper, we propose a common hybrid model-based framework that learns representations with the help of generative models. We explore instance-specific adapted Gaussian mixture models for environmental audio scenes and instance-specific hidden Markov models for sound events to compute a robust, compact, and discriminatory representations. A discriminative model based classifier is then used to recognize these representations as environmental audio scenes and sound events. The performance of the proposed approaches is evaluated using the DCASE2013 scene dataset and TUT-DCASE2016 scene dataset for EASR task. Environmental Sound Classification (ESC-10) and UrbanSound8K datasets are used for SER task. The recognition accuracy of the proposed framework is significantly better than many of the state-of-the-art approaches proposed in the recent literature. The discriminative nature of the model-driven representations leads to improved efficiency for EASR and SER task. The proposed approaches are more suitable for tasks with less training data.

Environmental Audio Scene and Sound Event Recognition for Autonomous Surveillance

Monitoring of human and social activities is becoming increasingly pervasive in our living environment for public security and safety applications. The recognition of suspicious events is important in both indoor and outdoor environments, such as child-care centers, smart-homes, old-age homes, residential areas, office environments, elevators, and smart cities. Environmental audio scene and sound event recognition are the fundamental tasks involved in many audio surveillance applications. Although numerous approaches have been proposed, robust environmental audio surveillance remains a huge challenge due to various reasons, such as various types of overlapping audio sounds, background noises, and lack of universal and multi-modal datasets. The goal of this article is to review various features of representing audio scenes and sound events and provide appropriate machine learning algorithms for audio surveillance tasks. Benchmark datasets are categorized based on the real-world scenarios of audio surveillance applications. To have a quantitative understanding, some of the state-of-the-art approaches are evaluated based on two benchmark datasets for audio scenes and sound event recognition tasks. Finally, we outline the possible future directions for improving the recognition of environmental audio scenes and sound events.

Learnt dictionary based active learning method for environmental sound event tagging

Sound event tagging is a process that adds texts or labels to sound segments based on their salient features and/or annotations. In the real world, since annotating cost is much expensive, tagged sound segments are limited, while untagged sound segments can be obtained easily and inexpensively. Thus, semi-automatic tagging becomes very important, which can assign labels to massive untagged sound segments according to a small number of manually annotated sound segments. Active learning is an effective technique to solve this problem, in which selected sound segments are manually tagged while other sound segments are automatically tagged. In this paper, a learnt dictionary based active learning method is proposed for environmental sound event tagging, which can significantly reduce the annotating cost in the process of semi-automatic tagging. The proposed method is based on a learnt dictionary, as dictionary learning is more adapt to sound feature extraction. Moreover, tagging accuracy and annotating cost are used to measure the performance of the proposed method. Experimental results demonstrate that the proposed method has higher tagging accuracy but requires much less annotating cost than other existing methods.

Listening to the World Improves Speech Command Recognition

We study transfer learning in convolutional network architectures applied to the task of recognizing audio, such as environmental sound events and speech commands. Our key finding is that not only is it possible to transfer representations from an unrelated task like environmental sound classification to a voice-focused task like speech command recognition, but also that doing so improves accuracies significantly. We also investigate the effect of increased model capacity for transfer learning audio, by first validating known results from the field of Computer Vision of achieving better accuracies with increasingly deeper networks on two audio datasets: UrbanSound8k and Google Speech Commands. Then we propose a simple multiscale input representation using dilated convolutions and show that it is able to aggregate larger contexts and increase classification performance. Further, the models trained using a combination of transfer learning and multiscale input representations need only 50% of the training data to achieve similar accuracies as a freshly trained model with 100% of the training data. Finally, we demonstrate a positive interaction effect for the multiscale input and transfer learning, making a case for the joint application of the two techniques.

Environmental sound event detection in wireless acoustic sensor networks for home telemonitoring

In this paper, we present an approach to improve the accuracy of environmental sound event detection in a wireless acoustic sensor network for home monitoring. Wireless acoustic sensor nodes can capture sounds in the home and simultaneously deliver them to a sink node for sound event detection. The proposed approach is mainly composed of three modules, including signal estimation, reliable sensor channel selection, and sound event detection. During signal estimation, lost packets are recovered to improve the signal quality. Next, reliable channels are selected using a multi-channel cross-correlation coefficient to improve the computational efficiency for distant sound event detection without sacrificing performance. Finally, the signals of the selected two channels are used for environmental sound event detection based on bidirectional gated recurrent neural networks using two-channel audio features. Experiments show that the proposed approach achieves superior performances compared to the baseline.

Children as Earwitnesses: Memory for Emotional Auditory Events

SummaryWe examined the influence of age and emotionality of auditory stimuli on long‐term memory for environmental sound events. Sixty children aged 7–11 years were presented with two environmental sound events: an emotional car crash and a neutral event, someone brushing their teeth. The sound events comprised six individual environmental sounds, and the participants passively listened to the sound events through a headset. After a two‐week delay, participants performed a cued recall task and a recognition task. Independent of age, children were notably poor at recalling the sound events. Children recalled and recognized significantly more sounds from the emotional sound event than the neutral sound event. Additionally, the older children performed the recall task better than the younger children. The present findings confirm and expand the previously reported superiority of emotional material in memory.Copyright © 2016 John Wiley & Sons, Ltd.

Identifying, rating, and remembering environmental sound events

Sound events are sequences of closely grouped and temporally related environmental sounds that tell a story or establish a sense of place. The goal of our project was to create a set of sound events depicting various scenarios (such as a car accident, cooking breakfast, and walking outdoors) and to gather normative data about how people understand them. Samples of college students listened to 22 sound events over headphones in three self-paced, computer-based studies. In the Identification Task, 43 participants used text boxes to type descriptions of what was happening in the sound events. In the Rating Task, 39 participants used Likert scales to rate the sound events on the attributes of familiarity, complexity, and pleasantness. In the Memory Task, 42 participants answered two multiple-choice questions immediately after listening to each sound event. Detailed tables are provided for the following: (1) Description of the sound events and their components; (2) accuracy and response time measurements for each of the 22 sound events across the three studies; and (3) rank-orderings of the sound events by ease of identification, recognition of details, and rated familiarity, complexity, and pleasantness. Digital files of the stimuli, which may be of interest to auditory cognition researchers and clinical neuropsychologists, may be downloaded from either www.psychonomic.org/archive or www.cofc.edu/-marcellm/sound event studies/sndevent.htm.

The Synthesis of Environmental Sound Textures by Iterated Nonlinear Functions, and its Ecological Relevance to Perceptual Modeling

This paper overviews an approach of digital sound synthesis based on iterated nonlinear functions, and describes its use in the creation of sounds of textural type. Due to the nonlinear dynamics in the iterated process, time-changing sonorities are synthesized reminiscent of environmental sound events and effects of “acoustic turbulence.” This opens to the modeling of perceptual attributes of complex auditory images. The experiments documented here are drawn from the author's computer music composing. They may extend to the creation of synthetic, but credible, auditory scenes in multimedia applications, virtual reality and film soundtracks. However, the paper mainly emphasizes a systemic view of sound synthesis. By arguing that special attention should be payed to the dynamics internal to the sound generating mechanism, it raises the issue of the ecological relevance of the sound synthesis techniques to an approach of perceptual modeling.