Sound Event Recognition Research Articles

Recognition of marine sound events using an echo state reservoir and a K-nearest neighbor classifier

A sound event detection and classification algorithm was developed to sift through passive acoustic data, and to extract and classify signals of interest. Segments of data containing interesting sound events were identified from raw audio files using framing techniques and standard deviation-based event detection, and statistical acoustic features from the temporal and spectral domains were extracted for each segment. An adapted cyclical reservoir model was employed to accumulate time varying information over the duration of each sound event, evolving raw features to an echo state space representation with a fading temporal memory. A K-Nearest Neighbor classifier was trained on the echo states extracted from a library of recordings corresponding to broad classes of common ambient marine sound types and self-produced noise from autonomous underwater vehicles. A simple majority voting scheme was then utilized to obtain a single representative class label for each sound event. Validation of the classifier using available underwater acoustic libraries and passive acoustic datasets collected using Seagliders equipped with acoustic sensors indicates small improvements in both the per-frame and per-label performance for a model trained on states over one trained on raw features.A sound event detection and classification algorithm was developed to sift through passive acoustic data, and to extract and classify signals of interest. Segments of data containing interesting sound events were identified from raw audio files using framing techniques and standard deviation-based event detection, and statistical acoustic features from the temporal and spectral domains were extracted for each segment. An adapted cyclical reservoir model was employed to accumulate time varying information over the duration of each sound event, evolving raw features to an echo state space representation with a fading temporal memory. A K-Nearest Neighbor classifier was trained on the echo states extracted from a library of recordings corresponding to broad classes of common ambient marine sound types and self-produced noise from autonomous underwater vehicles. A simple majority voting scheme was then utilized to obtain a single representative class label for each sound event. Validation of the classifier us...

Generative Model Driven Representation Learning in a Hybrid Framework for Environmental Audio Scene and Sound Event Recognition

The analysis of sound information is helpful for audio surveillance, multimedia information retrieval, audio tagging, and forensic applications. Environmental audio scene recognition (EASR) and sound event recognition (SER) for audio surveillance are challenging tasks due to the presence of multiple sound sources, background noises, and the existence of overlapping or polyphonic contexts. We focus on learning robust and compact representations for environmental audio scenes and sound events using mel-frequency cepstral coefficients as basic features, which have proved to be effective in speech and audio-related tasks. In this paper, we propose a common hybrid model-based framework that learns representations with the help of generative models. We explore instance-specific adapted Gaussian mixture models for environmental audio scenes and instance-specific hidden Markov models for sound events to compute a robust, compact, and discriminatory representations. A discriminative model based classifier is then used to recognize these representations as environmental audio scenes and sound events. The performance of the proposed approaches is evaluated using the DCASE2013 scene dataset and TUT-DCASE2016 scene dataset for EASR task. Environmental Sound Classification (ESC-10) and UrbanSound8K datasets are used for SER task. The recognition accuracy of the proposed framework is significantly better than many of the state-of-the-art approaches proposed in the recent literature. The discriminative nature of the model-driven representations leads to improved efficiency for EASR and SER task. The proposed approaches are more suitable for tasks with less training data.

Environmental Audio Scene and Sound Event Recognition for Autonomous Surveillance

Monitoring of human and social activities is becoming increasingly pervasive in our living environment for public security and safety applications. The recognition of suspicious events is important in both indoor and outdoor environments, such as child-care centers, smart-homes, old-age homes, residential areas, office environments, elevators, and smart cities. Environmental audio scene and sound event recognition are the fundamental tasks involved in many audio surveillance applications. Although numerous approaches have been proposed, robust environmental audio surveillance remains a huge challenge due to various reasons, such as various types of overlapping audio sounds, background noises, and lack of universal and multi-modal datasets. The goal of this article is to review various features of representing audio scenes and sound events and provide appropriate machine learning algorithms for audio surveillance tasks. Benchmark datasets are categorized based on the real-world scenarios of audio surveillance applications. To have a quantitative understanding, some of the state-of-the-art approaches are evaluated based on two benchmark datasets for audio scenes and sound event recognition tasks. Finally, we outline the possible future directions for improving the recognition of environmental audio scenes and sound events.

Common sounds in bedrooms (CSIBE) corpora for sound event recognition of domestic robots

Although sound event recognition attracted much attention in the scientific community, applications in the robotics domain have not been in the focus. A new database was published in this paper and classifiers were evaluated with this dataset to guide the future practical developments of domestic robots. A corpus (CSIBE-RAW) was collected from the internet to build acoustic models to recognize 13 sound events and omit ambient sounds. As a case study, CSIBE-RAW was rerecorded in four room settings (CSIBE-AIBO) to create reverberation-tolerant classifiers for a Sony ERS-7. After eight classifiers were reviewed, the convolutional neural network achieved the best accuracy (95.07%) after multi-conditional learning and it was suitable for real-time classification on the robot. The effects of lossy audio codecs were studied, lossy encoder-tolerant audio statistics were specified for the feature vector and the Ogg Vorbis encoder with 128 kbit VBR was found superior to store big data and avoid any significant accuracy loss with the compression ratio 1:8.

Sound Event Recognition Using Auditory-Receptive-Field Binary Pattern and Hierarchical-Diving Deep Belief Network

Automatic sound event recognition (SER) has recently attracted renewed interest. Although practical SER system has many useful applications in everyday life, SER is challenging owing to the variations among sounds and noises in the real-world environment. This paper presents a novel feature extraction and classification method to solve the problem of SER. An audio–visual descriptor, called the auditory-receptive-field binary pattern, is designed based on the spectrogram image feature, the cepstral features, and the human auditory receptive field model. The extracted features are then fed into a classifier to perform event classification. The proposed classifier, called the hierarchical-diving deep belief network, is a deep neural network system that hierarchically learns the discriminative characteristics from physical feature representation to the abstract concept. The performance of our proposed system was verified using several experiments under various conditions. Using the RWCP dataset, the proposed system achieved a recognition rate of 99.27% for real-world sound data in 105 categories. Under noisy conditions, the developed system is very robust, with which it achieved 95.06% recognition rate with 0 dB signal-to-noise ratio. Using the TUT sound event dataset, the proposed system achieves error rates of 0.81 and 0.73 in sound event detection in home and residential area scenes. The experimental results reveal that the proposed system outperformed the other systems in this field.

Auditory Suspicious Event Databases: DASE and Bi-DASE

The lack of open-access databases for auditory suspicious events’ detection and classification algorithms is a hurdle for researchers while evaluating and comparing the performance of suspicious sound event detection and recognition systems. This paper introduces two databases called DASE and bi-DASE. DASE contains nine of the most common acoustic suspicious events with a total of 3105 sound event recordings of glass breaking, dog barking, scream, gunshot, explosion, alarm, sirens, door slams, and footsteps. Bi-DASE database is prepared by mixing two auditory suspicious events from DASE to generate 25 possible real-life scenarios. Both databases are divided into two sets as a training set and testing set. ANOVA and regression analysis are performed in regard to commonly used features, energy, 12 MFCCs, pitch range, spectral centroid frequency, spectral spread, spectral flatness, tonality, harmonics, zero-crossing rate, and fundamental frequency. All features are found to be significantly important based on their $p$ -values for both databases, which are evaluated by designing two baseline systems using support vector machines and $k$ -nearest neighbors algorithm. Experimental results are presented. The databases are available for free to the research community upon request.

Continuous robust sound event classification using time-frequency features and deep learning

The automatic detection and recognition of sound events by computers is a requirement for a number of emerging sensing and human computer interaction technologies. Recent advances in this field have been achieved by machine learning classifiers working in conjunction with time-frequency feature representations. This combination has achieved excellent accuracy for classification of discrete sounds. The ability to recognise sounds under real-world noisy conditions, called robust sound event classification, is an especially challenging task that has attracted recent research attention. Another aspect of real-word conditions is the classification of continuous, occluded or overlapping sounds, rather than classification of short isolated sound recordings. This paper addresses the classification of noise-corrupted, occluded, overlapped, continuous sound recordings. It first proposes a standard evaluation task for such sounds based upon a common existing method for evaluating isolated sound classification. It then benchmarks several high performing isolated sound classifiers to operate with continuous sound data by incorporating an energy-based event detection front end. Results are reported for each tested system using the new task, to provide the first analysis of their performance for continuous sound event detection. In addition it proposes and evaluates a novel Bayesian-inspired front end for the segmentation and detection of continuous sound recordings prior to classification.

Lanczos kernel based spectrogram image features for sound classification

Automatic sound recognition (ASR) is a prominently emerging research area in recent years Recognition of sound events automatically through the computers in the complex audio environment is quite useful for machine hearing, acoustic surveillance and multimedia retrieval applications. Although a lot of features such as mel-frequency cepstral coefficients in ASR tasks provide very good results in noiseless environments, noisy conditions in the real world reduce success rates in a remarkable way. On the other hand, it was reported that spectrogram image features showed much better classification performance at low signal noise ratio values in many studies. In this article, it was proposed the preparation of feature vector after the images are reduced in size by applying the resizing process to spectrogram images with Lanczos kernel. Classification performance was compared by using deep artificial neural networks in different noise levels and although the feature vector was reduced, parallel values with results in the literature were obtained in the noiseless environment. It has remained slightly below the current state-of-the-art techniques using spectrogram features while better results compared to other commonly used features such as MFCC were obtained under the noisy conditions.

Human-assisted sound event recognition for home service robots.

This paper proposes and implements an open framework of active auditory learning for a home service robot to serve the elderly living alone at home. The framework was developed to realize the various auditory perception capabilities while enabling a remote human operator to involve in the sound event recognition process for elderly care. The home service robot is able to estimate the sound source position and collaborate with the human operator in sound event recognition while protecting the privacy of the elderly. Our experimental results validated the proposed framework and evaluated auditory perception capabilities and human–robot collaboration in sound event recognition.

An overview of applications and advancements in automatic sound recognition

Automatic sound recognition (ASR) has attracted increased and wide ranging interests in recent years. In this paper, we carry out a review of some important contributions in ASR techniques, mainly over the last one and a half decades. Similar to speech recognition systems, the robustness of an ASR system largely depends on the choice of feature(s) and classifier(s). We take a wider perspective in providing an overview of the features and classifiers used in ASR systems starting from early works in content-based audio classification to more recent developments in applications such as sound event recognition, audio surveillance, and environmental sound recognition. We also review techniques that have been utilized in noise robust sound recognition systems and feature optimization methods. Finally, some of the less commonly known applications of ASR are discussed.

Robust Sound Event Classification Using Deep Neural Networks

The automatic recognition of sound events by computers is an important aspect of emerging applications such as automated surveillance, machine hearing and auditory scene understanding. Recent advances in machine learning, as well as in computational models of the human auditory system, have contributed to advances in this increasingly popular research field. Robust sound event classification, the ability to recognise sounds under real-world noisy conditions, is an especially challenging task. Classification methods translated from the speech recognition domain, using features such as mel-frequency cepstral coefficients, have been shown to perform reasonably well for the sound event classification task, although spectrogram-based or auditory image analysis techniques reportedly achieve superior performance in noise. This paper outlines a sound event classification framework that compares auditory image front end features with spectrogram image-based front end features, using support vector machine and deep neural network classifiers. Performance is evaluated on a standard robust classification task in different levels of corrupting noise, and with several system enhancements, and shown to compare very well with current state-of-the-art classification techniques.

Acceleration of decision making in sound event recognition employing supercomputing cluster

Parallel processing of audio data streams is introduced to shorten the decision making time in hazardous sound event recognition. A supercomputing cluster environment with a framework dedicated to processing multimedia data streams in real time is used. The sound event recognition algorithms employed are based on detecting foreground events, calculating their features in short time frames, and classifying the events with Support Vector Machine. Different strategies for improving the decision time are introduced. The experiments with the presented strategies are conducted and the results are presented.

Overlapping sound event recognition using local spectrogram features and the generalised hough transform

In this paper, we address the challenging task of simultaneous recognition of overlapping sound events from single channel audio. Conventional frame-based methods are not well suited to the problem, as each time frame contains a mixture of information from multiple sources. Missing feature masks are able to improve the recognition in such cases, but are limited by the accuracy of the mask, which is a non-trivial problem. In this paper, we propose an approach based on Local Spectrogram Features (LSFs) which represent local spectral information that is extracted from the two-dimensional region surrounding “keypoints” detected in the spectrogram. The keypoints are designed to locate the sparse, discriminative peaks in the spectrogram, such that we can model sound events through a set of representative LSF clusters and their occurrences in the spectrogram. To recognise overlapping sound events, we use a Generalised Hough Transform (GHT) voting system, which sums the information over many independent keypoints to produce onset hypotheses, that can detect any arbitrary combination of sound events in the spectrogram. Each hypothesis is then scored against the class distribution models to recognise the existence of the sound in the spectrogram. Experiments on a set of five overlapping sound events, in the presence of non-stationary background noise, demonstrate the potential of our approach.

Selective Gammatone Envelope Feature for Robust Sound Event Recognition

Conventional features for Automatic Speech Recognition and Sound Event Recognition such as Mel-Frequency Cepstral Coefficients (MFCCs) have been shown to perform poorly in noisy conditions. We introduce an auditory feature based on the gammatone filterbank, the Selective Gammatone Envelope Feature (SGEF), for Robust Sound Event Recognition where channel selection and the filterbank envelope is used to reduce the effect of noise for specific noise environments. In the experiments with Hidden Markov Model (HMM) recognizers, we shall show that our feature outperforms MFCCs significantly in four different noisy environments at various signal-to-noise ratios.

Sound Event Recognition With Probabilistic Distance SVMs

Unlike other audio or speech signals, sound events have a relatively short time span. They are usually distinguished by their unique spectro-temporal signature. This paper proposes a novel classification method based on probabilistic distance support vector machines (SVMs). We study a parametric approach to characterizing sound signals using the distribution of the subband temporal envelope (STE), and kernel techniques for the subband probabilistic distance (SPD) under the framework of SVM. We show that generalized gamma modeling is well devised for sound characterization, and that the probabilistic distance kernel provides a closed form solution to the calculation of divergence distance, which tremendously reduces computational cost. We conducted experiments on a database of ten types of sound events. The results show that the proposed classification method significantly outperforms conventional SVM classifiers with Mel-frequency cepstral coefficients (MFCCs). The rapid computation of probabilistic distance also makes the proposed method an obvious choice for online sound event recognition.

Sound event recognition through expectancy-based evaluation ofsignal-driven hypotheses

A recognition system for environmental sounds is presented. Signal-driven classification is performed by applying machine-learning techniques on features extracted from a cochleogram. These possibly unreliable classifications are improved by creating expectancies of sound events based on context information.

Perceiving and encoding timbre.

Interactions with the auditory environment often require accurate recognition of sound objects and events (e.g., voices and instruments, and the utterances that emanate from them). Two lines of research in the lab have been exploring how listeners process the auditory dimension of timbre as one step toward understanding object recognition. In one, commonalties in processing different classes of timbres (human voices, musical instruments) using the selective adaptation paradigm are explored. In the second, the characteristics of timbre memory using the interpolated tone paradigm are examined [Deutsch, Science 168, 1604–1605]. Findings will be discussed in the context of similar results in the literature.