Sound Detection System Research Articles

Drone sound detection system based on feature result-level fusion using deep learning

Drone sound detection system based on feature result-level fusion using deep learning

ANALYSIS OF THE SOUND EVENT DETECTION METHODS AND SYSTEMS

Detection and recognition of loud sounds and characteristic noises can significantly increase the level of safety and ensure timely response to various emergency situations. Audio event detection is the first step in recognizing audio signals in a continuous audio input stream. This article presents a number of problems that are associated with the development of sound event detection systems, such as the deviation for each environment and each sound category, overlapping audio events, unreliable training data, etc. Both methods for detecting monophonic impulsive audio event and polyphonic sound event detection methods which are used in the state-of-the-art sound event detection systems are presented. Such systems are presented in Detection and Classification of Acoustic Scenes and Events (DCASE) challenges and workshops, which take place every year. Beside a majority of works focusing on the improving overall performance in terms of accuracy many other aspects have also been studied. Several systems presented at DCASE 2021 task 4 were considered, and based on their analysis, there was a conclusion about possible future for sound event detection systems. Also the actual directions in the development of modern audio analytics systems are presented, including the study and use of various architectures of neural networks, the use of several data augmentation techniques, such as universal sound separation, etc.

Exploring Regression-based Approach for Sound Event Detection in Noisy Environments

Sound-event detection enables machines to detect when a particular sound event has occurred in addition to classifying the type of event. Successful detection of various sound events is paramount in building secure surveillance systems and other smart home appliances. However, noisy events and environ-ments exacerbate the performance of many sound event detection models, rendering them ineffective in real-world scenarios. Hence, the need for robust sound event detection algorithms in noisy environments with low inference times arises. You Only Hear Once (YOHO) is a purely convolutional architecture that uses a regression-based approach for sound-event-detection instead of the more common, frame-wise classification-based approach. The YOHO architecture proved robust in noisy environments, outperforming convolutional recurrent neural networks popular in sound event detection systems. Additionally, different ways to enhance the performance of the YOHO architecture are explored, experimenting with different computer vision architectures, dy-namic convolutional layers, pretrained audio neural networks and data augmentation methods to help improve the performance of the models on noisy data. Amongst several modifications to the YOHO architecture, the Frequency Dynamic Convolution Layers helped improve the internal model data representations by enforcing frequency-dependent convolution operations, which helped improve YOHO performance on noisy audios in outdoor and vehicular environments. Similarly, the FilterAugment data augmentation method and Convolutional Block Attention Module helped improve YOHO’s performance on the VOICe dataset containing noisy audios by augmenting the data and improving internal model representations of the input audio data using attention, respectively.

Sound Event Detection for Human Safety and Security in Noisy Environments

The objective of a sound event detector is to recognize anomalies in an audio clip and return their onset and offset. However, detecting sound events in noisy environments is a challenging task. This is due to the fact that in a real audio signal several sound sources co-exist and that the characteristics of polyphonic audio are different from isolated recordings. It is also necessary to consider the presence of noise (e.g. thermal and environmental). In this contribution, we present a sound anomaly detection system based on a fully convolutional network which exploits image spatial filtering and an Atrous Spatial Pyramid Pooling module. To cope with the lack of datasets specifically designed for sound event detection, a dataset for the specific application of noisy bus environments has been designed. The dataset has been obtained by mixing background audio files, recorded in a real environment, with anomalous events extracted from monophonic collections of labelled audios. The performances of the proposed system have been evaluated through segment-based metrics such as error rate, recall, and F1-Score. Moreover, robustness and precision have been evaluated through four different tests. The analysis of the results shows that the proposed sound event detector outperforms both state-of-the-art methods and general purpose deep learning-solutions.

An Analysis of Sound Event Detection under Acoustic Degradation Using Multi-Resolution Systems

The Sound Event Detection task aims to determine the temporal locations of acoustic events in audio clips. In recent years, the relevance of this field is rising due to the introduction of datasets such as Google AudioSet or DESED (Domestic Environment Sound Event Detection) and competitive evaluations like the DCASE Challenge (Detection and Classification of Acoustic Scenes and Events). In this paper, we analyze the performance of Sound Event Detection systems under diverse artificial acoustic conditions such as high- or low-pass filtering and clipping or dynamic range compression, as well as under an scenario of high overlap between events. For this purpose, the audio was obtained from the Evaluation subset of the DESED dataset, whereas the systems were trained in the context of the DCASE Challenge 2020 Task 4. Our systems are based upon the challenge baseline, which consists of a Convolutional-Recurrent Neural Network trained using the Mean Teacher method, and they employ a multiresolution approach which is able to improve the Sound Event Detection performance through the use of several resolutions during the extraction of Mel-spectrogram features. We provide insights on the benefits of this multiresolution approach in different acoustic settings, and compare the performance of the single-resolution systems in the aforementioned scenarios when using different resolutions. Furthermore, we complement the analysis of the performance in the high-overlap scenario by assessing the degree of overlap of each event category in sound event detection datasets.

Smart IoT-based system for detecting RPW larvae in date palms using mixed depthwise convolutional networks

Smart agriculture and Internet of Things (IoT) technologies have become the key points for many intelligent decision-making applications to support agricultural experts and farmers, especially for crop pest management and control. In this work, we present an IoT-based sound detection model for identifying red palm weevil (RPW) larvae to protect date palm trees at the early stage of infestation. The proposed detection system is mainly based on a modified mixed depthwise convolutional network (MixConvNet) as a recent deep learning classifier. The public TreeVibes dataset, which contains short audio recordings of feeding and/or moving RPWs, was successfully tested and assessed with the proposed MixConvNet classifier. There were 146 and 351 specimens of infested and clean sounds examined, respectively. The classification results showed that our proposed MixConvNet is efficient and superior to other deep learning classifiers, such as Xception and residual network (Resnet) models in previous related studies, obtaining the best accuracy score of 97.38. Moreover, the MixConvNet classifier is capable of identifying RPW infestation cases precisely with a high accuracy value of 95.90% ± 1.46, using 10-fold cross-validation. Therefore, practical implementation of our proposed IoT-enabled early sound detection system of RPWs is considered the future milestone of this study.

Abnormal drone noise detection system based on the microphone array and self-supervised learning

The drone noise mainly comes from its rotating blades, providing plentiful information of the status of the drone. In the production line, the abnormal sound detection system has the advantages of no contact and simple deployment and can help to locate the fault products at relatively low costs. Therefore, this paper develops an abnormal drone noise detection system based on the microphone array and self-supervised learning. The microphone array is a part of the data acquisition module to pick up the drone noise. There are eight microphones in the array, forming four differential microphone pairs. Each of them is pointing to a blade of the drone. A four-channel noise sample is recorded and then analyzed. It is worth noting that drone noise samples are extremely unbalanced, because abnormal samples are difficult to encounter. Hence, a self-supervised learning strategy is adopted by creating auxiliary classification tasks to fine tune representations of the normal drone noise samples. With the consideration of low-complexity, the trained neural network models can be finally deployed even on a Raspberry Pi system with no graphic cards.

Click-event sound detection in automotive industry using machine/deep learning

In the automotive industry, despite the robotic systems on the production lines, factories continue employing workers in several custom tasks getting for semi-automatic assembly operations. Specifically, the assembly of electrical harnesses of engines comprises a set of connections between electrical components. Despite the task is easy to perform, employees tend not to notice that a few components are not being connected properly due to physical fatigue provoked by repetitive tasks. This yields a low quality of the assembly production line and possible hazards. In this work, we propose a sound detection system based on machine/deep learning (ML/DL) approaches to identify click sounds produced when electrical harnesses are connected. The purpose of this system is to count the number of connections properly made and to feedback to the employees. We collect and release a public dataset of 25,000 click sounds of 25 ms length at 22 kHz during three months of assembly operations in an automotive production line located in Mexico. Then, we design an ML/DL-based methodology for click sound detection of assembled harnesses under real conditions of a noisy environment (noise level ranging from −16.67 dB to −12.87 dB) including other machinery sounds. Our best ML/DL model (i.e., a combination between five acoustic features and an optimized convolutional neural network) is able to detect click sounds in a real assembly production line with an accuracy of 94.55±0.83 %. To the best of our knowledge, this is the first time a click sounds detection system in assembling electrical harnesses of engines for giving feedback to the workers is proposed and implemented in a real-world automotive production line. We consider this work valuable for the automotive industry on how to apply ML/DL approaches for improving the quality of semi-automatic assembly operations.

Performance analysis of weakly-supervised sound event detection system based on the mean-teacher convolutional recurrent neural network model

Performance analysis of weakly-supervised sound event detection system based on the mean-teacher convolutional recurrent neural network model

A Multi-Resolution CRNN-Based Approach for Semi-Supervised Sound Event Detection in DCASE 2020 Challenge

Sound Event Detection is a task with a rising relevance over the recent years in the field of audio signal processing, due to the creation of specific datasets such as Google AudioSet or DESED (Domestic Environment Sound Event Detection) and the introduction of competitive evaluations like the DCASE Challenge (Detection and Classification of Acoustic Scenes and Events). The different categories of acoustic events can present diverse temporal and spectral characteristics. However, most approaches use a fixed time-frequency resolution to represent the audio segments. This work proposes a multi-resolution analysis for feature extraction in Sound Event Detection, hypothesizing that different resolutions can be more adequate for the detection of different sound event categories, and that combining the information provided by multiple resolutions could improve the performance of Sound Event Detection systems. Experiments are carried out over the DESED dataset in the context of the DCASE 2020 Challenge, concluding that the combination of up to 5 resolutions allows a neural network-based system to obtain better results than single-resolution models in terms of event-based F1-score in every event category and in terms of PSDS (Polyphonic Sound Detection Score). Furthermore, we analyze the impact of score thresholding in the computation of F1-score results, finding that the standard value of 0.5 is suboptimal and proposing an alternative strategy based in the use of a specific threshold for each event category, which obtains further improvements in performance.

Convolutional Neural Networks for Scops Owl Sound Classification

Adopting a deep learning model into bird sound classification tasks becomes a common practice in order to construct a robust automated bird sound detection system. In this paper, we employ a four-layer Convolutional Neural Network (CNN) formulated to classify different species of Indonesia scops owls based on their vocal sounds. Two widely used representations of an acoustic signal: log-scaled mel-spectrogram and Mel Frequency Cepstral Coefficient (MFCC) are extracted from each sound file and fed into the network separately to compare the model performance with different inputs. A more complex CNN that can simultaneously process the two extracted acoustic representations is proposed to provide a direct comparison with the baseline model. The dual-input network is the well-performing model in our experiment that achieves 97.55% Mean Average Precision (MAP). Meanwhile, the baseline model achieves a MAP score of 94.36% for the mel-spectrogram input and 96.08% for the MFCC input.

Multiclass sound event detection for respiratory disease diagnosis

Recordings from daily life contain a mix of sound types such as speech, coughing sounds, breathing sounds, and environmental noise, which provide a wealth of information related to linguistic messages, acoustic scenes, and health. These sounds in their unprocessed form overlap in both time and frequency domains, posing challenges to current methods for information extraction and analysis. To address this challenge, we introduce a novel multiclass sound event detection system that discriminates between speech, coughs, breathing, and other miscellaneous sounds (e.g., dogs barking, toilets flushing, babies crying) in the Coswara database, a crowdsourced database using a website application launched in response to COVID-19. This method extracts a feature set that includes Zero Crossing Rates, Short-Term Energy, and Mel-Frequency Cepstral Coefficients and uses Random Forests for multiclass classification. Preliminary results of the system yield a balanced accuracy of 87.5% detecting between speech, coughs, breathing, and other miscellaneous sounds. Using the multiclass sound event detection system, a time and acoustic-mediated forced-alignment technique is employed to discriminate complex sounds in real-time. We envision that the system could be used on devices with existing information extraction methods formonitoring respiratory diseases, such as pneumonia, pertussis, and COVID-19.

Multi-Scale and Single-Scale Fully Convolutional Networks for Sound Event Detection

Among various Sound Event Detection (SED) systems, Recurrent Neural Networks (RNN), such as long short-term memory unit and gated recurrent unit, is used to capture temporal dependencies, but it is confined in its length of temporal dependencies, resulting in a failure to model sound events with long duration. What’s more, RNN is incapable to process datasets in parallel, leading to low efficiency and low industrial value. Given these shortcomings, we propose to use dilated convolution (and causal dilated convolution) to capture temporal dependencies, as its great ability to ensure high time resolution and obtain longer temporal dependencies under the filter size and the network depth unchanged. In addition, dilated convolution can be parallelized, so it has higher efficiency and industrial value. Based on this, we propose Single-Scale Fully Convolutional Networks (SS-FCN) composed of convolutional neural networks and dilated convolutional networks, with the former to provide frequency invariance and the later to capture temporal dependencies. With the help of dilated convolution to control the length of temporal dependencies, we observe SS-FCN modeling a single length of temporal dependencies achieves superior detection performance for finite kinds of events. For better performance, we propose Multi-Scale Fully Convolutional Networks (MS-FCN), in which the feature fusion module is introduced to capture long short-term dependencies by fusing features with different length of temporal dependencies. The proposed methods achieve competitive performance on three main datasets with higher efficiency. The results show that SED systems based on Fully Convolutional Networks have further research value and potential.

Mel-log energies analysis of authentic audible intrusion activities in a Malaysian forest

Wildlife has been endangered due to illegal activities. This requires more effective surveillance measures. Felling timber and poaching are regular illegal activities but challenging to detect. Hence authorities should resort to modern technologies such as employing autonoumous surveillance to stop them. The Malaysian forest audio data were recorded to lay a foundation in initiating a cheaper and practical approach. Hence this paper reports the collection, processing and analysis of audio data in preparation to develop an autonomous sound event detection system. The recording was an emulation of possible illegal activities in a reserved forest. Sounds of chainsaw and hand hatchet cutting tree trunks were taken. It was found that there was a distinct pattern in the Mel-log energies audio feature of the sound, which could be used to identify illegal activities. Thus, it is believed that a detection through audio is a possible approach to be employed as one of the methods to stop illegal activities in the tropical reserve forests like those in Malaysia.

Low Cost Portable Solution for Real-Time Complete Detection and Analysis of Heart Sound Components

Detection of heart sound and analysis is required to determine heart rate, heart sound components (S1, S2 and murmurs), and is an important step to diagnose cardiac disorder. The most important part of detection method is to achieve a noise free signal staring from detection to separation of heart components to improve the accuracy of signal analysis. Though several innovative individual methods are found in the literature related to detection, noise filter or analyse the characteristic patterns of the heart signals, but some of them have computationally complex algorithms, some are equipped with expensive instruments and to the best of our knowledge, it is rare to found a low cost complete solution starting from detection to analysis with simple instruction based methodology to be handled by the literate ordinary people. There is also difficulty to implement them on real-time portable systems and also require medical expertise to handle the equipment and perform the operations. In this work, we propose an innovative simple solution for complete heart sound detection and analysis system with low cost portable heart sound detection devices, best effort wavelet de-noising and computationally less complex start and end point detection algorithms to separate and analyse heart sound components. MATLAB and DSP processor (TMS320C6713) has been used for the real time implementation of the algorithms. This process will be very helpful in a place with low medical establishment to take mass scale measurement and analysis to be finally sent to doctor for advise and treatment. We also provide frequency domain power spectrum density of the detected heart sound components for better diagnosis. At the end, we provide a comparative table with the available literature.

Automatic Recognition of Porcine Abnormalities Based on a Sound Detection and Recognition System

Abstract. With the rapid development of large-scale breeding, manual long-term monitoring of the daily activities and health of livestock is costly and time-consuming. Therefore, the application of bio-acoustics to automatic monitoring has received increasing attention. Although bio-acoustical techniques have been applied to the recognition of animal sounds in many studies, there is a dearth of studies on the automatic recognition of abnormal sounds from farm animals. In this study, an automatic detection and recognition system based on bio-acoustics is proposed to hierarchically recognize abnormal animal states in a large-scale pig breeding environment. In this system, we extracted the mel-frequency cepstral coefficients (MFCC) and subband spectrum centroid (SSC) as composite feature parameters. At the first level, support vector data description (SVDD) is used to detect abnormal sounds in the acoustic data. At the second level, a back-propagation neural network (BPNN) is used to classify five kinds of abnormal sounds in pigs. Furthermore, improved spectral subtraction is developed to reduce the noise interference as much as possible. Experimental results show that the average detection accuracy and the average recognition accuracy of the proposed system are 94.2% and 95.4%, respectively. The effectiveness of the proposed sound detection and recognition system was also verified through tests at a pig farm. Keywords: Abnormal sounds, MFCC, SSC, States of pigs, SVDD.

Microanalysis Using Acoustically Actuated Droplets Pinned Onto a Thread

An acoustic wave-assisted microscale assay platform is demonstrated. Liquid droplets containing samples and reagents are dispensed and pinned onto a hydrophobic thread, and actuated by sound waves. The transport of droplets is achieved by modulating sound emitted by two sources. The resulting sound intensity gradient pushes the droplets in one direction. The available distance of droplet actuation is related to droplet size. The moving droplets merge at the end of their actuation range, and the merged droplet travels further due to its greater size. Fluorometric analysis of the merged droplet is accomplished by positioning an excitation light source above a defined interrogation zone along the thread. The emitted light is analyzed by a miniature spectrometer positioned orthogonally to the excitation light beam. The sound sources and detection system are controlled by universal electronic modules (Raspberry Pi, Arduino). The system is operated automatically with the aid of a custom-developed graphical user interface. Here, we demonstrate its application in enzyme assays (trypsin, β-galactosidase). In-droplet enzymatic reaction progress curves are readily obtained. The actuated droplets can be put in motion and stopped without a significant delay. Only microliter-range volumes of sample and reagent are consumed in every analysis, while sub-milliliter volumes of chemical waste are produced.

Audio Signal and Troubleshooting System Based on Wireless Sensor

To explore the application of audio signal in the troubleshooting system, a sound detection system was designed. The system consists of three parts: voice acquisition node, aggregation node and host computer monitoring software. Time protocol synchronization (TPSN) algorithm was used to realize the synchronization between nodes. The algorithm and the trilateration method were applied to the system. The applications of the sound detection system in the field of fault sound source localization were realized.Wireless sensor networks were used in sound detection systems, which had many advantages. On the one hand, complicated wiring was avoided. It had the advantages of easy to set up and easy to move.On the other hand, some self-organizing and adaptive features in wireless sensor networks and some methods of synchronization and localization could be introduced. Thewhole system was more flexible and its application was more extensive.Using monitoring software, users can remotely access to the scene of the voice signal.The results showed that the system had high transmission rate, stable operation and small positioning error. Therefore, it has good application prospects.

Auditory Suspicious Event Databases: DASE and Bi-DASE

The lack of open-access databases for auditory suspicious events’ detection and classification algorithms is a hurdle for researchers while evaluating and comparing the performance of suspicious sound event detection and recognition systems. This paper introduces two databases called DASE and bi-DASE. DASE contains nine of the most common acoustic suspicious events with a total of 3105 sound event recordings of glass breaking, dog barking, scream, gunshot, explosion, alarm, sirens, door slams, and footsteps. Bi-DASE database is prepared by mixing two auditory suspicious events from DASE to generate 25 possible real-life scenarios. Both databases are divided into two sets as a training set and testing set. ANOVA and regression analysis are performed in regard to commonly used features, energy, 12 MFCCs, pitch range, spectral centroid frequency, spectral spread, spectral flatness, tonality, harmonics, zero-crossing rate, and fundamental frequency. All features are found to be significantly important based on their $p$ -values for both databases, which are evaluated by designing two baseline systems using support vector machines and $k$ -nearest neighbors algorithm. Experimental results are presented. The databases are available for free to the research community upon request.

Making higher performance bowel sound spectrum templates for real-time bowel sound detection system

Making higher performance bowel sound spectrum templates for real-time bowel sound detection system