UrbanSound8K Dataset Research Articles

Hierarchical-Concatenate Fusion TDNN for sound event classification.

Semantic feature combination/parsing issue is one of the key problems in sound event classification for acoustic scene analysis, environmental sound monitoring, and urban soundscape analysis. The input audio signal in the acoustic scene classification is composed of multiple acoustic events, which usually leads to low recognition rate in complex environments. To address this issue, this paper proposes the Hierarchical-Concatenate Fusion(HCF)-TDNN model by adding HCF Module to ECAPA-TDNN model for sound event classification. In the HCF module, firstly, the audio signal is converted into two-dimensional time-frequency features for segmentation. Then, the segmented features are convolved one by one for improving the small receptive field in perceiving details. Finally, after the convolution is completed, the two adjacent parts are combined before proceeding with the next convolution for enlarging the receptive field in capturing large targets. Therefore, the improved model further enhances the scalability by emphasizing channel attention and efficient propagation and aggregation of feature information. The proposed model is trained and validated on the Urbansound8K dataset. The experimental results show that the proposed model can achieve the best classification accuracy of 95.83%, which is an approximate improvement of 5% (relatively) over the ECAPA-TDNN model.

HADNet: A Novel Lightweight Approach for Abnormal Sound Detection on Highway Based on 1D Convolutional Neural Network and Multi-Head Self-Attention Mechanism

Video surveillance is an effective tool for traffic management and safety, but it may face challenges in extreme weather, low visibility, areas outside the monitoring field of view, or during nighttime conditions. Therefore, abnormal sound detection is used in traffic management and safety as an auxiliary tool to complement video surveillance. In this paper, a novel lightweight method for abnormal sound detection based on 1D CNN and Multi-Head Self-Attention Mechanism on the embedded system is proposed, which is named HADNet. First, 1D CNN is employed for local feature extraction, which minimizes information loss from the audio signal during time-frequency conversion and reduces computational complexity. Second, the proposed block based on Multi-Head Self-Attention Mechanism not only effectively mitigates the issue of disappearing gradients, but also enhances detection accuracy. Finally, the joint loss function is employed to detect abnormal audio. This choice helps address issues related to unbalanced training data and class overlap, thereby improving model performance on imbalanced datasets. The proposed HADNet method was evaluated on the MIVIA Road Events and UrbanSound8K datasets. The results demonstrate that the proposed method for abnormal audio detection on embedded systems achieves high accuracy of 99.6% and an efficient detection time of 0.06 s. This approach proves to be robust and suitable for practical applications in traffic management and safety. By addressing the challenges posed by traditional video surveillance methods, HADNet offers a valuable and complementary solution for enhancing safety measures in diverse traffic conditions.

A barking emotion recognition method based on Mamba and Synchrosqueezing Short-Time Fourier Transform

Dog barks are a crucial means for dogs to express their emotions, conveying needs and emotional states such as anger, sadness, or excitement. With the increasing number of pet dogs, understanding their emotional states is essential for better care and interaction. However, accurately interpreting the emotional content of dog barks is often challenging for humans. Therefore, developing a technology for recognizing the emotions in dog barks is of great importance. In this paper, we establish a dataset for dog bark emotion recognition and propose a method using the Synchrosqueezing Short-Time Fourier Transform (SST_STFT) to extract the time–frequency features of dog barks. Considering the characteristics of dog barks, we design an emotion recognition model based on Mamba. This model leverages the state-space model (Select SMM) for global modeling of long sequence features, enabling rapid and effective processing of time–frequency features and accurate perception of their variations. Experiments on the self-built DogEmotionSound dataset, as well as the public IEMOCAP and UrbanSound8K datasets, demonstrate that our model outperforms existing state-of-the-art sound recognition models across various evaluation metrics. The recognition accuracies on the three datasets are 91.97%, 95.36%, and 67.25%, respectively. The code is open-source at https://github.com/yangcjya/A-barking-emotion-recognition-method-based-on-Mamba.git.

ESC-NAS: Environment Sound Classification Using Hardware-Aware Neural Architecture Search for the Edge

The combination of deep-learning and IoT plays a significant role in modern smart solutions, providing the capability of handling task-specific real-time offline operations with improved accuracy and minimised resource consumption. This study provides a novel hardware-aware neural architecture search approach called ESC-NAS, to design and develop deep convolutional neural network architectures specifically tailored for handling raw audio inputs in environmental sound classification applications under limited computational resources. The ESC-NAS process consists of a novel cell-based neural architecture search space built with 2D convolution, batch normalization, and max pooling layers, and capable of extracting features from raw audio. A black-box Bayesian optimization search strategy explores the search space and the resulting model architectures are evaluated through hardware simulation. The models obtained from the ESC-NAS process achieved the optimal trade-off between model performance and resource consumption compared to the existing literature. The ESC-NAS models achieved accuracies of 85.78%, 81.25%, 96.25%, and 81.0% for the FSC22, UrbanSound8K, ESC-10, and ESC-50 datasets, respectively, with optimal model sizes and parameter counts for edge deployment.

SIAlex: Species identification and monitoring based on bird sound features

The combination of deep learning and bird sound recognition is widely employed in bird species conservation monitoring. A complex network structure is not conducive for deploying bird sound recognition devices, resulting in problems such as long inference time and low efficiency. Using AlexNet as the backbone model, we explore the potential of shallow and straightforward models without complex connection techniques or attention mechanisms, named SIAlex, to recognise and classify 20 bird sound datasets, which are simultaneously validated on a 10 class UrbanSound8k dataset. Using the structural re-parameterization method, the number of model layers is reduced, computational efficiency is improved, and the inference time is significantly reduced, achieving a decoupling of training and inference time in the structure. To increase the nonlinearity of the model, a cascaded approach is utilised to increase the number of activation functions, thereby significantly improving the generalisation performance of the model. Simultaneously, in the classifier section, convolutional layer replaces the original fully connected layer, thereby reducing the inference time and increasing the feature extraction ability of the model, improving accuracy, and effectively recognising bird speech. The experimental data show that the SIAlex network on the Birdsdata dataset improves the accuracy to 93.66%, and the inference time for a piece of data is only 2.466 ms. The accuracy of the UrbanSound8k dataset reaches 96.04%, and the inference time for a piece of data is 3.031 ms. A large number of experimental comparisons have shown that the method proposed in this paper achieves good results in reducing the inference time of the model, bringing breakthroughs in the application of shallow, simple models.

NeuProNet: neural profiling networks for sound classification

Real-world sound signals exhibit various aspects of grouping and profiling behaviors, such as being recorded from identical sources, having similar environmental settings, or encountering related background noises. In this work, we propose novel neural profiling networks (NeuProNet) capable of learning and extracting high-level unique profile representations from sounds. An end-to-end framework is developed so that any backbone architectures can be plugged in and trained, achieving better performance in any downstream sound classification tasks. We introduce an in-batch profile grouping mechanism based on profile awareness and attention pooling to produce reliable and robust features with contrastive learning. Furthermore, extensive experiments are conducted on multiple benchmark datasets and tasks to show that neural computing models under the guidance of our framework gain significant performance gaps across all evaluation tasks. Particularly, the integration of NeuProNet surpasses recent state-of-the-art (SoTA) approaches on UrbanSound8K and VocalSound datasets with statistically significant improvements in benchmarking metrics, up to 5.92% in accuracy compared to the previous SoTA method and up to 20.19% compared to baselines. Our work provides a strong foundation for utilizing neural profiling for machine learning tasks.

A novel Approach for Audio-based Video Analysis via MFCC Features

The video analysis is a momentous task of computer vision where larger videos are analyzed and converted to comparatively shorter summaries with key-frames satisfying a typical criterion. However, in the recent times audio-based video analysis is an important field where, audio features are used to categorize video frames into various classes. In this work, we present an audio-based video analysis model named as ABVS. The proposed approach employs 40 MFCC audio features from input audios to learn a multiclass classifier. The ABVS model is trained on ANN, SVM, KNN, DT and a majority voting ensemble to categorize a given video based on audio at high precision and low error rate. Our proposed approach is trained and validated on a benchmark urbansound8k dataset with an accuracy of 90% and loss of 0.10 on ANN and SVM model whereas, KNN achieves an accuracy of 89%. The ABVS model can be deployed in real-time applications to extract key-frames from the video scenes to troubleshoot the malfunctioning of critical infrastructure in industrial sectors. Our proposed model may be deployed in audio forensics for criminal voice investigation.

Urban Sound Classification

There are many sounds all around us and our brain can easily and clearly identify them. Furthermore, our brain processes the received sound signals continuously and provides us with relevant environmental knowledge. Although not up to the level of accuracy of the brain, there are some smart devices which can extract necessary information from an audio signal with the help of different algorithms. Over the years, various models like Convolutional Neural Networks (CNNs), Artificial Neural Networks (ANNs), Region- Convolutional Neural Networks (R-CNNs), and numerous machine learning techniques have been employed for sound classification. These methods have shown impressive results in distinguishing spectra-temporal patterns and different sound categories. The novelty of our research lies in showing that the long-short term memory (LSTM) shows a better result in classification accuracy compared to CNN for many features used. Additionally, we've evaluated model accuracy using different techniques such as data augmentation and feature stacking. With our RNN model, we achieved a remarkable accuracy of 87%, setting a new benchmark in performance on the UrbanSound8k dataset. Our findings not only advance the field of sound classification but also underscore the potential of LSTM networks and the importance of innovative techniques such as data augmentation and feature stacking in improving the accuracy of sound recognition systems. Key Words: Sound Classification, Urbansound8k, Librosa, Spectrograms, deep learning, CNN, LSTM.

A CNN Sound Classification Mechanism Using Data Augmentation.

Sound classification has been widely used in many fields. Unlike traditional signal-processing methods, using deep learning technology for sound classification is one of the most feasible and effective methods. However, limited by the quality of the training dataset, such as cost and resource constraints, data imbalance, and data annotation issues, the classification performance is affected. Therefore, we propose a sound classification mechanism based on convolutional neural networks and use the sound feature extraction method of Mel-Frequency Cepstral Coefficients (MFCCs) to convert sound signals into spectrograms. Spectrograms are suitable as input for CNN models. To provide the function of data augmentation, we can increase the number of spectrograms by setting the number of triangular bandpass filters. The experimental results show that there are 50 semantic categories in the ESC-50 dataset, the types are complex, and the amount of data is insufficient, resulting in a classification accuracy of only 63%. When using the proposed data augmentation method (K = 5), the accuracy is effectively increased to 97%. Furthermore, in the UrbanSound8K dataset, the amount of data is sufficient, so the classification accuracy can reach 90%, and the classification accuracy can be slightly increased to 92% via data augmentation. However, when only 50% of the training dataset is used, along with data augmentation, the establishment of the training model can be accelerated, and the classification accuracy can reach 91%.

Sound-Based Real-Time Scrap Falling State Monitoring in the Shearing Process of Wide Heavy Plate Mill

The scrap falling state in the shearing process of a Wide Heavy Plate Mill depends on manual monitoring and judgment. The steel plate can only move forward after the scrap falls, otherwise, the equipment will be damaged. An experienced operator can use the sound they hear to estimate the falling state. To automatically recognize sound events, a scheme of sound data acquisition processing and recognition for monitoring the scrap state of the shearing process is proposed in this paper. Firstly, a pickup is installed at the monitoring point of the crop shear to collect the sound. Afterward, the sound data is processed and finally fed into the model for recognition. Since many sound segments in the field are not related to key actions, an endpoint detection technique based on a short-term energy threshold and the variable window length strategy is applied in the data processing section to obtain the sound segments to be identified. The Log-Mel spectrogram feature of the sound segment is then extracted. After a data filling to ensure a uniform data shape, the feature map is fed into a lightweight deep convolution neural network called MobileNetV2 to quickly identify sound events. For the shearing process events, the model recognition accuracy can reach 95% of the F1 score in a realistic noise environment and we can identify the scrap falling 0.3s after the action starts. Finally, the model was applied to the UrbanSound8K dataset and achieved 95% recognition accuracy, validating the performance of the method.

An Automatic Classification System for Environmental Sound in Smart Cities.

With the continuous promotion of "smart cities" worldwide, the approach to be used in combining smart cities with modern advanced technologies (Internet of Things, cloud computing, artificial intelligence) has become a hot topic. However, due to the non-stationary nature of environmental sound and the interference of urban noise, it is challenging to fully extract features from the model with a single input and achieve ideal classification results, even with deep learning methods. To improve the recognition accuracy of ESC (environmental sound classification), we propose a dual-branch residual network (dual-resnet) based on feature fusion. Furthermore, in terms of data pre-processing, a loop-padding method is proposed to patch shorter data, enabling it to obtain more useful information. At the same time, in order to prevent the occurrence of overfitting, we use the time-frequency data enhancement method to expand the dataset. After uniform pre-processing of all the original audio, the dual-branch residual network automatically extracts the frequency domain features of the log-Mel spectrogram and log-spectrogram. Then, the two different audio features are fused to make the representation of the audio features more comprehensive. The experimental results show that compared with other models, the classification accuracy of the UrbanSound8k dataset has been improved to different degrees.

Investigation of the effectiveness of time-frequency domain images and acoustic features in urban sound classification

Rapid urbanization and population growth worldwide seriously challenge building livable and sustainable cities. This increase causes the increase and diversification of urban sounds. They were transforming these sounds into information instead of just being heard, as noise plays an important role in the concept of smart cities. For this purpose, two basic methods are used to classify urban sounds. In the first of these, the sounds are processed by signal processing methods, and handcrafted features are obtained. In the other method, sounds are represented visually and classified with deep learning models. This study investigated the effect of the individual and hybrid use of features used in both approaches on the classification of urban sounds. In addition, a CNN model was created to classify hybrid features. The results obtained showed that both approaches produced successful results in classification. Among the visual representation methods, mel-spectrogram, scalogram, and spectrogram images achieved the highest classification success. Using mel-spectrogram and acoustic features and the SVM classifier positively affected accuracy. Experiments were performed on the ESC-10 and UrbanSound8k datasets. The highest accuracy for the ESC-10 was 98.33% when using the scalogram and acoustic features with the AVCNN model. The highest accuracy for UrbanSound8k was obtained as 97.70% by classifying the mel-spectrogram and acoustic features obtained from the AVCNN model with the SVM classifier.

Modulating STDP With Back-Propagated Error Signals to Train SNNs for Audio Classification

Audio classification has many practical applications such as noise pollution detection, wildlife monitoring, speech recognition, and more. For many of these applications, deploying classifiers on low powered devices for persistent deployment is desirable. Artificial neural networks (ANN) have achieved state-of-the-art performance on audio classification tasks; however, it is not always feasible to deploy modern ANNs to embedded devices due to their high power consumption. Biologically inspired spiking neural networks (SNN) have been shown to significantly reduce power consumption during inference when compared with equivalent ANNs, and they have also been theoretically proven to be more computationally powerful than stateless ANNs. This work proposes an audio classification system using SNNs, and a learning algorithm is developed for classification with multilayer SNNs which combines biologically plausible spike-timing-dependent plasticity (STDP) with spatial error backpropagation. By allowing the STDP process to account for both temporal dependencies and the non-differentiable activation function derivative, the proposed learning rule successfully trains multilayer SNNs for the considered classification tasks. Through the STDP process, the proposed learning rule is also capable of online learning; explicit storage of values from previous timesteps is not required, in contrast to the widely adopted backpropagation through time (BPTT) algorithm. The proposed approach approaches the performance of SNNs trained via BPTT on the classification tasks. SNNs trained with the proposed learning rule are evaluated on the Iris Flower dataset, the Real-World Computing Partnership sounds dataset, the Free Spoken Digits Dataset, and the UrbanSound8k dataset.

IoT smart city framework using AI for urban sound classification

The advances of artificial intelligence approaches for automatically extracting and classifying disturbing sounds have great potential and application in the development of smart cities. In this paper, an urban sound event classification system based on deep learning technologies has been created by using the MEL-frequency cepstral coefficients as feature extractors and the Convolutional Neural Networks as classifiers. The designed system was trained and tested using the UrbanSound8K dataset which resulted in high classification accuracy of 92.67% of the tested results. In addition to this, validation of unknown sound events was applied. Furthermore, we propose the creation of a stand-alone system capable of recording and classifying sounds within an urban environment and sending the classification data to the cloud. The implementation of such units could result in real-time sound classification which can be used in smart cities based on the Internet of Things technology. According to this, an IoT smart city framework using AI for urban sound classification was proposed. The created system could find its use in many applications by making contribution in creating a feasible and deployable real-time sound classification system.

Environmental Sound Classification using Hybrid Ensemble Model

Environmental sound classification (ESC) is the most trending research areas. The sounds in the surroundings such as screaming, air conditioners, and rain droplets can help in the development of context-aware applications. It is complex to process the envi- ronmental sounds as compared to speech and music due to the unstructured essence of environmental sounds. In the past, certain preprocessing techniques, feature extraction, and classification algorithms are used for ESC. Several researchers have applied ma- chine learning classifiers for ESC and certain ensemble classifiers are also used but the accuracy can be increased if instead of combining homogeneous classifiers, heterogeneous classifiers can be ensembled. In this paper, a hybrid ensemble classifier is used for ESC on the UrbanSound8k dataset and cepstral features Mel Frequency Cepstral Coefficients are used. Five different machine learning classifiers- Decision Tree, Support Vector Machine, Logistic Regression, K- Nearest Neighbour, and Naive Bayes are used to develop a hybrid ensemble model. The highest accuracy is obtained when all the five classifiers are combined. The proposed approach gives an accuracy of 79.4% and is compared with the benchmark results using individual classifiers and the former out- performs the latter. The results of the hybrid ensemble model on the UrbanSound8K dataset are also compared with the dataset ESC-10.

Comparative Analysis of Different Sampling Rates on Environmental Sound Classification Using the Urbansound8k Dataset

Environmental sound classification (ESC) has gained increasing attention in recent years. This study focuses on the evaluation of the popular public dataset Urbansound8k (Us8k) at different sampling rates using hand crafted features. The Us8k dataset contains environment sounds recorded at various sampling rates, and previous ESC works have uniformly resampled the dataset. Some previous work converted this data to different sampling rates for various reasons. Some of them chose to convert the rest of the dataset to 44,100, as the majority of the Us8k files were already at that sampling rate. On the other hand, some researchers down sampled the dataset to 8000, as it reduced computational complexity, while others resampled it to 16,000, aiming to achieve a balance between higher classification accuracy and lower computational complexity. In this research, we assessed the performance of ESC tasks using sampling rates of 8000 Hz, 16,000 Hz, and 44,100 Hz by extracting the hand crafted features Mel frequency cepstral coefficient (MFCC), gamma tone cepstral coefficients (GTCC), and Mel Spectrogram (MelSpec). The results indicated that there was no significant difference in the classification accuracy among the three tested sampling rates.

Profiling of Urban Noise Using Artificial Intelligence

Noise pollution tends to receive less awareness compared to other types of pollution, however, it greatly impacts the quality of life for humans such as causing sleep disruption, stress or hearing impairment. Profiling urban sound through the identification of noise sources in cities could help to benefit livability by reducing exposure to noise pollution through methods such as noise control, planning of the soundscape environment, or selection of safe living space. In this paper, we proposed a self-attention long short-term memory (LSTM) method that can improve sound classification compared to previous baselines. An attention mechanism will be designed solely to capture the key section of an audio data series. This is practical as we only need to process important parts of the data and can ignore the rest, making it applicable when gathering information with long-term dependencies. The dataset used is the Urbansound8k dataset which specifically pertains to urban environments and data augmentation was applied to overcome imbalanced data and dataset scarcity. All audio sources in the dataset were normalized to mono signals. From the dataset above, an experiment was conducted to confirm the suitability of the proposed model when applied to the mel-spectrogram and MFCC (Mel-Frequency Cepstral Coefficients) datasets transformed from the original dataset. Improving the classification accuracy depends on the machine learning models as well as the input data, therefore we have evaluated different class models and extraction methods to find the best performing. By combining data augmentation techniques and various extraction methods, our classification model has achieved state-of-the-art performance, each class accuracy is up to 98%.

An Efficient Incremental Learning Algorithm for Sound Classification

This article proposes an efficient audio incremental learning method to reduce the computational complexity and catastrophic forgetting during the incremental addition of the audio data in Deep Neural Networks (DNN). The computational complexity is reduced by performing training of only fully-connected layers and catastrophic forgetting is reduced by sharing the knowledge from the old learned classes without using previously learned data. Our method has been evaluated extensively on UrbanSound8K, ESC-10, and TUT datasets where state-of-the-art accuracies have been achieved. Moreover, our method has been evaluated on Nvidia 1080-ti GPU, Nvidia TX-2, and Nvidia Xavier development boards to demonstrate the training time and energy consumption savings as compared to recent state-of-the-art methods.

Sound Event Classification Based on Frequency-Energy Feature Representation and Two-Stage Data Dimension Reduction

The classification of environmental sound events is of great significance for applications such as machine hearing and acoustic surveillance. Feature representation and feature vector dimension directly affect system performance. To better extract features and reduce computational burden, a novel frequency-energy feature representation and two-stage dimension reduction system were proposed. First, a frequency-energy diagram is generated. Based on this, the importance screening is done and only the energy bins of high importance are retained, which reduces the dimension of feature vector while extracting key information. Then the Bicubic interpolation method is used to further reduce the dimension. And the appropriate feature vector dimension is determined based on the change of information entropy. The proposed frequency-energy feature representation and two-stage dimension reduction system are evaluated with Real Word Computing Partnership sound scene database (RWCP-SSD), UrbanSound8K, and ESC-50 datasets, which demonstrate that the robustness is satisfactory under low signal-to-noise ratios (SNRs) and 15 noise types from NOISEX-92 database.

CNN hyper-parameter optimization for environmental sound classification

Environmental sounds are being used widely in our lives. It is especially used in tasks such as managing smart cities, location determination, surveillance systems, machine hearing, and environmental monitoring. The main method for this, environmental sound classification (ESC), has been increasingly studied in recent years. However, the classification of these sounds is more difficult than other sounds because there are too many parameters that generate noise. The study tried to find the convolutional neural network (CNN) model that gave the highest accuracy for ESC tasks with the optimization of hyper-parameters. For this purpose, the Particle Swarm Optimization (PSO) algorithm was rearranged to represent the CNN architecture. Thus, the hyper-parameters in CNN are represented exactly without any transformation during optimization. Studies were carried out on the ESC-10, ESC-50, and Urbansound8k data sets, which are state-of-art for ESC tasks. Some data augmentation techniques have been used for data sets in the training of CNN models. The CNN models, which were obtained with PSO, achieved success rates of 98.64 % for ESC-10, 93.71 % for ESC-50, and 98.45 % for Urbansound8k, respectively. These results are the best accuracy values obtained with the pure CNN model when compared with previous studies. As a result, it has been made possible to automatically design CNN models for the classification of urban sounds, giving high classification accuracy. Thus, researchers who do not know much about CNN design can use this method in their desired datasets without the need for expert knowledge.