Sound Recognition System Research Articles

Vision assistant for visually impaired individuals

Visually impaired individuals face significant challenges in accessing environmental information, social interactions, and safety in their daily lives. This article aims to increase the independence of visually impaired individuals and simplify their daily lives using an AI-supported image and sound recognition system. The developed prototype is a pair of glasses consisting of a Raspberry Pi 4 microprocessor, integrated headphones, touch controls, and a powerful battery management system. The electronic diagram includes the integration of a camera and microprocessor. The AI algorithms developed successfully perform tasks such as object recognition, text reading, and facial recognition. The article provides a comprehensive literature review and introduces the prototype to visually impaired users. Based on the feedback from the introduction, the prototype's ease of use and effectiveness were evaluated, and improvements were made based on user feedback. The algorithms were tested for accuracy, with the prototype achieving a 78% accuracy score, close to GPT-4's 82%. Additionally, tests with 20 users resulted in an 85% user satisfaction rate.

Visual‐Audio Thermoelectric Detectors for Images and Sound Recognition

AbstractInherent in humans is the capacity to perceive music and art, engaging both the visual and auditory senses, with profound effects on physiological and psychological states. Sound and light possess the remarkable ability to transform into thermal energy and, ultimately, electrical signals, playing a crucial role in human sensory perception. This research introduces a previously unmentioned synesthesia‐inspired image and sound recognition system, diverging from conventional image/sound acquisition techniques based on photo/mechanical‐electrical conversion. Leveraging the photo/acoustic‐thermal‐electric effects, the system utilizes micro‐/commercial thermoelectric devices as a conduit for energy conversion. It successfully discriminates monochromatic red, green, blue (RGB) and color coverage, showcasing its proficiency in distinguishing ten digital paintings. Additionally, by probing fiber responses to varied sound frequencies and loudness levels, the system achieves time‐domain identification of four classical music compositions. The device exhibits high sensitivity to detecting input energy and its inputting rate power, offering a novel approach to image and sound recognition through thermal signals. Potential applications span from bionic image sensors and time‐domain thermal monitoring of audio. With further exploration, this thermoelectric‐based system holds promise in quantifying emotional responses to images and sound.

Common Bird Sound Recognition at Vietnam Based on CNN

This article about developing a software extracting bird sound from a website [13], that has sounds of different bird species in Vietnam, explores the CNN model to develop a bird sound recognition system. The process includes conducting methodological experiments on self-collected datasets, providing assessments based on obtained results and building a bird sound recognition application.

Topic Identification in Voice Recordings

Abstract The paper shows the understanding of a topic recognition problem like the speech recognition system based on Natural Language Processing (NLP) and the steps of its implementation of a rules-based approach, which is able to classify given audio materials based on predefined topics in real-time. During implementation, a statistical vocabulary was developed. Google Speech API (Application Programming Interface) was employed for subtitling audio materials, and the most ideal time frame for reception was identified through several experiments. The motivation of this work is based on the deficiency of similar simple systems for Hungarian topic recognition, even though numerous international languages already utilize multiple Automatic Sound Recognition (ASR) systems.

Urban Sound Classification

There are many sounds all around us and our brain can easily and clearly identify them. Furthermore, our brain processes the received sound signals continuously and provides us with relevant environmental knowledge. Although not up to the level of accuracy of the brain, there are some smart devices which can extract necessary information from an audio signal with the help of different algorithms. Over the years, various models like Convolutional Neural Networks (CNNs), Artificial Neural Networks (ANNs), Region- Convolutional Neural Networks (R-CNNs), and numerous machine learning techniques have been employed for sound classification. These methods have shown impressive results in distinguishing spectra-temporal patterns and different sound categories. The novelty of our research lies in showing that the long-short term memory (LSTM) shows a better result in classification accuracy compared to CNN for many features used. Additionally, we've evaluated model accuracy using different techniques such as data augmentation and feature stacking. With our RNN model, we achieved a remarkable accuracy of 87%, setting a new benchmark in performance on the UrbanSound8k dataset. Our findings not only advance the field of sound classification but also underscore the potential of LSTM networks and the importance of innovative techniques such as data augmentation and feature stacking in improving the accuracy of sound recognition systems. Key Words: Sound Classification, Urbansound8k, Librosa, Spectrograms, deep learning, CNN, LSTM.

Hopf physical reservoir computer for reconfigurable sound recognition

The Hopf oscillator is a nonlinear oscillator that exhibits limit cycle motion. This reservoir computer utilizes the vibratory nature of the oscillator, which makes it an ideal candidate for reconfigurable sound recognition tasks. In this paper, the capabilities of the Hopf reservoir computer performing sound recognition are systematically demonstrated. This work shows that the Hopf reservoir computer can offer superior sound recognition accuracy compared to legacy approaches (e.g., a Mel spectrum + machine learning approach). More importantly, the Hopf reservoir computer operating as a sound recognition system does not require audio preprocessing and has a very simple setup while still offering a high degree of reconfigurability. These features pave the way of applying physical reservoir computing for sound recognition in low power edge devices.

Practical Study of Recurrent Neural Networks for Efficient Real-Time Drone Sound Detection: A Review

The detection and classification of engine-based moving objects in restricted scenes from acoustic signals allow better Unmanned Aerial System (UAS)-specific intelligent systems and audio-based surveillance systems. Recurrent Neural Networks (RNNs) provide wide coverage in the field of acoustic analysis due to their effectiveness in widespread practical applications. In this work, we propose to study SimpleRNN, LSTM, BiLSTM, and GRU recurrent network models for real-time UAV sound recognition systems based on Mel-spectrogram using Kapre layers. The main goal of the work is to study the types of RNN networks in a practical sense for a reliable drone sound recognition system. According to the results of an experimental study, the GRU (Gated Recurrent Units) network model demonstrated a higher prediction ability than other RNN architectures for detecting differences and the state of objects from acoustic signals. That is, RNNs gave higher recognition than CNNs for loaded and unloaded audio states of various UAV models, while the GRU model showed about 98% accuracy for determining the UAV load states and 99% accuracy for background noise, which consisted of more other data.

Research on Improved DenseNets Pig Cough Sound Recognition Model Based on SENets

In order to real-time monitor the health status of pigs in the process of breeding and to achieve the purpose of early warning of swine respiratory diseases, the SE-DenseNet-121 recognition model was established to recognize pig cough sounds. The 13-dimensional MFCC, ΔMFCC and Δ2MFCC were transverse spliced to obtain six groups of parameters that could reflect the static, dynamic and mixed characteristics of pig sound signals respectively, and the DenseNet-121 recognition model was used to compare the performance of the six sets of parameters to obtain the optimal set of parameters. The DenseNet-121 recognition model was improved by using the SENets attention module to enhance the recognition model’s ability to extract effective features from the pig sound signals. The results showed that the optimal set of parameters was the 26-dimensional MFCC + ΔMFCC, and the rate of recognition accuracy, recall, precision and F1 score of the SE-DenseNet-121 recognition model for pig cough sounds were 93.8%, 98.6%, 97% and 97.8%, respectively. The above results can be used to develop a pig cough sound recognition system for early warning of pig respiratory diseases.

Domain‐specific neural networks improve automated bird sound recognition already with small amount of local data

Abstract An automatic bird sound recognition system is a useful tool for collecting data of different bird species for ecological analysis. Together with autonomous recording units (ARUs), such a system provides a possibility to collect bird observations on a scale that no human observer could ever match. During the last decades, progress has been made in the field of automatic bird sound recognition, but recognizing bird species from untargeted soundscape recordings remains a challenge. In this article, we demonstrate the workflow for building a global identification model and adjusting it to perform well on the data of autonomous recorders from a specific region. We show how data augmentation and a combination of global and local data can be used to train a convolutional neural network to classify vocalizations of 101 bird species. We construct a model and train it with a global data set to obtain a base model. The base model is then fine‐tuned with local data from Southern Finland in order to adapt it to the sound environment of a specific location and tested with two data sets: one originating from the same Southern Finnish region and another originating from a different region in German Alps. Our results suggest that fine‐tuning with local data significantly improves the network performance. Classification accuracy was improved for test recordings from the same area as the local training data (Southern Finland) but not for recordings from a different region (German Alps). Data augmentation enables training with a limited number of training data and even with few local data samples significant improvement over the base model can be achieved. Our model outperforms the current state‐of‐the‐art tool for automatic bird sound classification. Using local data to adjust the recognition model for the target domain leads to improvement over general non‐tailored solutions. The process introduced in this article can be applied to build a fine‐tuned bird sound classification model for a specific environment.

Design of Abnormal Heart Sound Recognition System Based on HSMM and Deep Neural Network.

IntroductionHeart sound signal is an important physiological signal of human body, and the identification and research of heart sound signal is of great significance.MethodsFor abnormal heart sound signal recognition, an abnormal heart sound recognition system, combining hidden semi-Markov models (HSMM) with deep neural networks, is proposed. Firstly, HSMM is used to build a heart sound segmentation model to accurately segment the heart sound signal, and then the segmented heart sound signal is subjected to feature extraction. Finally, the trained deep neural network model is used for recognition.ResultsCompared with other methods, this method has a relatively small amount of input feature data and high accuracy, fast recognition speed.DiscussionHSMM combined with deep neural network is expected to be deployed on smart mobile devices for telemedicine detection.

A High-Fidelity Skin-Attachable Acoustic Sensor for Realizing Auditory Electronic Skin.

Wearable auditory sensors are critical in user-friendly sound-recognition systems for smart human-machine interaction and the Internet of Things. However, previously reported wearable sensors have limited sound-sensing quality as a consequence of a poor frequency response and a narrow acoustic-pressure range. Here, a skin-attachable acoustic sensor is presented that has higher sensing accuracy in wider auditory field than human ears, with flat frequency response (15-10 000Hz) and a good range of linearity (29-134 dBSPL ) as well as high conformality to flexible surfaces and human skin. This high sound-sensing quality is achieved by exploiting the low residual stress and high processability of polymer materials in a diaphragm structure designed using acousto-mechano-electric modeling. Thus, this acoustic sensor shows high acoustic fidelity by sensing human-audible sounds, even loud sounds and low-frequency sounds that human ears cannot detect without distorting them. The polymer-based ultrasmall (<9 mm2 ) and thin sensor maintains sound-detection quality on flexible substrates and in a wide temperature range (25 to 90°C). The acoustic sensor shows a significant potential of auditory electronic skin, by recognizing voice successfully when the sensor attached on human skin is connected to a commercial mobile device running the latest artificial intelligence assistant.

A Study on Sound Recognition System Based on 2-D Transformation and CNN Deep Learning

본 논문은 일상생활에서 흔히 들을 수 있는 소리(비명소리, 박수 소리, 여러 명의 박수 소리, 자동차 지나가는 소리, 배경음 등)를 감지하는 음향 인식을 위하여, 신호처리 및 딥러닝을 적용하는 연구에 관한 것이다. 제안된 음향 인식에서는, 인식 정확도의 향상을 위해서 음향 파형의 스펙트럼, 음향 데이터의 증강, 2차원(2-D) 이미지 변환에 관한 기술들이 사용되었고, 예측의 정확도를 향상을 위한 앙상블 학습, Convolution Neural Network(CNN) 딥러닝 기술들이 적용된다. 제안된 음향 인식 기술은 실험을 통해 다양한 음향을 정확하게 인식할 수 있음을 보여준다.

Learning Representations for New Sound Classes With Continual Self-Supervised Learning

In this paper, we work on a sound recognition system that continually incorporates new sound classes. Our main goal is to develop a framework where the model can be updated without relying on labeled data. For this purpose, we propose adopting representation learning, where an encoder is trained using unlabeled data. This learning framework enables the study and implementation of a practically relevant use case where only a small amount of the labels is available in a continual learning context. We also make the empirical observation that a similarity-based representation learning method within this framework is robust to forgetting even if no explicit mechanism against forgetting is employed. We show that this approach obtains similar performance compared to several distillation-based continual learning methods when employed on self-supervised representation learning methods.

URBAN SOUND RECOGNITION USING DIFFERENT FEATURE EXTRACTION TECHNIQUES

The application of the advanced methods for noise analysis in the urban areas through the development of systems for classification of sound events significantly improves and simplifies the process of noise assessment. The main purpose of sound recognition and classification systems is to develop algorithms that can detect and classify sound events that occur in the chosen environment, giving an appropriate response to their users. In this research, a supervised system for recognition and classification of sound events has been established through the development of feature extraction techniques based on digital signal processing of the audio signals that are further used as an input parameter in the machine learning algorithms for classification of the sound events. Various audio parameters were extracted and processed in order to choose the best set of parameters that result in better recognition of the class to which the sounds belong. The created acoustic event detection and classification (AED/C) system could be further implemented in sound sensors for automatic control of environmental noise using the source classification that leads to reduced amount of required human validation of the sound level measurements since the target noise source is evidently defined.

English Phrase Speech Recognition Based on Continuous Speech Recognition Algorithm and Word Tree Constraints

This paper combines domestic and international research results to analyze and study the difference between the attribute features of English phrase speech and noise to enhance the short-time energy, which is used to improve the threshold judgment sensitivity; noise addition to the discrepancy data set is used to enhance the recognition robustness. The backpropagation algorithm is improved to constrain the range of weight variation, avoid oscillation phenomenon, and shorten the training time. In the real English phrase sound recognition system, there are problems such as massive training data and low training efficiency caused by the super large-scale model parameters of the convolutional neural network. To address these problems, the NWBP algorithm is based on the oscillation phenomenon that tends to occur when searching for the minimum error value in the late training period of the network parameters, using the K-MEANS algorithm to obtain the seed nodes that approach the minimal error value, and using the boundary value rule to reduce the range of weight change to reduce the oscillation phenomenon so that the network error converges as soon as possible and improve the training efficiency. Through simulation experiments, the NWBP algorithm improves the degree of fitting and convergence speed in the training of complex convolutional neural networks compared with other algorithms, reduces the redundant computation, and shortens the training time to a certain extent, and the algorithm has the advantage of accelerating the convergence of the network compared with simple networks. The word tree constraint and its efficient storage structure are introduced, which improves the storage efficiency of the word tree constraint and the retrieval efficiency in the English phrase recognition search.

Sound Sources Recognition System Based on Convolutional Neural Network Using a New Dataset in Noisy Environment

A sound source recognition system based on a convolutional neural network (CNN) using a new dataset in a noisy environment was proposed and its performance was analyzed in this paper. The deep learning model in the system consists of a convolutional layer with max-pooling and a fully connected layer based on CNN, and classifies short audio clips of surrounding sounds. The performance of the proposed system was produced by measuring with the confusion matrix (CM) and the sound source recognition accuracy (SSRA). As a result of comparing the average SSRA of the proposed system applying the new sound dataset reflecting the background noise condition and the existing system applying the noiseless dataset, it was measured as 72–76% and 40–55%, respectively. Finally, it can be seen that the case of applying the new sound source dataset was superior to the case of applying the noiseless dataset by about 21–32% on average.

Human beatbox sound recognition using an automatic speech recognition toolkit

Human beatboxing is a vocal art making use of speech organs to produce vocal drum sounds and imitate musical instruments. Beatbox sound classification is a current challenge that can be used for automatic database annotation and music-information retrieval. In this study, a large-vocabulary human-beatbox sound recognition system was developed with an adaptation of Kaldi toolbox, a widely-used tool for automatic speech recognition. The corpus consisted of eighty boxemes, which were recorded repeatedly by two beatboxers. The sounds were annotated and transcribed to the system by means of a beatbox specific morphographic writing system (Vocal Grammatics). The recognition-system robustness to recording conditions was assessed on recordings of six different microphones and settings. The decoding part was made with monophone acoustic models trained with a classical HMM-GMM model. A change of acoustic features (MFCC, PLP, Fbank) and a variation of different parameters of the beatbox recognition system were tested: (i) the number of HMM states, (ii) the number of MFCC, (iii) the presence or not of a pause boxeme in right and left contexts in the lexicon and (iv) the rate of silence probability. Our best model was obtained with the addition of a pause in left and right contexts of each boxeme in the lexicon, a 0.8 silence probability, 22 MFCC and three states HMM. Boxeme error rate in such configuration was lowered to 13.65%, and 8.6 boxemes over 10 were well recognized. The recording settings did not greatly affect system performance, apart from recording with closed-cup technique.

GIS Mechanical Fault Diagnosis Method Based on Middle Time Mel Cepstrum Coefficient

Mechanical fault is a common fault of gas insulated switchgear (GIS). If not found in time, it will cause major safety hazards such as opening and closing fault. In this paper, a diagnosis method based on improved Mel cepstrum coefficient for GIS mechanical fault on-line monitoring is proposed. Firstly, the Mel frequency cepstral coefficients (MFCC) are extracted from the preprocessed sound signals; in order to adapt to the characteristics of smooth sound energy change in GIS operation, the MFCC is optimized to obtain improved features; Support Vector Machine (SVM) is introduced to build a GIS mechanical fault diagnosis model based on acoustics and uses bagging algorithm to integrate the SVM model. In this study, mechanical fault is simulated on real GIS to obtain real fault sound signals for training and testing. The experimental results show that compared with the traditional MFCC, the improved MFCC has a higher recognition accuracy in GIS fault sound recognition system. The F1-score is generally over 80%, and the F1-score of specific mechanisms can even reach 99%.

Efficient and robust approaches for three-dimensional sound source recognition and localization using humanoid robots sensor arrays

Efficient and robust sound source recognition and localization is one of the basic techniques for humanoid robots in terms of reaction to environments. Due to the fixed sensor arrays and limited computation resources in humanoid robots, there comes challenge for sound source recognition and localization. This article proposes a sound source recognition and localization framework to realize real-time and precise sound source recognition and localization system using humanoid robots’ sensor arrays. The type of the audio is recognized according to the cross-correlation function. And steered response power-phase transform function in discrete angle space is used to search the sound source direction. The sound source recognition and localization framework presents a new multi-robots collaboration system to get the precise three-dimensional sound source position and introduces a distance weighting revision way to optimize the localization performance. Additionally, the experiment results carried out on humanoid robot NAO demonstrate that the proposed approaches can recognize and localize the sound source efficiently and robustly.

Environment Sound Classification System Based on Hybrid Feature and Convolutional Neural Network

At present, the environment sound recognition system mainly identifies environment sounds with deep neural networks and a wide variety of auditory features. Therefore, it is necessary to analyze which auditory features are more suitable for deep neural networks based ESCR systems. In this paper, we chose three sound features which based on two widely used filters:the Mel and Gammatone filter banks. Subsequently, the hybrid feature MGCC is presented. Finally, a deep convolutional neural network is proposed to verify which features are more suitable for environment sound classification and recognition tasks. The experimental results show that the signal processing features are better than the spectrogram features in the deep neural network based environmental sound recognition system. Among all the acoustic features, the MGCC feature achieves the best performance than other features. Finally, the MGCC-CNN model proposed in this paper is compared with the state-of-the-art environmental sound classification models on the UrbanSound 8K dataset. The results show that the proposed model has the best classification accuracy.