Sound Recognition Research Articles

Silicon integrated photonic-electronic neuron for noise-resilient deep learning

This paper presents an experimental demonstration of the photonic segment of a photonic-electronic multiply accumulate neuron (PEMAN) architecture, employing a silicon photonic chip with high-speed electro-absorption modulators for matrix-vector multiplications. The photonic integrated circuit has been evaluated through a noise-sensitive three-layer neural network (NN) with 1350 trainable parameters targeting heartbeat sound classification for health monitoring purposes. Its experimental validation revealed F1-scores of 85.9% and 81% at compute rates of 10 and 20 Gbaud, respectively, exploiting quantization- and noise-aware deep learning techniques and introducing a novel activation function slope stretching strategy for mitigating noise impairments. The enhanced noise-resilient properties of this novel training model are confirmed via simulations for varying noise levels, being in excellent agreement with the respective experimental data obtained at 10, 20, and 30 Gbaud symbol rates.

An intrinsic electromechanical design method for MXene-based electrodes of ultra-highly sensitive and stable pseudocapacitive pressure device

To achieve the optimal performances of flexible pressure devices, the current approaches typically involve extensive experimental trials. However, such methods lack guidance from theoretical models in a positive fashion, not only introducing unpredictability and uncertainty into results but also potentially limiting the applicability of any achieved advancements to the alternative material systems. Here, an electrical-mechanical design model, combining density functional theory and molecular dynamics simulations, was proposed for calculating the specific capacitance and mechanical strength of various modified MXene films. Based on which, the optimal hybrid film entailed the cross-linking of holey reduced graphene oxide and MXene via cysteamine was prepared. This film, upon electromechanical characterizations, has effectively overcome the issues of MXene self-stacking and brittleness, thus validating the conclusions of theoretical predictions. Subsequently, the proposed film was integrated as electrode into a pseudocapacitive pressure sensor, achieving highly sensitivity (∼143,258 kPa−1) and mechanical stability (with capacitance drift below 1.2 % after 10,000-repeated test). Ultimately, the fabricated sensors were developed into sound collection system and robotic skins, respectively, to achieve recognition of distinctive sounds and stable perception of gravity. The calculation-based approach holds enormous potential for the design of two-dementional materials-based pressure sensors.

Crack-Based Composite Flexible Sensor with Superhydrophobicity to Detect Strain and Vibration.

Vibration sensors are widely applied in the detection of faults and analysis of operational states in engineering machinery and equipment. However, commercial vibration sensors with a feature of high hardness hinder their usage in some practical applications where the measured objects have irregular surfaces that are difficult to install. Moreover, as the operating environments of machinery become increasingly complex, there is a growing demand for sensors capable of working in wet and humid conditions. Here, we present a flexible, superhydrophobic vibration sensor with parallel microcracks. The sensor is fabricated using a femtosecond laser direct writing ablation strategy to create the parallel cracks on a PDMS film, followed by spray-coating with a conductive ink composed of MWCNTs, CB, and PDMS. The results demonstrate that the developed flexible sensor exhibits a high-frequency response of up to 2000 Hz, a high acceleration response of up to 100 m/s2, a water contact angle as high as 159.61°, and a linearity of 0.9812 between the voltage signal and acceleration. The results indicate that the sensor can be employed for underwater vibration, sound recognition, and vibration monitoring in fields such as shield cutters, holding significant potential for mechanical equipment vibration monitoring and speech-based human-machine interaction.

Snn and sound: a comprehensive review of spiking neural networks in sound.

The rapid advancement of AI and machine learning has significantly enhanced sound and acoustic recognition technologies, moving beyond traditional models to more sophisticated neural network-based methods. Among these, Spiking Neural Networks (SNNs) are particularly noteworthy. SNNs mimic biological neurons and operate on principles similar to the human brain, using analog computing mechanisms. This capability allows for efficient sound processing with low power consumption and minimal latency, ideal for real-time applications in embedded systems. This paper reviews recent developments in SNNs for sound recognition, underscoring their potential to overcome the limitations of digital computing and suggesting directions for future research. The unique attributes of SNNs could lead to breakthroughs in mimicking human auditory processing more closely.

Bio-inspired self-healing polyurethane system: Mimicking connective tissue with hydrogen-bonding mechanism

Achieving a balance between mechanical strength and self-healing efficiency in self-healing materials is challenging yet crucial. This work introduces PCL-HBU, a polyurethane material designed for the first time to mimic collagen fibers, elastic fibers, and vascular networks of connective tissue, with excellent mechanical properties and self-healing efficiency. Meanwhile, by adjusting the 2-ureido-4-[1H]-pyrimidinone (UPy) and N,N-bis(2-hydroxyethyl)oxamide (BHO) ratio, this work creatively developed PCL-HBU3 with three distinct phase structures: amorphous region, soft segment crystallisation, and hard segment crystallisation, which approach confers excellent mechanical properties to the material. PCL-HBU3 exhibits a tensile strength of 70.0 MPa and elongation at break of 1500 %. It can recover from 600 % elongation within 20 s, achieving 100 % elongation at break self-healing efficiency. Furthermore, the composite conductive material fabricated using PCL-HBU3 as the matrix showcases self-healing abilities and holds promise in motion detection and sound recognition applications.

Signal recognition based on transfer learning for Φ-OTDR fiber optic distributed disturbance sensor

Abstract Traditional classification models for optical fiber vibration signals can achieve good recognition results when there are a large number of samples. However, it is difficult to obtain sufficient training samples in real scenes. A method of Φ-OTDR sensor signal recognition based on VGGish transfer learning is proposed in this paper. Firstly, a selection indicator is designed to filter out vibration signals from sparse signals. Then, the VGGish network pretrained for sound classification is used for transfer learning as a feature extractor for optical fiber vibration signals. This method was applied to classify 6 kinds of optical fiber vibration signals. When the number of training samples is reduced to 480 samples, the method can still achieve 84.17% classification accuracy. Compared to the method of training from scratch, this proposed method reduces training time by 73.9%.

Sound classification and power consumption to sound intensity relation as a tool for wood machining monitoring

Sound classification and power consumption to sound intensity relation as a tool for wood machining monitoring

A barking emotion recognition method based on Mamba and Synchrosqueezing Short-Time Fourier Transform

Dog barks are a crucial means for dogs to express their emotions, conveying needs and emotional states such as anger, sadness, or excitement. With the increasing number of pet dogs, understanding their emotional states is essential for better care and interaction. However, accurately interpreting the emotional content of dog barks is often challenging for humans. Therefore, developing a technology for recognizing the emotions in dog barks is of great importance. In this paper, we establish a dataset for dog bark emotion recognition and propose a method using the Synchrosqueezing Short-Time Fourier Transform (SST_STFT) to extract the time–frequency features of dog barks. Considering the characteristics of dog barks, we design an emotion recognition model based on Mamba. This model leverages the state-space model (Select SMM) for global modeling of long sequence features, enabling rapid and effective processing of time–frequency features and accurate perception of their variations. Experiments on the self-built DogEmotionSound dataset, as well as the public IEMOCAP and UrbanSound8K datasets, demonstrate that our model outperforms existing state-of-the-art sound recognition models across various evaluation metrics. The recognition accuracies on the three datasets are 91.97%, 95.36%, and 67.25%, respectively. The code is open-source at https://github.com/yangcjya/A-barking-emotion-recognition-method-based-on-Mamba.git.

Sound Classification for Non-Destructive Diagnosis of Basal Stem Rot Disease based on Stem Density in Oil Palm Trunks

Oil palm trees play a crucial role in the economies of Southeast Asian countries, providing palm oil that is essential for various products and consumption. However, the Basal Stem Rot (BSR) disease caused by the white-rot fungus Ganoderma spp. poses a significant challenge, leading to wilting and death of the trees. This study proposes a practical and accessible approach for the routine detection of Basal Stem Rot (BSR) disease in oil palm trees caused by Ganoderma spp. A prototype system for initial disease diagnosis is developed by utilizing the local wisdom technique of tapping and assessing knocking sounds based on stem density in oil palm trunks. Three algorithms, Convolutional Neural Network (CNN), Support Vector Machine Classifier (SVC), and Multi-Layer Perceptron (MLP) models are compared for accuracy in two sound recognition experiments. The first experiment involves a three-class classification model (healthy, boundary, and infected areas), while the second excludes the boundary area. Results demonstrate that the CNN model outperforms SVC and MLP, achieving the highest accuracy of 90.73% in the two-class scenario and 84.97% in the three-class scenario, along with superior precision, recall, and F1 score. The CNN model proves to be the most effective in accurately classifying the data in both scenarios compared to the other models. This prototype system offers promising implications for the early detection and proactive management of Ganoderma spp. disease, contributing to the preservation and sustainability of oil palm plantations in Southeast Asia.

Gated parametric neuron for spike-based audio recognition

Spiking neural networks (SNNs) aim to simulate real neural networks in the human brain with biologically plausible neurons. The leaky integrate-and-fire (LIF) neuron is one of the most widely studied SNN architectures. However, it has the vanishing gradient problem when trained with backpropagation. Additionally, its neuronal parameters are often manually specified and fixed, in contrast to the heterogeneity of real neurons in the human brain. This paper proposes a gated parametric neuron (GPN) to process spatio-temporal information effectively with the gating mechanism. Compared with the LIF neuron, the GPN has two distinguishing advantages: (1) it copes well with the vanishing gradients by improving the flow of gradient propagation; and, (2) it learns spatio-temporal heterogeneous neuronal parameters automatically. Additionally, we use the same gate structure to eliminate initial neuronal parameter selection and design a hybrid recurrent neural network-SNN structure. Experiments on two spike-based audio datasets demonstrated that the GPN network outperformed several state-of-the-art SNNs, could mitigate vanishing gradients, and had spatio-temporal heterogeneous parameters. Our work shows the ability of SNNs to handle long-term dependencies and achieve high performance simultaneously.

The EU Detergent Industry Network ‘DetNet’ – an industry approach for the EU CLP classification of detergent and cleaning products for skin and eye effects

Abstract The implementation of the EU CLP Regulation for mixtures in 2015 led to a reduction of the cut-off levels for mixtures containing severely irritating or corrosive ingredients. As a result, most everyday detergents would be required to carry the same corrosive pictogram on the label as caustic or highly acidic specialty cleaners. However, in most cases, calculation-derived skin and eye hazard classifications do not reflect in-market experience and consumer understanding of the hazardous properties of household detergents. The detergent industry responded to this situation by forming the Detergent Industry Network ‘DetNet’. It is a transparent process that supports mixture classification decisions by sharing data, expertise, and best practices between manufacturers of detergent and cleaning products. At the core of DetNet is its database providing access to compositional information and toxicological data on more than 240 detergents and cleaning products. This paper describes and discusses the main features of DetNet, its organisational structure and database. Based on a case study, the process and functioning of DetNet to support a scientifically sound classification of an untested mixture for skin and eye effects based on data available for similar mixtures is demonstrated.

Coherent Feature Extraction with Swarm Intelligence Based Hybrid Adaboost Weighted ELM Classification for Snoring Sound Classification.

For patients suffering from obstructive sleep apnea and sleep-related breathing disorders, snoring is quite common, and it greatly interferes with the quality of life for them and for the people surrounding them. For diagnosing obstructive sleep apnea, snoring is used as a screening parameter, so the exact detection and classification of snoring sounds are quite important. Therefore, automated and very high precision snoring analysis and classification algorithms are required. In this work, initially the features are extracted from six different domains, such as time domain, frequency domain, Discrete Wavelet Transform (DWT) domain, sparse domain, eigen value domain, and cepstral domain. The extracted features are then selected using three efficient feature selection techniques, such as Golden Eagle Optimization (GEO), Salp Swarm Algorithm (SSA), and Refined SSA. The selected features are finally classified with the help of eight traditional machine learning classifiers and two proposed classifiers, such as the Firefly Algorithm-Weighted Extreme Learning Machine hybrid with Adaboost model (FA-WELM-Adaboost) and the Capuchin Search Algorithm-Weighted Extreme Learning Machine hybrid with Adaboost model (CSA-WELM-Adaboost). The analysis is performed on the MPSSC Interspeech dataset, and the best results are obtained when the DWT features with the refined SSA feature selection technique and FA-WELM-Adaboost hybrid classifier are utilized, reporting an Unweighted Average Recall (UAR) of 74.23%. The second-best results are obtained when DWT features are selected with the GEO feature selection technique and a CSA-WELM-Adaboost hybrid classifier is utilized, reporting an UAR of 73.86%.

Sounds in common: Time-frequency as the classification parameters for pulsed sounds produced by Delphinus delphis

Sounds produced by dolphins can be grouped into tonal (whistles) and pulsed sounds (e.g., echolocation clicks and burst sounds). Clicks are broadband pulses temporarily spaced to allow echo processing between the sound source and the object. Echolocation is related mainly to prey detection and environmental recognition. Echolocation click trains tend to present a decreasing inter-click interval due to a continuous changing of the target's location when the animal approaches to capture the prey. In addition to foraging and feeding contexts, burst pulsed sounds have been associated with short social communication. Although echolocation clicks are relatively well documented, there is no consensus regarding the broad variety of the burst pulsed signals. The present study analyzed time-frequency characteristics by conducting a clustering and discrimination analysis to classify pulsed sounds. A total of 64 click trains were analyzed from short-beaked common dolphins recorded in the slope region of the western South Atlantic Ocean. Three analyses (time; frequency; and combined time-frequency parameters) were compared through k-means clustering and posterior cluster validation using Random forest analysis. The k-means clustering resulted in four clusters for all groups of analysis. The time parameters were the most accurate among the comparisons, with the first two-dimensional axis corresponding to 87 % (Dim1 = 70.2 % and Dim2 = 17.2 %). The random forest analysis showed that the time-frequency dataset was the best classification of pulsed sounds in D. delphis (Accuracy = 84.6 %; confidence interval CI = 65.1 %-95.6 %; p < 0.01). This result considers the animal an acoustical identity, emphasizing the importance of certain parameters that influence this identity and thus reflecting the energy-cost optimization for sound production.

A Study of Improved Two-Stage Dual-Conv Coordinate Attention Model for Sound Event Detection and Localization.

Sound Event Detection and Localization (SELD) is a comprehensive task that aims to solve the subtasks of Sound Event Detection (SED) and Sound Source Localization (SSL) simultaneously. The task of SELD lies in the need to solve both sound recognition and spatial localization problems, and different categories of sound events may overlap in time and space, making it more difficult for the model to distinguish between different events occurring at the same time and to locate the sound source. In this study, the Dual-conv Coordinate Attention Module (DCAM) combines dual convolutional blocks and Coordinate Attention, and based on this, the network architecture based on the two-stage strategy is improved to form the SELD-oriented Two-Stage Dual-conv Coordinate Attention Model (TDCAM) for SELD. TDCAM draws on the concepts of Visual Geometry Group (VGG) networks and Coordinate Attention to effectively capture critical local information by focusing on the coordinate space information of the feature map and dealing with the relationship between the feature map channels to enhance the feature selection capability of the model. To address the limitation of a single-layer Bi-directional Gated Recurrent Unit (Bi-GRU) in the two-stage network in terms of timing processing, we add to the structure of the two-layer Bi-GRU and introduce the data enhancement techniques of the frequency mask and time mask to improve the modeling and generalization ability of the model for timing features. Through experimental validation on the TAU Spatial Sound Events 2019 development dataset, our approach significantly improves the performance of SELD compared to the two-stage network baseline model. Furthermore, the effectiveness of DCAM and the two-layer Bi-GRU structure is confirmed by performing ablation experiments.

Review of Phonocardiogram Signal Analysis: Insights from the PhysioNet/CinC Challenge 2016 Database

The phonocardiogram (PCG) is a crucial tool for the early detection, continuous monitoring, accurate diagnosis, and efficient management of cardiovascular diseases. It has the potential to revolutionize cardiovascular care and improve patient outcomes. The PhysioNet/CinC Challenge 2016 database, a large and influential resource, encourages contributions to accurate heart sound state classification (normal versus abnormal), achieving promising benchmark performance (accuracy: 99.80%; sensitivity: 99.70%; specificity: 99.10%; and score: 99.40%). This study reviews recent advances in analytical techniques applied to this database, and 104 publications on PCG signal analysis are retrieved. These techniques encompass heart sound preprocessing, signal segmentation, feature extraction, and heart sound state classification. Specifically, this study summarizes methods such as signal filtering and denoising; heart sound segmentation using hidden Markov models and machine learning; feature extraction in the time, frequency, and time-frequency domains; and state-of-the-art heart sound state recognition techniques. Additionally, it discusses electrocardiogram (ECG) feature extraction and joint PCG and ECG heart sound state recognition. Despite significant technical progress, challenges remain in large-scale high-quality data collection, model interpretability, and generalizability. Future directions include multi-modal signal fusion, standardization and validation, automated interpretation for decision support, real-time monitoring, and longitudinal data analysis. Continued exploration and innovation in heart sound signal analysis are essential for advancing cardiac care, improving patient outcomes, and enhancing user trust and acceptance.

HM–GDM: Hybrid Measures and Graph-Dependent Modeling for Environmental Sound Classification

One of the upcoming areas of research is the automated environmental sound classification, as its application is highly utilized in criminal investigations scenarios, surveillance systems, biomedical fields, radio navigation systems, etc. The procedure of environmental sound classification deals with standardized techniques such as feature extraction and feature selection, followed by classification with machine learning or deep learning techniques. In this work, five feature extraction techniques are used initially such as the discrete wavelet transform (DWT)-based empirical mode decomposition (EMD) technique, phase space reconstruction (PSR), kernel principal component analysis (KPCA), variational mode decomposition (VMD) and tunable Q-wavelet transform (TQWT). The extracted features are then selected through seven feature selection techniques, out of which two have been proposed newly and five have already existed. The proposed feature selection techniques are stratified clustered collective technique (SCCT) and fuzzy-based template shape clustering (FTSC). The other feature selection techniques used are the generalized discriminant analysis (GDA), ReliefF, Fisher discriminant criterion (FDC), Kruskal–Wallis test and coati optimization algorithm (COA). The selected features are then classified through the proposed swarm intelligence-based hybrid Adaboost – random forest termed as SIHAR classifier and the selected features are classified with the less explored sparse representation classifier (SRC) along with eight other traditionally used machine learning classifiers. The work also proposes graph-dependent modeling for the environmental sounds where the rhythms are extracted for the similarity assessment and later the decision-making is done using hypothesis testing. The work is tested on Firat ESC-50 dataset and the best results are produced in terms of a high classification accuracy of 87.48% which is obtained for the TQWT + SCCT + SIHAR combination.

Towards Understanding Cat Vocalizations: A Novel Cat Sound Classification Model Based on Vision Transformers

Animal sound perception systems are highly developed compared to humans, crucial for survival in natural environments, with some species possessing specialized sensory capabilities such as vision, hearing, touch, and environmental awareness. Understanding animal sounds not only aids in their own communication and survival but also benefits humans in various fields including security, natural disaster prediction, ecological research, bioacoustics, precision agriculture, and search and rescue operations. Motivated by this fact, this study investigated the classification of cat sounds using deep learning models based on Vision Transformer (ViT) and Convolutional Neural Network (CNN) architectures. Cat vocalizations, represented as mel-spectrograms, were classified using models trained on a diverse dataset of cat sounds. Experimental results demonstrated the superiority of the proposed model based on Microsoft’s BERT Pre-Training of Image Transformers (BEiT) over the state-of-the-art as it obtained an exceptional accuracy of 96.95%. Additionally, it was observed that the proposed models based on ViT outperformed CNN-based models, highlighting the efficacy of transformer architectures in capturing complex patterns within audio data. These findings underscore the potential of ViT architectures in decoding animal communication systems and advancing wildlife conservation efforts.

The heart sound classification of congenital heart disease by using median EEMD-Hurst and threshold denoising method.

Heart sound signals are vital for the machine-assisted detection of congenital heart disease. However, the performance of diagnostic results is limited by noise during heart sound acquisition. A limitation of existing noise reduction schemes is that the pathological components of the signal are weak, which have the potential to be filtered out with the noise. In this research, a novel approach for classifying heart sounds based on median ensemble empirical mode decomposition (MEEMD), Hurst analysis, improved threshold denoising, and neural networks are presented. In decomposing the heart sound signal into several intrinsic mode functions (IMFs), mode mixing and mode splitting can be effectively suppressed by MEEMD. Hurst analysis is adopted for identifying the noisy content of IMFs. Then, the noise-dominated IMFs are denoised by an improved threshold function. Finally, the noise reduction signal is generated by reconstructing the processed components and the other components. A database of 5000 heart sounds from congenital heart disease and normal volunteers was constructed. The Mel spectral coefficients of the denoised signals were used as input vectors to the convolutional neural network for classification to verify the effectiveness of the preprocessing algorithm. An accuracy of 93.8%, a specificity of 93.1%, and a sensitivity of 94.6% were achieved for classifying the normal cases from abnormal one.

Evaluation of Primary Reading Teaching in Multigrade Classes According to Teachers' Views

Multigrade classes which are at least two or more grade’s students are brought together in the same classroom and educational activities are carried out simultaneously by a single teacher. The main purpose of this study is how this practice, closely related to many teachers, students and parents, is evaluated by teachers. The aim of this study is to reveal how teachers explain this experience, its positive and negative aspects, and the difficulties experienced during sound-based sentence teaching. Semi-structured interviews were held with 7 teachers in order to evaluate primary reading instruction in multigrade classes according to teachers' opinions. The data regarding the participants selected by the criterion sampling method were transferred to the word environment on the computer and descriptive analysis was performed. The analyses were evaluated and interpreted. Teachers stated that teaching in multigrade classrooms is quite different from teaching in independent classrooms. They stated that this experience provides greater satisfaction at the teaching profession and increases the sense of belonging to the school. They also stated that it is important to step in the role of community leadership and to reside in the village. Among the positive aspects of teaching in multigrade classes; supporting students’ social and emotional development; supporting cognitive development by more effective usage of peer teaching, and the small class member number. Among the negative aspects, there are time constraints, emotional difficulties experienced by teachers, insufficient readiness of students, upper classes to make fun of lower classes, physical limitations and material deficiencies. It has been mentioned that some teachers changed the order of the letters during the teaching reading, that the region where the school is located is a place with intense dialect, making the process difficult, and the difficulties of performing some sound recognition activities. In order to allocate more resources to the relevant schools in order to overcome the difficulties experienced in the multigrade classrooms and to enable the teachers to start the process more ready, it is recommended that the education courses in the multi-grade classrooms conducted at the university should be more practice-based. The participants in this study are teachers. A study can be conducted that addresses the students' perspective.

Impact of interference on vocal and instrument recognition.

Voices arguably occupy a superior role in auditory processing. Specifically, studies have reported that singing voices are processed faster and more accurately and possess greater salience in musical scenes compared to instrumental sounds. However, the underlying acoustic features of this superiority and the generality of these effects remain unclear. This study investigates the impact of frequency micro-modulations (FMM) and the influence of interfering sounds on sound recognition. Thirty young participants, half with musical training, engage in three sound recognition experiments featuring short vocal and instrumental sounds in a go/no-go task. Accuracy and reaction times are measured for sounds from recorded samples and excerpts of popular music. Each sound is presented in separate versions with and without FMM, in isolation or accompanied by a piano. Recognition varies across sound categories, but no general vocal superiority emerges and no effects of FMM. When presented together with interfering sounds, all sounds exhibit degradation in recognition. However, whereas /a/ sounds stand out by showing a distinct robustness to interference (i.e., less degradation of recognition), /u/ sounds lack this robustness. Acoustical analysis implies that recognition differences can be explained by spectral similarities. Together, these results challenge the notion of general vocal superiority in auditory perception.