Sound Detection Research Articles

Prototype Smart Baby’s Room Berbasis Iot Di PT Ozami Inti Synergy Untuk Orang Tua Sibuk

This research aims to design and develop a prototype of the Smart Baby's Room based on the Internet of Things (IoT) at PT Ozami Inti Synergy,as a solution for busy parents. This tool is designed to monitor baby's condition and surrounding environment in real-time through a mobile application. Key features include monitoring temperature, humidity, air quality, as well as sound and motion detection of the baby. Data from these sensors is transmitted to an IoT platform, allowing parents to receive notifications and remotely monitor the conditions remotely. This implementation is expected to improve baby's safety and comfort, as well as providing peace of mind for parents who cannot always be at home. parents who cannot always be at home. This prototype was tested to ensure its reliability and effectiveness in a controlled environment.

Joint Spatio-Temporal-Frequency Representation Learning for Improved Sound Event Localization and Detection.

Sound event localization and detection (SELD) is a crucial component of machine listening that aims to simultaneously identify and localize sound events in multichannel audio recordings. This task demands an integrated analysis of spatial, temporal, and frequency domains to accurately characterize sound events. The spatial domain pertains to the varying acoustic signals captured by multichannel microphones, which are essential for determining the location of sound sources. However, the majority of recent studies have focused on time-frequency correlations and spatio-temporal correlations separately, leading to inadequate performance in real-life scenarios. In this paper, we propose a novel SELD method that utilizes the newly developed Spatio-Temporal-Frequency Fusion Network (STFF-Net) to jointly learn comprehensive features across spatial, temporal, and frequency domains of sound events. The backbone of our STFF-Net is the Enhanced-3D (E3D) residual block, which combines 3D convolutions with a parameter-free attention mechanism to capture and refine the intricate correlations among these domains. Furthermore, our method incorporates the multi-ACCDOA format to effectively handle homogeneous overlaps between sound events. During the evaluation, we conduct extensive experiments on three de facto benchmark datasets, and our results demonstrate that the proposed SELD method significantly outperforms current state-of-the-art approaches.

Blind assistance system for appliance control and public transport safety using CNN, MobileNet V2 and Yolo V8

The purpose of this research is to address the challenges faced by visually impaired individuals, particularly in handling household appliances independently. With approximately 285 million visually impaired individuals worldwide, technological solutions are crucial to enhancing their accessibility and independence. This paper introduces a Smart Assistance System designed to empower visually impaired individuals to interact with household appliances in real-time without assistance. In this study, three Convolutional Neural Network (CNN) algorithms are compared to develop the system. The evaluation metrics include accuracy, precision, recall, F1 score, and hamming loss on validation images. The performance comparison reveals that the custom architecture CNN, MobileNetv2, and YOLO models achieve F1 scores of 0.43, 0.63, and 0.24, respectively. To enhance object detection and classification, the paper suggests implementing bounding box buttons categorization using YOLOv8, which demonstrates superior performance with a 95% classification accuracy on testing images of home appliance buttons. They face similar difficult while in public and accessing public property. Expanding upon the proposed system’s capabilities, the paper introduces the concept of panic button detection and activation in a bus environment tailored for blind individuals. This system relies on various factors such as the number of people onboard, heart rate monitoring, and the detection of distress signals or SOS sounds emitted by the user. By integrating advanced sensing technologies and intelligent algorithms, this panic button detection system aims to provide prompt assistance and ensure the safety of visually impaired passengers in public transportation settings.

EXPRESS: Pure-tone audiometry and dichotic listening in primary progressive aphasia and Alzheimer's disease

Hearing is multifaceted and the relative contributions of peripheral and central hearing loss are rarely considered together in the context of dementia. Here, we assessed peripheral (as measured with pure-tone audiometry) and central (as measured with dichotic listening) hearing in 19 patients with typical amnestic Alzheimer’s disease (tAD), 10 patients with logopenic variant primary progressive aphasia (lvPPA), 11 patients with nonfluent/agrammatic variant PPA (nfvPPA), 15 patients with semantic variant PPA (svPPA), and 28 healthy age-matched individuals. Participants also underwent neuropsychological assessment and magnetic resonance image scanning, allowing us to use voxel-based morphometry to assess associations between hearing scores and grey matter volume. Dichotic listening was impaired in all patient groups relative to healthy controls. In the combined patient (but not healthy control) cohort, dichotic listening scores were significantly correlated with measures of global cognitive functioning and speech-based neuropsychological tasks. Pure-tone audiometry scores were not significantly elevated in any patient group relative to the healthy control group, and no significant correlations were observed between peripheral hearing and neuropsychological task performance in either the combined patient or healthy control cohorts. Neuroanatomically, dichotic listening performance was associated with grey matter volume in a bilateral fronto-temporo-parietal network over the combined patient cohort, but no correlates were identified for pure-tone audiometry. Our findings highlight the importance of speech parsing mechanisms beyond elementary sound detection in driving cognitive test performance, underline the importance of assessing central hearing alongside peripheral hearing in people with dementia, and further delineate the complex auditory profiles of neurodegenerative dementias.

Anomaly Detection in Sound Activity with Generative Adversarial Network Models

In state-of-art anomaly detection research, prevailing methodologies predominantly employ Generative Adversarial Networks and Autoencoders for image-based applications. Despite the efficacy demonstrated in the visual domain, there remains a notable dearth of studies showcasing the application of these architectures in anomaly detection within the sound domain. This paper introduces tailored adaptations of cutting-edge architectures for anomaly detection in audio and conducts a comprehensive comparative analysis to substantiate the viability of this novel approach. The evaluation is performed on the DCASE 2020 dataset, encompassing over 180 hours of industrial machinery sound recordings. Our results indicate superior anomaly classification, with an average Area Under the Curve (AUC) of 88.16% and partial AUC of 78.05%, surpassing the performance of established baselines. This study not only extends the applicability of advanced architectures to the audio domain but also establishes their effectiveness in the challenging context of industrial sound anomaly detection.

A sound event detection support system for smart home based on “two-to-one” teacher–student learning

Sound event detection (SED) is a core technology in smart home projects that rely on detected sound events to trigger specific actions. SED systems face two major challenges: high labeling costs and complex acoustic environments. To reduce labeling costs, some semi-supervised systems extract both global and local features for classification. However, these methods treat global and local features equally, not accounting for their varying importance when recognizing different types of sound events. Furthermore, to address complex acoustic environments, some studies use multitask learning frameworks to introduce SED-related tasks as auxiliaries to improve detection performance. However, these methods fail to align tasks within the framework, leading to conflicting outputs that may limit system performance. To address these issues, in this paper we propose a “two-to-one” teacher-student learning based semi-supervised SED system. This system employs a gating mechanism to selectively enhance global and local features, improving adaptability to different types of sound events, and incorporates a cross-task alignment module to interact SED with related tasks, reducing the risk of performance degradation caused by conflicting outputs. Experimental results on two datasets demonstrate that our system achieves the best performance in all metrics, with EB-F1 scores of 48.1 % and 64.7 %, representing improvements of 15.3 % and 10.6 % over the baseline ConformerSED system, respectively. Our work offers an effective SED solution for smart home projects by providing a semi-supervised SED system that performs well while reducing labeling costs.

Multi-granularity acoustic information fusion for sound event detection

Most previous works on sound event detection (SED) are based on binary hard labels of sound events, leaving other scales of information underexplored. To address this problem, we introduce multiple granularities of knowledge into the system to perform hierarchical acoustic information fusion for SED. Specifically, we present an interactive dual-conformer (IDC) module to adaptively fuse the medium-grained and fine-grained acoustic information based on the hard and soft labels of sound events. In addition, we propose a scene-dependent mask estimator (SDME) module to extract the coarse-grained information from acoustic scenes, introducing the scene-event relationships into the SED system. Experimental results show that the proposed IDC and SDME modules efficiently fuse the acoustic information at different scales and therefore further improve the SED performance. The proposed system achieved Top 1 performance in DCASE 2023 Challenge Task 4B.

Fed-MStacking: Heterogeneous Federated Learning With Stacking Misaligned Labels for Abnormal Heart Sound Detection.

Ubiquitous sensing has been widely applied in smart healthcare, providing an opportunity for intelligent heart sound auscultation. However, smart devices contain sensitive information, raising user privacy concerns. To this end, federated learning (FL) has been adopted as an effective solution, enabling decentralised learning without data sharing, thus preserving data privacy in the Internet of Health Things (IoHT). Nevertheless, traditional FL requires the same architectural models to be trained across local clients and global servers, leading to a lack of model heterogeneity and client personalisation. For medical institutions with private data clients, this study proposes Fed-MStacking, a heterogeneous FL framework that incorporates a stacking ensemble learning strategy to support clients in building their own models. The secondary objective of this study is to address scenarios involving local clients with data characterised by inconsistent labelling. Specifically, the local client contains only one case type, and the data cannot be shared within or outside the institution. To train a global multi-class classifier, we aggregate missing class information from all clients at each institution and build meta-data, which then participates in FL training via a meta-learner. We apply the proposed framework to a multi-institutional heart sound database. The experiments utilise random forests (RFs), feedforward neural networks (FNNs), and convolutional neural networks (CNNs) as base classifiers. The results show that the heterogeneous stacking of local models performs better compared to homogeneous stacking.

Lung sound disease detection using attention over pre-trained efficientnet architecture

Lung sound disease detection using attention over pre-trained efficientnet architecture

Bragg grating etalon-based optical fiber for ultrasound and optoacoustic detection

Fiber-based interferometers receive significant interest as they lead to miniaturization of optoacoustic and ultrasound detectors without the quadratic loss of sensitivity common to piezoelectric elements. Nevertheless, in contrast to piezoelectric crystals, current fiber-based ultrasound detectors operate with narrow ultrasound bandwidth which limits the application range and spatial resolution achieved in imaging implementations. We port the concept of silicon waveguide etalon detection to optical fibers using a sub-acoustic reflection terminator to a Bragg grating embedded etalon resonator (EER), uniquely implementing direct and forward-looking access to incoming ultrasound waves. Precise fabrication of the terminator is achieved by continuously recording the EER spectrum during polishing and fitting the spectra to a theoretically calculated spectrum for the selected thickness. Characterization of the EER inventive design reveals a small aperture (10.1 µm) and an ultra-wide bandwidth (160 MHz) that outperforms other fiber resonators and enables an active detection area and overall form factor that is smaller by more than an order of magnitude over designs based on piezoelectric transducers. We discuss how the EER paves the way for the most adept fiber-based miniaturized sound detection today, circumventing the limitations of currently available designs.

Automatic detection of unidentified fish sounds: a comparison of traditional machine learning with deep learning

Automatic detection of unidentified fish sounds: a comparison of traditional machine learning with deep learning

Wnt7b acts in concert with Wnt5a to regulate tissue elongation and planar cell polarity via noncanonical Wnt signaling.

Intercellular signaling mediated by evolutionarily conserved planar cell polarity (PCP) proteins aligns cell polarity along the tissue plane and drives polarized cell behaviors during tissue morphogenesis. Accumulating evidence indicates that the vertebrate PCP pathway is regulated by noncanonical, β-catenin-independent Wnt signaling; however, the signaling components and mechanisms are incompletely understood. In the mouse hearing organ, both PCP and noncanonical Wnt (ncWnt) signaling are required in the developing auditory sensory epithelium to control cochlear duct elongation and planar polarity of resident sensory hair cells (HCs), including the shape and orientation of the stereociliary hair bundle essential for sound detection. We have recently discovered a Wnt/G-protein/PI3K pathway that coordinates HC planar polarity and intercellular PCP signaling. Here, we identify Wnt7b as a ncWnt ligand acting in concert with Wnt5a to promote tissue elongation in diverse developmental processes. In the cochlea, Wnt5a and Wnt7b are redundantly required for cochlear duct coiling and elongation, HC planar polarity, and asymmetric localization of core PCP proteins Fzd6 and Dvl2. Mechanistically, Wnt5a/Wnt7b-mediated ncWnt signaling promotes membrane recruitment of Daple, a nonreceptor guanine nucleotide exchange factor for Gαi, and activates PI3K/AKT and ERK signaling, which promote asymmetric Fzd6 localization. Thus, ncWnt and PCP signaling pathways have distinct mutant phenotypes and signaling components, suggesting that they act as separate, parallel pathways with nonoverlapping functions in cochlear morphogenesis. NcWnt signaling drives tissue elongation and reinforces intercellular PCP signaling by regulating the trafficking of PCP-specific Frizzled receptors.

A hybrid offline-online method for sound event localization and detection

A hybrid offline-online method for sound event localization and detection

A Study of Improved Two-Stage Dual-Conv Coordinate Attention Model for Sound Event Detection and Localization.

Sound Event Detection and Localization (SELD) is a comprehensive task that aims to solve the subtasks of Sound Event Detection (SED) and Sound Source Localization (SSL) simultaneously. The task of SELD lies in the need to solve both sound recognition and spatial localization problems, and different categories of sound events may overlap in time and space, making it more difficult for the model to distinguish between different events occurring at the same time and to locate the sound source. In this study, the Dual-conv Coordinate Attention Module (DCAM) combines dual convolutional blocks and Coordinate Attention, and based on this, the network architecture based on the two-stage strategy is improved to form the SELD-oriented Two-Stage Dual-conv Coordinate Attention Model (TDCAM) for SELD. TDCAM draws on the concepts of Visual Geometry Group (VGG) networks and Coordinate Attention to effectively capture critical local information by focusing on the coordinate space information of the feature map and dealing with the relationship between the feature map channels to enhance the feature selection capability of the model. To address the limitation of a single-layer Bi-directional Gated Recurrent Unit (Bi-GRU) in the two-stage network in terms of timing processing, we add to the structure of the two-layer Bi-GRU and introduce the data enhancement techniques of the frequency mask and time mask to improve the modeling and generalization ability of the model for timing features. Through experimental validation on the TAU Spatial Sound Events 2019 development dataset, our approach significantly improves the performance of SELD compared to the two-stage network baseline model. Furthermore, the effectiveness of DCAM and the two-layer Bi-GRU structure is confirmed by performing ablation experiments.

Study on sound wave kinematic characteristics and temperature sensing mechanism during the warming process of loose coals

Coal spontaneous combustion is a major threat for safe production. In order to achieve the application of sound wave technology to the monitoring and early warning of coal spontaneous combustion, clarify the propagation characteristics of sound waves in loose coal heating process, and illustrate the temperature sensing mechanism of the sound wave detection and early warning of coal spontaneous combustion, the present study built a test system of sound wave kinematic parameter for loose coals to survey the attenuation characteristics of sound wave propagation during loose coal heating. The elastic wave CT algorithm was used to reconstruct the sound wave velocity field, clarify the propagation path, and identify the temperature anomaly area. The results show that with the increase of temperature, the time for sound wave propagation reduces rapidly and then slowly, the velocity increases gradually, and the internal structure and pores of coals become enlarged. In the sound velocity field reconstructed on the basis of SIRT algorithm, the area with high value corresponds to the high temperature area of the coal, which better reflects the changes of the temperature anomaly area during coal heating.

Echoes from Sensory Entrainment in Auditory Working Memory for Pitch.

Ongoing neural oscillations reflect cycles of excitation and inhibition in local neural populations, with individual neurons being more or less likely to fire depending upon the oscillatory phase. As a result, the oscillations could determine whether or not a sound is perceived and/or whether its neural representation enters into later processing stages. While empirical support for this idea has come from sound detection studies, large gaps in knowledge still exist regarding memory for sound events. In the current study, it was investigated how sensory entrainment impacts the fidelity of working memory representations for pitch. In two separate experiments, an 8 Hz amplitude modulated (AM) entraining stimulus was presented prior to a multitone complex having an f0 between 270 and 715 Hz. This "target" sound could be presented at phases from 0 to 2π radians in relation to the previous AM. After a retention interval of 4 s (Experiment 1; n = 26) or 2 s (Experiment 2; n = 28), listeners were tasked to reproduce the target sound's pitch by moving their finger along the horizontal axis of a response pad. It was hypothesized that if entrainment modulates auditory working memory fidelity, reproductions of a target's pitch would be more accurate and precise when targets were presented in phase with the entrainment. Cosine fits of the average data for both experiments showed a significant entrainment "echo" in the accuracy of pitch matches. There was no apparent echo in the matching precision. Fitting of the individual data accuracy showed that the optimal phase was consistent across individuals, aligning near the next AM peak had the AM continued. The results show that sensory entrainment modulates auditory working memory in addition to stimulus detection, consistent with the proposal that ongoing neural oscillatory activity modulates higher-order auditory processes.

Transformer and Graph Convolution-Based Unsupervised Detection of Machine Anomalous Sound Under Domain Shifts

Transformer and Graph Convolution-Based Unsupervised Detection of Machine Anomalous Sound Under Domain Shifts

Machine anomalous sound detection based on audio synthesis generative adversarial network

Abstract In the pursuit of implementing safety monitoring within Industry 4.0 smart production, deep learning methodologies are employed to harness the audio signals emanating from the operation of machinery and equipment, facilitating anomaly detection. Aiming to address the challenge of the few-shot samples for anomalous sound detection in an unsupervised setting, this paper introduces a machine anomalous sound detection method using an Audio Synthesis Adversarial Generative Network (ASGAN). The augmentation of classifiers within the network’s discriminator is proposed to execute a multi-classification task, thereby transforming the unsupervised training milieu into a semi-supervised one and enhancing the discriminator’s accuracy. To address the oversight of audio frequency domain features by conventional generative adversarial networks, the squeeze-and-excitation network is applied to concentrate on crucial frequency bands within the frequency domain. Experimental validation is conducted by using the DCASE Challenge Task2 dataset, revealing that the performance of the anomaly detection method leveraging audio synthesis generative adversarial networks surpasses that of the baseline system.

Adaptive iterative transfer learning for effective snapping shrimp sound detection.

This study aims to detect the bioacoustics signal in the underwater soundscape, specifically those produced by snapping shrimp, using adaptive iterative transfer learning. The proposed network is initially trained with pre-classified snapping shrimp sounds and Gaussian noise, then applied to classify and remove snapping-free noise from field data. This separated ambient noise is subsequently used for transfer learning. This process was iterated to distinguish more effectively between ambient noise and snapping shrimp sounds characteristics, resulting in improved classification. Through iterative transfer learning, significant improvements in precision and recall were observed. The application to field data confirmed that the trained network could detect signals that were difficult to identify using existing threshold classification methods. Furthermore, it was found that the rate of false detection decreased, and detection probability improved with each stage. This research demonstrates that incorporating the noise characteristics of field data into the trained network via iterative transfer learning can generate more realistic training data. The proposed network can successfully detect signals that are challenging to identify using existing threshold classification methods.

Smart skin of underwater vehicle with dual functions of sound detection and absorption

The rapid development of underwater vehicles has posed the pressing and urgent requirement for smart underwater skin. Developing the multi-function underwater skin for sound detection and absorption is of significance for environment investigation and stable operation of underwater vehicles. Herein, a novel underwater acoustic skin with dual functions of sensitive sound detection and effective sound absorption under high hydrostatic pressure was reported. With addition of piezoelectric multiple yolk–shell particles and carbon nanotube network, the smart skin exhibited excellent sound absorption performance and enhanced mechanical property, which retained a high sound coefficient under pressure of 3 MPa compared to the resin skin. The enhanced skin with piezoelectric particles also exhibited sensitive sound detection capability with admirable directivity. The results proved the broad application prospect of the acoustic skin based on piezoelectric multiple yolk–shell particles and conductive network in the underwater extreme environment.