Sound Detection Research Articles

Deafness due to loss of a TRPV channel eliminates mating behavior in Aedes aegypti males

Attraction and mating between male and female animals depend on effective communication between conspecifics. However, in mosquitoes, we have only a rudimentary understanding of the sensory cues and receptors critical for the communication that is essential for reproductive behavior. While it is known that male Aedes aegypti use sound to help them identify females, it is not unclear whether sound detection is absolutely required since other cues such as vision may also participate in mating behavior. To determine the effect of eliminating hearing on mating success, we knocked out the Ae. aegypti TRPVa channel, which is a protein expressed in chordotonal neurons in the Johnston's organ (JO) that respond to sound-induced movements in the antenna. Loss of trpVa eradicated sound-induced responses from the JO, thereby abolishing hearing. Strikingly, mutation of trpVa eliminated mating behavior in males. In contrast, trpVa-null females mated, although this behavior was slightly delayed relative to wild-type females. Males and females produce sounds as they beat their wings at distinct frequencies during flight. Sound mimicking the female wingbeat induced flight, attraction, and copulatory-like behavior in wild-type males without females present, but not in trpVa-null males. Males are known to modulate their wingbeat frequencies before mating in the air, which is a phenomenon referred to as rapid frequency modulation (RFM). We found that RFM was absent in mosquitoes lacking TRPVa. We conclude that the requirement for trpVa and hearing for male reproductive behavior in Aedes is absolute, as mating in the deaf males is eliminated.

Development of explainable machine intelligence models for heart sound abnormality detection

Development of explainable machine intelligence models for heart sound abnormality detection

Low-Complexity Attention-Based Unsupervised Anomalous Sound Detection Exploiting Separable Convolutions and Angular Loss

Low-Complexity Attention-Based Unsupervised Anomalous Sound Detection Exploiting Separable Convolutions and Angular Loss

HADNet: A Novel Lightweight Approach for Abnormal Sound Detection on Highway Based on 1D Convolutional Neural Network and Multi-Head Self-Attention Mechanism

Video surveillance is an effective tool for traffic management and safety, but it may face challenges in extreme weather, low visibility, areas outside the monitoring field of view, or during nighttime conditions. Therefore, abnormal sound detection is used in traffic management and safety as an auxiliary tool to complement video surveillance. In this paper, a novel lightweight method for abnormal sound detection based on 1D CNN and Multi-Head Self-Attention Mechanism on the embedded system is proposed, which is named HADNet. First, 1D CNN is employed for local feature extraction, which minimizes information loss from the audio signal during time-frequency conversion and reduces computational complexity. Second, the proposed block based on Multi-Head Self-Attention Mechanism not only effectively mitigates the issue of disappearing gradients, but also enhances detection accuracy. Finally, the joint loss function is employed to detect abnormal audio. This choice helps address issues related to unbalanced training data and class overlap, thereby improving model performance on imbalanced datasets. The proposed HADNet method was evaluated on the MIVIA Road Events and UrbanSound8K datasets. The results demonstrate that the proposed method for abnormal audio detection on embedded systems achieves high accuracy of 99.6% and an efficient detection time of 0.06 s. This approach proves to be robust and suitable for practical applications in traffic management and safety. By addressing the challenges posed by traditional video surveillance methods, HADNet offers a valuable and complementary solution for enhancing safety measures in diverse traffic conditions.

Child Abuse Incidences Per Child Year Predicted from the Available Average Incidence Combined with Foreign Incidences Per Child Year: Towards a New Policy

Child abuse is a worldwide recognized serious problem. Reliable child abuse incidences, preferably per child year, are fundamental for a sound detection and prevention program. Unfortunately, in most countries where child abuse data is available, incidences are not determined per child year but as an average over the child age range. In this paper we suggest a possible "next-best" solution for deriving child abuse incidences per child year when only an average value is available in an area or country. As method, we combined the country's measured average incidence with available (foreign) incidences per child year. The country's next-best incidences per child year will be estimated from its average, multiplied by the foreign incidences per child year divided by the foreign average. As results, we calculated the next-best Dutch age-dependent incidences by combining the Dutch average value with US and Ontario age-related incidences. We found comparable results for infants above 1 year and marked differences for children <1 year, likely due to cultural differences between the US and Ontario. In conclusion, next-best age-related child abuse incidences are obtainable in large areas or countries by choosing a smaller but representative region, the latter estimated from Ontario-data as ≥210,000 inhabitants, and establishing as perfectly as possible the optimal infra structure. A future perspective towards a new policy could be to initiate and stimulate this approach in the various European Union and United Nations Convention on the Rights of the Child member states.

Utilization of Detection of Non-Speech Sound for Sustainable Quality of Life for Deaf and Hearing-Impaired People: A Systematic Literature Review

Several assistive technologies (ATs) have been manufactured and tested to alleviate the challenges of deaf or hearing-impaired people (DHI). One such technology is sound detection, which has the potential to enhance the experiences of DHI individuals and provide them with new opportunities. However, there is a lack of sufficient research on using sound detection as an assistive technology, specifically for DHI individuals. This systematic literature review (SLR) aims to shed light on the application of non-verbal sound detection technology in skill development for DHI individuals. This SLR encompassed recent, high-quality studies from the prestigious databases of IEEE, ScienceDirect, Scopus, and Web of Science from 2014 to 2023. Twenty-six articles that met the eligibility criteria were carefully analyzed and synthesized. The findings of this study underscore the significance of utilizing sound detection technology to aid DHI individuals in achieving independence, access to information, and safety. It is recommended that additional studies be conducted to explore the use of sound detection tools as assistive technology, to enhance DHI individual’s sustainable quality of life.

An acoustic nanogenerator based on porous PDMS/ZnO/PVDF-TrFE structure for self-powered intermediated-frequency sound monitoring

Abstract Sound energy is widely present in natural environment, nowadays predominantly collected using voluminous and heavy rigid devices, limiting their portability. Till now, there are few reports about using flexible materials to collect sound energy, and it is usually susceptible to environmental humidity. This study combines flexible polymer polydimethylsiloxane (PDMS), piezoelectric material polyvinylidene difluoride trifluoroethylene (PVDF-TrFE), nano zinc oxide (ZnO), and conductive carbon nanotubes (CNTs) to fabricate a porous nano ZnO/PVDF-TrFE/CNTs PDMS sound-driven nanogenerator (PZPCP SNG). It amalgamates frictional and piezoelectric effects: its porous structure enables efficient collection of vibration and frictional energy generated by sound waves; the piezoelectric effect of PVDF-TrFE and nano ZnO facilitates the conversion of acoustic mechanical energy, collectively enhancing the generator's output. Experimental optimization yields the best production conditions, achieving optimal outputs of 481.1 mV (open-circuit voltage), 209.13 nA (short-circuit current) under 400 Hz/125 dB sound stimuli, with a surface power density of 9.1 µW/m² (volumetric power density of 2.28 mW/m³). And it can convert sound ranging from 63-3000 Hz. With hardware circuitry, up to 5 series-connected light-emitting diodes (LEDs) can be illuminated within the circuit, demonstrating a certain degree of sound recognition capability. The proposed PZPCP SNG design is simple, effective, and lightweight, enabling flexible and stable intermediated-frequency acoustic energy harvesting. Its hydrophobic structural characteristics render it adaptable to various humidity, presenting a new approach toward widespread low-power sound detection and self-powering applications.

Improving bird abundance estimates in harvested forests with retention by limiting detection radius through sound truncation

Abstract An inherent challenge with acoustically surveying birds is that the distance at which they can be detected depends on how far their song can be heard. We developed a distance-based sound detection space truncation method to correct for variable sampling radii due to surveying in forested or open conditions. The method was pivotal in evaluating bird responses to retention patches; without this methodological advancement, the impact of retention patches on songbird abundance was vastly underestimated. In the boreal forest, these patches of live trees are retained in regenerating harvested forests to provide ecological services for species adapted to natural disturbances. Although we did not verify our a priori assumption with ground observations, our findings suggest that limited-distance sampling better captures the effects of retention patches on bird use of harvested forests. When evaluated using unlimited distance surveys, retained trees had a negligible effect on bird abundance, whereas applying detection distance truncation highlighted the importance of retention on forest birds. We found that early to mid-seral forest songbirds benefited from retention patches, with notable increases in abundance after 10 years of regeneration. The size of retention patches, ranging from 0.1 to 1.2 hectares, did not have a linear relationship with bird abundance. Instead, edge effects stemming from the configuration of these patches emerged as key determinants of abundance for the majority of the species studied. Retention patches that were nearest to unharvested forests were used the most, compared to further into harvest areas. Our research not only highlights the underestimated impact of small-scale live tree retention on forest songbirds but also introduces a significant methodological innovation in the field of acoustic monitoring.

Enhancing Acoustic Data Mobilization: A User-Friendly Data Platform for the Asian Soundscape Monitoring Network

Passive Acoustic Monitoring (PAM) has emerged as a crucial tool in monitoring efforts to track environmental changes and evaluate conservation measures in response to the biodiversity crisis (Sugai et al. 2018). PAM now offers long-term and continuous insights into biodiversity, using acoustic indices correlating with biodiversity and deep-learning tools to aid in the detection and identification of animal sounds (Sueur et al. 2014, Kahl et al. 2021, Wu et al. 2022). However, managing and mobilizing original recordings remains challenging due to their large size and lack of structure, hindering broader applications of these valuable data. Established in 2014 by Academia Sinica and Taiwan Forestry Research Institute, the Asian Soundscape Monitoring Network has amassed over 20 million minutes of audible and ultrasonic recordings from diverse landscapes, including forests, wetlands, urban parks and farmlands across Malaysia, Thailand, Taiwan and Vietnam. To enhance data mobilization and utilization, we developed a user-friendly data platform enabling browsing, searching, visualization, exploration and retrieval of the original recordings (Fig. 1), as well as derived data such as acoustic indices and species occurrences, and associated weather records (Fig. 2) The platform also allows users to visualize the temporal dynamics of the soundscape and identify acoustic events at monitoring sites by examining long-term spectrograms and time series of acoustic indices (Fig. 3). Detailed information about the monitoring sites and recorder deployment is provided. Each recording is tagged with a Creative Commons license, and a unique Archival Resource Key is assigned to every data retrieval for persistent identification, facilitating data reuse. Additionally, recordings containing human voices are identified and restricted to protect privacy. Our aim with this data platform is to streamline the mobilization of acoustic data, foster diverse applications, and enhance the overall value of the data.

Study on frequency dependent convolution methods for sound event detection

Sound event detection (SED) aims to classify target sound events from a given audio clip with the corresponding onset and offset time localization. SED, a core research field in machine listening, has been rapidly growing by adopting the methods by applying deep learning methods in image recognition fields. One representative example is 2D convolution which is applied to 2D time-frequency audio features. However, 2D audio data exhibits physical discrepancies from 2D image data, one of which is that frequency dimension of 2D audio data is translational-variant while position dimension of 2D image data is translational-invariant. Therefore, to make 2D convolution more consistent to 2D audio data, we need to make 2D convolution dependent to frequency dimension. Previously, to address this problem, Frequency Dynamic Convolution (FDYConv) was proposed to replaces 2D convolution. FDYConv is dependent on frequency dimensions since it applies different kernels on frequency dimension. While it has proven its outstanding performance on SED, there are several more alternative frequency dependent convolution methods which are yet to be explored, In this work, we propose other simpler frequency dependent convolution methods and compare them with FDYConv in order to further investigate the effect of frequency dependence of 2D convolution methods.

Partial equivalent sound pressure level as an approach to manage irrelevant sounds in environmental noise measurements

Residual sound and unwanted events are known factors affecting the accuracy of environmental noise measurements. ISO 1996-2 provides methods to manage these irrelevant sounds, for example by applying correction factors, but the methods require a degree of knowledge of the irrelevant sounds that is not always practical or possible to obtain when performing long-term measurements with unattended monitoring stations in complex urban soundscapes. While automatic detection of irrelevant sounds provides the required information, it also allows for new approaches not described in ISO 1996-2. In this paper, we discuss a metric called partial equivalent sound pressure level (Partial Leq), calculated after data samples with residual sound and unwanted events are identified automatically and replaced by project-specific values. Our hypothesis is that better estimates of equivalent sound pressure levels (Leq) can be achieved with this metric, compared with the general methods given in ISO 1996-2. Comparisons of results are presented, together with a discussion of applicability of the Partial Leq metric and experiences gathered from its use.

Integrating Mixup and ArcFace for enhanced anomalous sound detection

In machine condition monitoring, accurate detection of anomalous sounds is essential for timely maintenance interventions. Traditional deep neural network models employ classification-based self-supervised learning to discriminate normal and abnormal operational sounds using classification confidence. However, these models often suffer from overfitting or excessive confidence, limiting their effectiveness. To overcome this, we introduce a hybrid technique that combines the strengths of Mixup and ArcFace techniques. Mixup enhances model generalization by training on mixed sound data, while ArcFace promotes greater inter-class distances with an angular margin loss. However, there is a challenge in combining these methods because the angular margin for ArcFace is defined for a single target class, and hence, can be hardly defined for the mixed data utilized in Mixup. The proposed technique employs the class-dependent angular margins based on the mixing ratio of data. SoftMax losses for individual classes are constructed using the class-dependent margins and then integrated into the Mixup loss. When applied to the DCASE2020 Task2 dataset, the proposed method significantly outperforms the state-of-the-art NoisyArcMix model in anomaly detection. This approach demonstrates the potential of leveraging class-dependent angular margins in mixed data for improved performance in machine condition monitoring.

Designing a practical physical framework for AI-enhanced sound detection and localization in vehicles

This paper presents the practical framework of an AI-implemented system currently in development for the detection and localization of external sound events to the vehicle. We delve into the initial block of the automotive project, analyzing sounds of interest and the noise we encounter, addressing the strategic arrangement of the four microphones on the sides of the vehicle, conceived as the sound input system. This configuration not only optimizes signal capture but also acts as sensitive ears, allowing the system to effectively analyze the acoustic environment. To mitigate wind interference, we have designed a microphone placement system that considers the vehicle's aerodynamics. The strategic location and implementation of aerodynamic protectors significantly minimize the impact of wind on the microphones, ensuring a sharper capture of acoustic signals. Complementing this arrangement, we apply advanced noise elimination processes, further enhancing the quality of the captured signals. Although in its early stages, this project aspires to provide effective solutions for vehicular acoustic perception, marking the beginning of a journey towards innovation in the detection and localization of sounds in mobile environments.

Sound event detection of passing trains using convolutional recurrent neural network (CRNN)

Acoustic measurements near railway infrastructure typically involve precise identification and labelling of train passages in time series of noise measurements. The labelling of events is done manually by an acoustician, by cross-referencing several sources of information, such as train timetable data, audio data, images and/or videos. In this paper, we present a method which automatically detects and labels the passage of trains in front of a microphone using a convolutional recurrent neural network (CRNN). The advantage of this type of model is that it can be trained using existing data (time series of LAeq,1s and 1/3 octave band with a good labelling of passing trains); which has been widely available for a number of years. The results are promising if limited to binary classification but would require further development if the aim included the recognition and categorisation of various train types.

Development of an experimental technique for measuring polar diagrams of adaptive directivity implemented in hearing aids

Nowadays, hearing aids have efficient signal processing options to improve speech intelligibility in noise. This work concerns the characterization of the microphone systems directivity available in hearing aids. In recent decades, "adaptive" directivities have been implemented in hearing aids, thus making it possible to focus on useful sound sources in the hearing aid patient environment. These options use the principles of sound environments detection, acoustic beamforming and noise reduction. Performances of these options can be estimated with subjective methods (as speech audiometry in noise), and objective methods (listening effort, intelligibility indicators as HASPI, SII, STI, etc.). But these methods provide an overall assessment of all the processing involved in speech enhancement and are not designed to characterize the effects of adaptive directivity. We developed an experimental technique based on sound sources separation with phase opposition diffusion (known as "Hagerman" method) to draw polar diagrams of adaptive directivity, for realistic sound environments. We used a sound multi-diffusion system to generate various sound environments with noise and speech sources around an artificial head equipped with hearing aids. The first results show the adaptive directivity effects associated with noise reduction provided by the latest hearing aids generation on everyday hearing scenes.

Enhanced railway sound event detection using YAMNet and classification criteria

The segmentation-by-classification method has become a popular way to detect sound events. It uses neural networks, trained to detect sound sources on short signals (one tenth of a second to a second), or long-term signals divided into a succession of short segments. In this paper, we focus on detecting train pass-bys from long term signals, where it is assumed that railway noise is higher than ambient noise. Using the neural network YAMNet, we show that a processing stage is necessary to improve the segmentation of such events and reduce the high false positive rate. The applied criteria are related to the nature of a train pass-by, lasting several seconds and with broad frequency band. Around 90% of events are detected with proper boundaries. However, we observe that YAMNet is not designed to distinguish railway vehicles from other vehicles. False positive rate remains high whenever other vehicles travel near the measurement location. Additional AI models are proposed to perform this specific distinction. Their efficiency depends on train type, as recent passenger train noise resembles roadway noise. Otherwise, false positive rate decreases below 10%, and the railway noise contribution estimate aligns with the reference level, within a 0.5 dB(A) margin on average.

Semi-supervised sound event detection with dynamic convolution and confidence-aware mean teacher

Recently, sound event detection (SED) has made significant advancements through the application of deep learning, but there are still many difficulties and challenges to be addressed. One of the major challenges is the diversity of sound events, leading to substantial variations in time-frequency domain features. Additionally, most existing SED models can not effectively handle sound events of different scales, particularly those of short duration. Another challenge is the lack of well labeled dataset. The commonly used solution is mean teacher method, but inaccurate pseudo-labels could lead to confirmation bias and performance imbalance. In this paper, we introduce the multi-dimensional frequency dynamic convolution, which endows convolutional kernels with frequency-adaptive dynamic properties to enhance the feature representation capability. Moreover, we propose dual self attention pooling function to achieve more precise temporal localization. Finally, to solve the incorrect pseudo-labels problems, we propose the confidence-aware mean teacher to increase pseudo-labels accuracy and train the student model with high confidence labels. Experimental results on DCASE2017, DCASE2018 and DCASE2023 Task4 dataset validate the superior performance of proposed methods.

Impulsive pile driving sound does not induce hearing loss in the longfin squid (Doryteuthis pealeii)a).

Offshore windfarms are a key means to produce clean energy as we seek to limit climate change effects. Impulsive pile driving used for their construction in shallow water environments is among the most intense anthropogenic sound sources. There is an increasing understanding that an array of marine invertebrates detects acoustic cues, yet little is known about how pile driving sound could impact their sound detection abilities. We experimentally quantified potential changes in sound sensitivity for an abundant, commercially and ecologically important squid species (Doryteuthis pealeii) exposed to actual in situ pile driving. The pile was 0.3-m diameter and 10-m long; hammer energy reached 16 kJ per strike. Sound detection thresholds were determined using auditory evoked potentials in animals with no exposure, after one 15-min or five repeated 15-min long pile driving sound sequences, corresponding to cumulative sound exposure levels of 110 and 131 dB re (1 μm s-2)2 s for acceleration and 187 and 214 dB re (1 μPa)2 s for pressure. We found no statistical evidence of temporary threshold shifts in any squid exposed to pile driving sound sequences. These results, combined with companion behavioral studies, suggest that squid may be robust to the sound impacts during offshore windfarm construction.

Heart Sound Abnormality Detection From Multi-Institutional Collaboration: Introducing a Federated Learning Framework.

Early diagnosis of cardiovascular diseases is a crucial task in medical practice. With the application of computer audition in the healthcare field, artificial intelligence (AI) has been applied to clinical non-invasive intelligent auscultation of heart sounds to provide rapid and effective pre-screening. However, AI models generally require large amounts of data which may cause privacy issues. Unfortunately, it is difficult to collect large amounts of healthcare data from a single centre. In this study, we propose federated learning (FL) optimisation strategies for the practical application in multi-centre institutional heart sound databases. The horizontal FL is mainly employed to tackle the privacy problem by aligning the feature spaces of FL participating institutions without information leakage. In addition, techniques based on deep learning have poor interpretability due to their "black-box" property, which limits the feasibility of AI in real medical data. To this end, vertical FL is utilised to address the issues of model interpretability and data scarcity. Experimental results demonstrate that, the proposed FL framework can achieve good performance for heart sound abnormality detection by taking the personal privacy protection into account. Moreover, using the federated feature space is beneficial to balance the interpretability of the vertical FL and the privacy of the data. This work realises the potential of FL from research to clinical practice, and is expected to have extensive application in the federated smart medical system.

Online real-time machining chatter sound detection using convolutional neural network by adopting expert knowledge

In machining processes, chatter detection remains a pivotal challenge, impacting both the quality and efficiency of manufacturing operations. This study introduces an approach that synergizes expert knowledge with the capabilities of advanced convolutional neural networks (CNNs) to enhance chatter detection. A comprehensive monitoring framework is proposed to adopt expert knowledge that digitizes machine tool and sound data, effectively labeling chatter events. This study merges human expertise in identifying milling tool chatter sounds with CNN architecture, marking a notable advancement in blending machining insights of experts with modern artificial intelligence (AI) technologies. The proposed chatter prediction architecture is distinguished by its incorporation of an attention block, fusing outputs from the AlexNet model with cutting parameters. This model outshines baseline models in both in-distribution (ID) and out-of-distribution (OOD) testing datasets. In OOD testing, the proposed model achieved an impressive accuracy of 94.51%, markedly surpassing the standalone CNN model’s accuracy of 88.66%. Real-time 3D visualization of machining operations is demonstrated through the successful implementation of the trained model on a Raspberry Pi.