Complex Acoustic Environments Research Articles

Cortical Processing of Conspecific Vocalizations in Zebra Finches Depends on the Early Acoustical Environment.

Sensory experience during development has lasting effects on perception and neural processing. Exposing animals to artificial stimuli early in life influences the tuning and functional organization of the auditory cortex, but less is known about how the rich acoustical environments experienced by vocal communicators affect the processing of complex vocalizations. Here, we show that in zebra finches ( Taeniopygia guttata ), a colonial-breeding songbird species, exposure to a naturalistic social-acoustical environment during development has a profound impact on cortical-level auditory responses to conspecific song. Compared to birds raised by pairs in acoustic isolation, birds raised in a breeding colony had higher average firing rates, selectivity, and discriminability, especially in the narrow-spiking, putatively inhibitory neurons of a higher-order auditory area, the caudomedial nidopallium (NCM). Neurons in colony-reared birds were also less correlated in their tuning and more efficient at encoding the spectrotemporal structure of conspecific song. These results suggest that the auditory cortex adapts to noisy, complex acoustical environments by strengthening inhibitory circuitry, functionally decoupling excitatory neurons while maintaining overall excitatory-inhibitory balance.

A novel transformer-enhanced and acoustic-based approach for wind turbine blade fault detection with integrated system implementation

Rising global wind power capacity urgently requires effective real-time turbine monitoring. Complex installations, high maintenance costs, and the occasional need for turbine shutdown hinder conventional monitoring methods. Acoustic-based techniques, while cost-efficient for health assessments, currently face challenges with detection precision and signal processing capabilities in complex acoustic environments. To overcome these challenges, this study proposes a low-cost, AI-driven acoustic-based health monitoring system framework for wind turbine blades, featuring enhanced detection accuracy and signal processing capabilities in complex sound environments. Firstly, a microphone array is employed to capture blade sounds with rich spatial attributes, while beamforming algorithms with a fixed orientation integrate prior spatial information. Furthermore, to further strengthen signal processing and fault detection capabilities, the Transformer model based on self-attention mechanisms is employed for feature extraction and fault diagnosis. An actual wind turbine blade health monitoring system is designed and developed to validate the usability of the proposed framework. Experimental results based on a scaled-down wind turbine model validate the proposed algorithm's effectiveness and practicality, presenting an effective approach in the wind turbine blade health monitoring domain.

Comparison of 2D and 3D multichannel audio rendering methods for hearing research applications using technical and perceptual measures

Virtual reality with multichannel audio playback is increasingly used in hearing aid research. The purpose of this study is to compare horizontal (2D) and periphonic (3D) rendering methods in terms of localization, minimum audible angle, and perceptual ratings related to spatial quality. Higher Order Ambisonics, Vector-Base Amplitude Panning, and Nearest Speaker Selection were used, with playback through 16, 29 and 45 speakers. The results show that an improvement in vertical localization can be obtained by using periphonic rendering instead of horizontal rendering. The perceptual advantage of periphonic rendering depends on the spatial complexity of the scene; it disappears in complex acoustic environments. Scenes with low acoustic complexity, such as a single primary sound source in a room, benefit from Nearest Speaker Selection rendering. For more complex scenes with multiple sound sources, such as a symphony orchestra in a concert hall with many primary sources, or traffic on a road with moving sources, horizontal rendering methods such as 2D Higher Order Ambisonics will provide similar or better performance.

Development of an adaptive test of musical scene analysis abilities for normal-hearing and hearing-impaired listeners.

Auditory scene analysis (ASA) is the process through which the auditory system makes sense of complex acoustic environments by organising sound mixtures into meaningful events and streams. Although music psychology has acknowledged the fundamental role of ASA in shaping music perception, no efficient test to quantify listeners' ASA abilities in realistic musical scenarios has yet been published. This study presents a new tool for testing ASA abilities in the context of music, suitable for both normal-hearing (NH) and hearing-impaired (HI) individuals: the adaptive Musical Scene Analysis (MSA) test. The test uses a simple 'yes-no' task paradigm to determine whether the sound from a single target instrument is heard in a mixture of popular music. During the online calibration phase, 525 NH and 131 HI listeners were recruited. The level ratio between the target instrument and the mixture, choice of target instrument, and number of instruments in the mixture were found to be important factors affecting item difficulty, whereas the influence of the stereo width (induced by inter-aural level differences) only had a minor effect. Based on a Bayesian logistic mixed-effects model, an adaptive version of the MSA test was developed. In a subsequent validation experiment with 74 listeners (20 HI), MSA scores showed acceptable test-retest reliability and moderate correlations with other music-related tests, pure-tone-average audiograms, age, musical sophistication, and working memory capacities. The MSA test is a user-friendly and efficient open-source tool for evaluating musical ASA abilities and is suitable for profiling the effects of hearing impairment on music perception.

A novel audio-visual based beamformer using a 360° camera and planar microphone array system

Beamforming systems that rely only on microphone arrays may suffer from poor performance in complex acoustic environments with multiple sources and room reverberation. To overcome this limitation, we propose a novel method that combines a planar microphone array and a 360° camera to steer the beamformer. The camera provides visual information about the location and orientation of sound sources, which can be used to adjust the beamformer parameters. We will present results that compare the signal-to-noise ratio (SNR) of the proposed method with a conventional beamforming microphone-based method in a simulated room with different source positions, orientations, and noise levels to investigate the benefit of visual information for beamforming in different environments. The proposed method has the potential to improve the performance of beamforming systems in applications such as speech enhancement, speech recognition, and speaker localization.

Subcortical measures of multi-talker speech processing

Communication in everyday acoustic environments is facilitated by the ability to locate, separate, and process speech from different locations. We do this by combining information received by both ears. Typically, the auditory system does not simply relay information delivered by both ears; rather, bilateral information is combined through elaborate binaural pathways in both the brainstem and cortex to allow us to communicate in complex acoustic environments. But traditionally it has been difficult to evaluate the subcortical underpinnings of speech communication using more complex, naturalistic continuous speech stimuli. Recently, the “peaky” speech method was created that uses stories to evoke brainstem responses using EEG. We have recorded brainstem responses from adults with normal hearing who listen to one or multiple narrators, and narrators coming from one or different perceived spatial locations. This talk will highlight the brainstem work undertaken towards creating subcortical measures of speech processing and spatial hearing.

A binaural room impulse response dataset and Shorelining psychophysical task for the evaluation of auditory sensory augmentation

Sensory augmentation using spatial sound presented in augmented-reality (AR) can assist people with low vision or blindness in navigating their environment (Katz et al., 2012). Nonetheless, in many situations, the poor quality of the binaural sound rendered using current tool sets limits the potential capability of the assistive technology. In particular, acoustic environments with near-field sources and reflections pose significant challenges. In this work, we provide a reference binaural room impulse response (BRIR) dataset with near-field sources and reflections and an associated shorelining psychophysical task that is useful for the evaluation of AR spatial audio. The dataset consists of 12∼small loudspeakers arranged on a 3-by-4 grid in a complex acoustic environment. BRIR measurements are recorded using the Head and Torso Simulator (HATS) for 17∼different receiver positions with a 5o angular resolution. Room impulse response measurements are also recorded using the Eigenmike for each of the 17 receiver positions. Using the Razer Anzu smart glasses to render the binaural AR spatial audio, we compare psychophysical performance on the shorelining navigation task using the recorded dataset and various existing binaural AR tool sets.

The relationship between extended high-frequency hearing and the binaural spatial advantage in young to middle-aged firefightersa).

Relationships between extended high-frequency (EHF) thresholds and speech-in-spatialized noise were examined in firefighters with a history of occupational noise and airborne toxin exposure. Speech recognition thresholds were measured for co-located and spatially separated (±90° azimuth) sentences in a competing signal using the Listening in Spatialized Noise-Sentences test. EHF hearing was significantly correlated with the spatial advantage, indicating that firefighters with poorer EHF thresholds experienced less benefit from spatial separation. The correlation between EHF thresholds and spatial hearing remained significant after controlling for age. Deficits in EHF and spatial hearing suggest firefighters may experience compromised speech understanding in job-related complex acoustic environments.

Neural dynamics underlying successful auditory short-term memory performance.

Listeners often operate in complex acoustic environments, consisting of many concurrent sounds. Accurately encoding and maintaining such auditory objects in short-term memory is crucial for communication and scene analysis. Yet, the neural underpinnings of successful auditory short-term memory (ASTM) performance are currently not well understood. To elucidate this issue, we presented a novel, challenging auditory delayed match-to-sample task while recording MEG. Human participants listened to 'scenes' comprising three concurrent tone pip streams. The task was to indicate, after a delay, whether a probe stream was present in the just-heard scene. We present three key findings: First, behavioural performance revealed faster responses in correct versus incorrect trials as well as in 'probe present' versus 'probe absent' trials, consistent with ASTM search. Second, successful compared with unsuccessful ASTM performance was associated with a significant enhancement of event-related fields and oscillatory activity in the theta, alpha and beta frequency ranges. This extends previous findings of an overall increase of persistent activity during short-term memory performance. Third, using distributed source modelling, we found these effects to be confined mostly to sensory areas during encoding, presumably related to ASTM contents per se. Parietal and frontal sources then became relevant during the maintenance stage, indicating that effective STM operation also relies on ongoing inhibitory processes suppressing task-irrelevant information. In summary, our results deliver a detailed account of the neural patterns that differentiate successful from unsuccessful ASTM performance in the context of a complex, multi-object auditory scene.

Rapid, Activity-Dependent Intrinsic Plasticity in the Developing Zebra Finch Auditory Cortex.

The acoustic environment an animal experiences early in life shapes the structure and function of its auditory system. This process of experience-dependent development is thought to be primarily orchestrated by potentiation and depression of synapses, but plasticity of intrinsic voltage dynamics may also contribute. Here, we show that in juvenile male and female zebra finches, neurons in a cortical-level auditory area, the caudal mesopallium (CM), can rapidly change their firing dynamics. This plasticity was only observed in birds that were reared in a complex acoustic and social environment, which also caused increased expression of the low-threshold potassium channel Kv1.1 in the plasma membrane and endoplasmic reticulum (ER). Intrinsic plasticity depended on activity, was reversed by blocking low-threshold potassium currents, and was prevented by blocking intracellular calcium signaling. Taken together, these results suggest that Kv1.1 is rapidly mobilized to the plasma membrane by activity-dependent elevation of intracellular calcium. This produces a shift in the excitability and temporal integration of CM neurons that may be permissive for auditory learning in complex acoustic environments during a crucial period for the development of vocal perception and production.SIGNIFICANCE STATEMENT Neurons can change not only the strength of their connections to other neurons, but also how they integrate synaptic currents to produce patterns of action potentials. In contrast to synaptic plasticity, the mechanisms and functional roles of intrinisic plasticity remain poorly understood. We found that neurons in the zebra finch auditory cortex can rapidly shift their spiking dynamics within a few minutes in response to intracellular stimulation. This plasticity involves increased conductance of a low-threshold potassium current associated with the Kv1.1 channel, but it only occurs in birds reared in a rich acoustic environment. Thus, auditory experience regulates a mechanism of neural plasticity that allows neurons to rapidly adapt their firing dynamics to stimulation.

Interspecific avoidance of song overlap in tropical songbirds: species-specific responses to acoustically similar and different intruders

To ensure effective acoustic communication, signals should reach receivers in the least distorted form possible. Animals use various short- and long-term strategies to avoid signal degradation and masking. However, we still have an insufficient understanding of how animals’ vocal behaviour is impacted by the vocalisations of other animals in their acoustic communities. We experimentally examined how two tropical, sedentary, territorial songbirds in Western Uganda—the scaly-breasted illadopsis (Illadopsis albipectus) and the green-backed camaroptera (Camaroptera brachyura)—modify their singing behaviour after the simulated appearance of new, unfamiliar acoustic competitors, whose songs vary in similarity to those of the species studied. We found that scaly-breasted illadopsis sang significantly less during the playback of songs of acoustically similar species than of acoustically different species or silence and avoided song overlapping with acoustically similar species but not with acoustically different species. Green-backed camaroptera sang significantly more during the playback of both acoustically similar and different simulated intruders than during the control containing silence, and patterns of overlap with the songs of both the acoustically similar and different species were random. Our results show that even a single-point noise source present within a territory can modify a bird’s singing behaviour. The new sound may affect species differently, depending in part on the level of acoustic similarity with the species’ song. To mitigate the effect of song masking, different species may use different strategies, such as temporal avoidance or signal redundancy. Studies examining the adaptive abilities of species in natural and modified habitats are needed to predict the consequences of changes in acoustic community structure.Significance statementTo ensure effective communication, birds may use different strategies to avoid signal masking in common acoustic space, particularly in the complex acoustic environment of a tropical forest. While multiple studies have focused on responses to interference caused by anthropogenic noise, the effect of new individual species on the acoustic community structure has received little attention. We simulated intrusions by unfamiliar species with different levels of song similarity into the territories of two tropical songbird species. The appearance of new simulated acoustic intruders modified the birds’ singing behaviour, but the two study species responded differently. These results suggest that the level of acoustic similarity, as well as the species ecology, may affect the species response, which may be particularly important when predicting the effects of new species appearance as a result of changes in habitat and climate.

Attentional modulation of neural sound tracking in children with and without dyslexia.

Auditory selective attention forms an important foundation of children's learning by enabling the prioritisation and encoding of relevant stimuli. It may also influence reading development, which relies on metalinguistic skills including the awareness of the sound structure of spoken language. Reports of attentional impairments and speech perception difficulties in noisy environments in dyslexic readers are also suggestive of the putative contribution of auditory attention to reading development. To date, it is unclear whether non-speech selective attention and its underlying neural mechanisms are impaired in children with dyslexia and to which extent these deficits relate to individual reading and speech perception abilities in suboptimal listening conditions. In this EEG study, we assessed non-speech sustained auditory selective attention in 106 7-to-12-year-old children with and without dyslexia. Children attended to one of two tone streams, detecting occasional sequence repeats in the attended stream, and performed a speech-in-speech perception task. Results show that when children directed their attention to one stream, inter-trial-phase-coherence at the attended rate increased in fronto-central sites; this, in turn, was associated with better target detection. Behavioural and neural indices of attention did not systematically differ as a function of dyslexia diagnosis. However, behavioural indices of attention did explain individual differences in reading fluency and speech-in-speech perception abilities: both these skills were impaired in dyslexic readers. Taken together, our results show that children with dyslexia do not show group-level auditory attention deficits but these deficits may represent a risk for developing reading impairments and problems with speech perception in complex acoustic environments. RESEARCH HIGHLIGHTS: Non-speech sustained auditory selective attention modulates EEG phase coherence in children with/without dyslexia Children with dyslexia show difficulties in speech-in-speech perception Attention relates to dyslexic readers' speech-in-speech perception and reading skills Dyslexia diagnosis is not linked to behavioural/EEG indices of auditory attention.

Computation-efficient solution for fully-connected active noise control window: Analysis and implementation of multichannel adjoint least mean square algorithm

The multichannel active noise control (MCANC) system, in which multiple reference sensors and actuators are used to enlarge the noise-cancellation zone, is widely utilized in complex acoustic environments. However, as the number of channels increases, the practicality decreases due to the exponential rise in computational complexity. This paper, therefore, revisits the adjoint least mean square (ALMS) algorithm and its multichannel applications. The computational analysis reveals that the multichannel adjoint least mean square (McALMS) algorithm11The code is shared on https://www.mathworks.com/matlabcentral/fileexchange/127554-multichannel-adjoint-lms-algorithm-for-multichannel-anc. has a significantly lower computation cost when implementing the fully connected active noise control (ANC) structure. In addition to this advantage, the theoretical analysis presented in this paper demonstrates that the McALMS algorithm can achieve the same optimal solution as the standard adaptive algorithm without the assumptions of input independence and white Gaussian noise. In addition, a practical step-size estimation strategy based on the Golden-section search (GSS) method is proposed to predict the fast step size of the McALMS algorithm. The numerical simulations in a multichannel ANC system demonstrate the effectiveness of the McALMS algorithm and validate the derived theoretical analysis. Furthermore, the McALMS algorithm with proposed step-size approach is used to implement a multichannel noise cancellation window that achieves satisfactory global noise reduction performance for tonal, broadband, and even real-world noises.

Intensity discrimination and neural representation of a masked tone in the presence of three types of masking release.

Hearing ability is usually evaluated by assessing the lowest detectable intensity of a target sound, commonly referred to as a detection threshold. Detection thresholds of a masked signal are dependent on various auditory cues, such as the comodulation of the masking noise, interaural differences in phase, and temporal context. However, considering that communication in everyday life happens at sound intensities well above the detection threshold, the relevance of these cues for communication in complex acoustical environments is unclear. Here, we investigated the effect of three cues on the perception and neural representation of a signal in noise at supra-threshold levels. First, we measured the decrease in detection thresholds produced by three cues, referred to as masking release. Then, we measured just-noticeable difference in intensity (intensity JND) to quantify the perception of the target signal at supra-threshold levels. Lastly, we recorded late auditory evoked potentials (LAEPs) with electroencephalography (EEG) as a physiological correlate of the target signal in noise at supra-threshold levels. The results showed that the overall masking release can be up to around 20 dB with a combination of these three cues. At the same supra-threshold levels, intensity JND was modulated by the masking release and differed across conditions. The estimated perception of the target signal in noise was enhanced by auditory cues accordingly, however, it did not differ across conditions when the target tone level was above 70 dB SPL. For the LAEPs, the P2 component was more closely linked to the masked threshold and the intensity discrimination than the N1 component. The results indicate that masking release affects the intensity discrimination of a masked target tone at supra-threshold levels, especially when the physical signal-to-noise is low, but plays a less significant role at high signal-to-noise ratios.

Anti-interference self-powered acoustic fabric for complex acoustic environments

Traditional airborne microphones are at risk of failure due to their dependence on airborne media in complex acoustic environments (CAEs). Here, we report an anti-interference self-powered acoustic fabric (ASAF) that can shield the interfering factors in the process of sound production and propagation to serve as a precise and wearable sound receiver. The use of the soft and safe woven structure polyvinylidene fluoride (PVDF) as a vibration-sensitive layer enables the ASAF to record human speech at wide vibration frequencies (0–5000 Hz). A speech recognition system is established which can recognize 25 words related to extreme weather conditions, with more than 95.8% accuracy. This speech recognition is carried out in CAEs such as wearing masks, silent communication, windstorms, and rainstorms, with corresponding losses of accuracy less than 1.6%, 6.7%, 6.3%, and 5.8%, respectively. The ASAF is expected to facilitate outdoor rescuers, journalists, students, and other professionals working in CAEs.

Multi-Scale Feature Learning for Language Identification of Overlapped Speech

Language identification is the front end of multilingual speech-processing tasks. The study aims to enhance the accuracy of language identification in complex acoustic environments by proposing a multi-scale feature extraction method. This method replaces the baseline feature extraction network with a multi-scale feature extraction network (SE-Res2Net-CBAM-BILSTM) to extract multi-scale features. A multilingual cocktail party dataset was simulated, and comparative experiments were conducted with various models. The experimental results show that the proposed model achieved language identification accuracies of 97.6% for an Oriental language dataset and 75% for a multilingual cocktail party dataset Furthermore, comparative experiments show that our model outperformed three other models in the accuracy, recall, and F1 values. Finally, a comparison of different loss functions shows that the model performance was better when using focal loss.

Psychophysical performance differences associated with auditory spatial attention in older listeners with and without hearing loss

In a busy listening environment with multiple speakers, even normal-hearing listeners may face difficulties in understanding speech (i.e., “the cocktail party problem”). Individual factors related to bottom-up and/or top-down processing difficulties may be associated with the listener’s ability to attend to speech, a key component in navigating complex acoustic environments. The present study tested listeners on a battery of psychophysical measures and compared listener performance to cortical measures (EEG) of spatial-change tuning and auditory spatial attention. The behavioral tests measured both perceptual and cognitive performance and included: frequency modulation detection, spectro-temporal modulation sensitivity, spatial release from masking, temporal gap detection, auditory and non-auditorysustained attention, and visuo-spatial problem solving. The EEG paradigm tested the listener’s ability to attend to moving speech in quiet or background babble in the free field. Older listeners with and without hearing loss (n = 20 per group) demonstrated wide individual performance for some measures and significant correlations between behavioral thresholds and cortical responses, consistent with a more refined view of age-related hearing loss and a need for a more holistic approach in diagnoses for individuals beyond standard clinical tests. [Work supported by NIH R21DC017832.]

A binaural heterophasic adaptive beamformer and its deep learning assisted implementation

Beamforming is one of the most effective approaches to distant sound acquisition in complex acoustic environments, where noise, reverberation, and interference coexist; as a result, a significant number of efforts have been devoted to it over the last few decades. However, conventional beamformers produce a monaural output or colinear outputs, which are not optimal from the perception perspective. To take advantage of the human binaural hearing properties, a new type of fixed beamforming methods were developed recently, which attempt not only to attenuate noise but also render the signal of interest and residual noise into different perceptual regions, thereby achieving higher speech intelligibility. This work extends the principle of fixed binaural beamforming and develops a binaural heterophasic minimum variance distortionless response (MVDR) beamformer. A deep neural network (DNN) based noise estimation method is used to assist the implementation of this heterophasic MVDR beamformer, which is advantageous over the traditional one as it renders the desired source signal and residual noise to different perceptual regions, thereby yielding higher intelligibility. In comparison with the fixed binaural heterophasic beamformers, it can take advantage of the statistics of the noise to achieve better array performance. Results of simulations and listening tests validate the properties of the proposed technique.

Binaural detection thresholds and audio quality of speech and music signals in complex acoustic environments.

Every-day acoustical environments are often complex, typically comprising one attended target sound in the presence of interfering sounds (e.g., disturbing conversations) and reverberation. Here we assessed binaural detection thresholds and (supra-threshold) binaural audio quality ratings of four distortions types: spectral ripples, non-linear saturation, intensity and spatial modifications applied to speech, guitar, and noise targets in such complex acoustic environments (CAEs). The target and (up to) two masker sounds were either co-located as if contained in a common audio stream, or were spatially separated as if originating from different sound sources. The amount of reverberation was systematically varied. Masker and reverberation had a significant effect on the distortion-detection thresholds of speech signals. Quality ratings were affected by reverberation, whereas the effect of maskers depended on the distortion. The results suggest that detection thresholds and quality ratings for distorted speech in anechoic conditions are also valid for rooms with mild reverberation, but not for moderate reverberation. Furthermore, for spectral ripples, a significant relationship between the listeners' individual detection thresholds and quality ratings was found. The current results provide baseline data for detection thresholds and audio quality ratings of different distortions of a target sound in CAEs, supporting the future development of binaural auditory models.

High-precision DOA estimation for underwater acoustic signals based on sparsity adaptation

The direction of arrival (DOA) estimation technique is to obtain the direction information of the source when it reaches the array by processing and analyzing the received signal. In recent years, the DOA estimation of an array signal has been a research hotspot. For application scenarios with a small number of snapshots and a low signal-to-noise ratio, the compressive sensing theory has been commonly used to estimate the DOA of an array signal to achieve better estimation performance. However, the DOA estimation methods based on compressive sensing theory require information on source sparsity. Moreover, the influence of a complex underwater acoustic environment limits the accuracy of estimation algorithms. To address this limitation, this study proposes a high-precision DOA estimation model for underwater acoustic signals based on sparsity adaptation. The proposed model includes mainly two parts. In the first part, a source sparsity adaptive model based on a causal convolutional neural network is proposed. The model is used to address the constraint that the source sparsity should be known a priori when compressed sensing is used for DOA estimation. In the second part, a differential combination matching pursuit (DCMP) algorithm is adopted. First, a differentiated path filtering strategy is employed to reduce algorithm complexity and avoid the problem of invalid filtering. In addition, the combined optimization strategy is used to improve the prediction accuracy of the algorithm, providing an efficient error correction idea for the compressed sensing application to DOA estimation. The results of simulations conducted under seven different signal-to-noise ratios and using three different array types show that the proposed source sparsity adaptive model can reach an average prediction accuracy of 89.6%. In addition, compared with the other reconstruction algorithm accuracy, on the basis of ensuring low time complexity, the proposed DCMP algorithm can achieve an accuracy improvement of 9.99%–19.94% under seven different signal-to-noise ratio values. Moreover, the mean absolute error of the proposed DCMP algorithm is lower by approximately 0.05°–14° than those of the OMP and MMP algorithms.