Acoustic Context Research Articles

A Reverberation-Time-Aware Approach to Speech Dereverberation Based on Deep Neural Networks

A reverberation-time-aware deep-neural-network DNN-based speech dereverberation framework is proposed to handle a wide range of reverberation times. There are three key steps in designing a robust system. First, in contrast to sigmoid activation and min-max normalization in state-of-the-art algorithms, a linear activation function at the output layer and global mean-variance normalization of target features are adopted to learn the complicated nonlinear mapping function from reverberant to anechoic speech and to improve the restoration of the low-frequency and intermediate-frequency contents. Next, two key design parameters, namely, frame shift size in speech framing and acoustic context window size at the DNN input, are investigated to show that RT60-dependent parameters are needed in the DNN training stage in order to optimize the system performance in diverse reverberant environments. Finally, the reverberation time is estimated to select the proper frame shift and context window sizes for feature extraction before feeding the log-power spectrum features to the trained DNNs for speech dereverberation. Our experimental results indicate that the proposed framework outperforms the conventional DNNs without taking the reverberation time into account, while achieving a performance only slightly worse than the oracle cases with known reverberation times even for extremely weak and severe reverberant conditions. It also generalizes well to unseen room sizes, loudspeaker and microphone positions, and recorded room impulse responses.

Robust Set-Membership Affine Projection Algorithm with Coefficient Vector Reuse

This paper proposes a robust set-membership affine projection algorithm with coefficient vector reuse (RSM-APA-CVR) for high background noise environment. In the proposed algorithm, the sum of the squared \(L_{2}\) norms of the differences between the updated weight vector and past weight vectors is minimized and a new robust error bound is designed. Simulation results in acoustic echo cancellation context show that the proposed algorithm has faster convergence rate and smaller steady-state misalignment as compared to the conventional RSM-APA.

Cognitive load makes speech sound fast, but does not modulate acoustic context effects

Abstract In natural situations, speech perception often takes place during the concurrent execution of other cognitive tasks, such as listening while viewing a visual scene. The execution of a dual task typically has detrimental effects on concurrent speech perception, but how exactly cognitive load disrupts speech encoding is still unclear. The detrimental effect on speech representations may consist of either a general reduction in the robustness of processing of the speech signal (‘noisy encoding’), or, alternatively it may specifically influence the temporal sampling of the sensory input, with listeners missing temporal pulses, thus underestimating segmental durations (‘shrinking of time’). The present study investigated whether and how spectral and temporal cues in a precursor sentence that has been processed under high vs. low cognitive load influence the perception of a subsequent target word. If cognitive load effects are implemented through ‘noisy encoding’, increasing cognitive load during the precursor should attenuate the encoding of both its temporal and spectral cues, and hence reduce the contextual effect that these cues can have on subsequent target sound perception. However, if cognitive load effects are expressed as ‘shrinking of time’, context effects should not be modulated by load, but a main effect would be expected on the perceived duration of the speech signal. Results from two experiments indicate that increasing cognitive load (manipulated through a secondary visual search task) did not modulate temporal (Experiment 1) or spectral context effects (Experiment 2). However, a consistent main effect of cognitive load was found: increasing cognitive load during the precursor induced a perceptual increase in its perceived speech rate, biasing the perception of a following target word towards longer durations. This finding suggests that cognitive load effects in speech perception are implemented via ‘shrinking of time’, in line with a temporal sampling framework. In addition, we argue that our results align with a model in which early (spectral and temporal) normalization is unaffected by attention but later adjustments may be attention-dependent.

An Efficient $$L_{0}$$ L 0 Norm Constraint Memory Improved Proportionate Affine Projection Algorithm

This paper proposes an \(L_{0}\) norm constraint memory improved proportionate affine projection algorithm with a lower computational complexity than the conventional \(L_{0}\) norm constraint improved proportionate affine projection algorithm. Particularly, to achieve a low computational complexity, we propose to remove the matrix before the zero attraction term. Moreover, the product of the proportionate matrix and input matrix is implemented using an efficient recursive scheme. Simulation results in acoustic echo cancellation context show that our algorithm not only significantly reduces the computational complexity but also achieves slightly improved steady-state misalignment.

Sonic departures: the dramaturgy of fade-outs

ABSTRACTIn technical and theoretical terms, this article addresses the fading-out of pre-recorded sound in theatre. Using models of listening and attention, with particular reference to how silence is constituted, it attempts to understand how fade-outs can interrupt or complicate the experience of sound in theatre and proposes solutions to the problem of fading-out in various dramaturgical contexts. This involves a survey of how sounds fade in other artistic and acoustic contexts and of existing attempts to model how sounds dissolve in the fields of synthesis and the design of artificial reverb. It ends with an attempt to conceptually situate an expanded use of fade-outs within contemporary theatre and other sound practices.

Auditory and acoustic-phonetic mechanisms of adaptation in the perception of sibilant fricatives

Previous research has demonstrated that speech perception is highly dependent on preceding acoustic context (e.g., Ladefoged & Broadbent, 1957), and suggested that this reflects adaptation to the long-term average spectrum (LTAS; e.g., Holt, 2006). The present study examined whether adaptation is further enhanced by acoustic-phonetic covariation among speech sounds within the natural class of sibilants. In each trial of Experiment 1, listeners were exposed to a [z]-initial CVC syllable and then categorized one member of an [s]-[ʃ] continuum. The distribution of the spectral center of gravity (COG) of [z] in the adaptation stimuli was manipulated (high vs. low). In Experiment 2, the [z]-initial syllables were replaced by LTAS-matched white noise. Listeners adjusted the [s]-[ʃ] boundary in accordance with the exposure conditions, showing contrastive adaptation in both experiments; however, the adaptation effect was significantly stronger with the speech precursor. Furthermore, in Experiment 1, fricative COG provided a better quantitative account of responses than COG of the full syllable (ΔBIC: 24). These findings indicate that while general auditory processes can give rise to adaptation, listeners also exploit acoustic-phonetic relations among sounds: here, high correlations among fricative COG means observed in a large multi-talker corpus ([s]-[z], r = 0.62, [s]-[ʃ], r = 0.53).

Repeated call sequences and behavioural context in long-finned pilot whales off Cape Breton, Nova Scotia, Canada

Repeated calls are part of the vocal repertoire of a diverse array of species, often presented in sequences that take time and effort on the part of the signal producer. Rhythmic repeated call sequences make up a significant portion of long-finned pilot whale (Globicephala melas) vocal production, yet the function of these sequences has not been investigated until now. In this study, we explored the relationship between behavioural context and the presence of these vocal sequences using recordings of a population of pilot whales found off Cape Breton, Nova Scotia, Canada. We used a binomial logit-link generalized linear model to look for possible predictors of the presence of repeated call sequences. They were more common in recordings of socializing whales than in those of whales in other behavioural states, and least common in resting whales. These vocal repetitions were also more common with larger group size. These results suggest that sequences function in maintaining contact and cohesion within this social species, possibly also serving in individual or group identification. The context of repeated call sequences indicate that they are not primarily mother–calf interactions, as they are heard just as commonly from groups without young. Future studies of pilot whale repeated call sequences should include individual-level behaviour and detailed acoustic calling context.

Processing Complex Sounds Passing through the Rostral Brainstem: The New Early Filter Model.

The rostral brainstem receives both “bottom-up” input from the ascending auditory system and “top-down” descending corticofugal connections. Speech information passing through the inferior colliculus of elderly listeners reflects the periodicity envelope of a speech syllable. This information arguably also reflects a composite of temporal-fine-structure (TFS) information from the higher frequency vowel harmonics of that repeated syllable. The amplitude of those higher frequency harmonics, bearing even higher frequency TFS information, correlates positively with the word recognition ability of elderly listeners under reverberatory conditions. Also relevant is that working memory capacity (WMC), which is subject to age-related decline, constrains the processing of sounds at the level of the brainstem. Turning to the effects of a visually presented sensory or memory load on auditory processes, there is a load-dependent reduction of that processing, as manifest in the auditory brainstem responses (ABR) evoked by to-be-ignored clicks. Wave V decreases in amplitude with increases in the visually presented memory load. A visually presented sensory load also produces a load-dependent reduction of a slightly different sort: The sensory load of visually presented information limits the disruptive effects of background sound upon working memory performance. A new early filter model is thus advanced whereby systems within the frontal lobe (affected by sensory or memory load) cholinergically influence top-down corticofugal connections. Those corticofugal connections constrain the processing of complex sounds such as speech at the level of the brainstem. Selective attention thereby limits the distracting effects of background sound entering the higher auditory system via the inferior colliculus. Processing TFS in the brainstem relates to perception of speech under adverse conditions. Attentional selectivity is crucial when the signal heard is degraded or masked: e.g., speech in noise, speech in reverberatory environments. The assumptions of a new early filter model are consistent with these findings: A subcortical early filter, with a predictive selectivity based on acoustical (linguistic) context and foreknowledge, is under cholinergic top-down control. A prefrontal capacity limitation constrains this top-down control as is guided by the cholinergic processing of contextual information in working memory.

Acoustic reduction, context, and inter-stimulus interval in cross-modal priming

Work such as Tucker (2011) and van de Ven et al. (2011) show that reduced acoustic contrasts can impede lexical access for a listener. Tucker (2011) also found preliminary evidence of a nonlinear (U-shaped) relationship between the degree of intensity dip in stops and lexical decision response times. The present talk summarizes results from a cross-modal identity priming experiment designed to explore this potential nonlinearity further. Utilizing three different inter-stimulus-intervals, trials were presented in which conversational words with variously reduced intervocalic stops served as auditory primes. Visual targets included the same word as the prime, words with a large degree of phonological overlap, and unrelated controls, as well as phonologically overlapping and non-overlapping pseudowords. Additionally, the auditory primes were presented with three degrees of surrounding context: isolation (the word only), phonetic context (including the vowels in neighboring syllables, providing primarily spe...

Dissociating contributions of talker gender and acoustic variability for spectral contrast effects in vowel categorization

Speech categorization is influenced by spectral contrast effects (SCEs), the perceptual magnification of spectral differences between successive sounds. Through SCEs, preceding acoustic contexts can bias categorization of following sounds away from reliable spectral properties. Recent findings (Assgari & Stilp, 2015 JASA) show that SCEs in vowel categorization can be modulated by talker characteristics: a clear SCE when the preceding context was 200 sentences spoken by a single talker was diminished when the context featured 200 talkers. This result was attributed to variability in mean pitch of the preceding sentences. However, neither mean sentence pitch nor talker gender was explicitly controlled, which challenges identification of the locus of the talker effect. The current study examined whether gender and pitch variability yield separable contributions to SCEs. Listeners heard precursor sentences then categorized a target vowel from an /ɪ/ to /ɛ/ continuum. Sentences were processed to add a modest l...

Enhanced novelty detection in auditory scenes through adaptation of inhibition

Animals and humans must behave efficiently in acoustically rich environments, where few sound sources are of interest of many others that are considered background noise. How does the auditory system implement such robust processing is a topic of intense research. In this work I am interested in the neural mechanisms underlying detection of novel objects in auditory scenes, such as a novel sound source at a given location. It has been shown psycho-physically that the perception of a sound depends on the acoustical context in which it occurs. A basic example of such effect is called enhancement [1]. A harmonic complex (i.e. a series of tones at different frequencies) presented in isolation will be perceived as a single auditory object. If one tone is deleted from the complex (the conditioner) and then reintroduced (test), it will perceptually pop out as a separate sound object. To maximally induce enhancement in humans, the conditioner must have an energy notch in its spectrum with a preferred width of 0.6 octave. This effect originates in the central nervous system; neural correlate thereof were found in the cochlear nucleus, two synapses away from the cochlea [2]. The standard model explaining the emergence of auditory enhancement is based on the adaptation of wide-band inhibition inputs [1,2]: the increase in output rate due to a preceding conditioner comes from a decrease in activity of the wide-band inhibitory inputs, i.e. inhibition have adapted during the conditioner while narrow band excitation has not. In this study I seek to test functions of enhancement effects in realistic listening situations. In particular, I hypothesize that the enhancement of speech signals by realistic environments is a neural implementation of a novelty detector. I implemented on the Brian simulator and its auditory package [3] a model of the auditory periphery followed by a network of adapting spiking neurons [4] comprising feed forward excitation and inhibition. This network was first tuned to reproduce the basic enhancement effects reported physiologically. In order to mimic realistic listening conditions, sounds are filtered by realistic environmental processes prior to being fed to the model. In particular I'm interested in the filtering induced in sounds arriving at the ear by the listener's body and by the surrounding room. The head and the shoulder create reflections of the direct sound, resulting in notches in the high frequency sound spectrum, which depend on the sound source elevation [5]. Similarly, reflections from the ground and the walls of a room will induce notches in the low-frequency spectrum, which depend on the sound source azimuth and distance [6]. As spectral notches are critical in inducing auditory enhancement I show that if a novel sound is preceded by other sounds located at different positions, the detection of this novel sound will be enhanced. The present functional study of auditory enhancement suggests that such contextual effects could be critical for animal survival and for hearing in realistic conditions. The results show how simple neural mechanisms modulate a sensory pathway so that it adapts to its surrounding environment and suggest that bias in perception could have ecological advantages. The model could also inspire the design of speech processing devices in real environments.

Two distinct representations of social vocalizations in the basolateral amygdala.

Acoustic communication signals carry information related to the types of social interactions by means of their "acoustic context," the sequencing and temporal emission pattern of vocalizations. Here we describe responses to natural vocal sequences in adult big brown bats (Eptesicus fuscus). We first assessed how vocal sequences modify the internal affective state of a listener (via heart rate). The heart rate of listening bats was differentially modulated by vocal sequences, showing significantly greater elevation in response to moderately aggressive sequences than appeasement or neutral sequences. Next, we characterized single-neuron responses in the basolateral amygdala (BLA) of awake, restrained bats to isolated syllables and vocal sequences. Two populations of neurons distinguished by background firing rates also differed in acoustic stimulus selectivity. Low-background neurons (<1 spike/s) were highly selective, responding on average to one tested stimulus. These may participate in a sparse code of vocal stimuli, in which each neuron responds to one or a few stimuli and the population responds to the range of vocalizations across behavioral contexts. Neurons with higher background rates (≥1 spike/s) responded broadly to tested stimuli and better represented the timing of syllables within sequences. We found that spike timing information improved the ability of these neurons to discriminate among vocal sequences and among the behavioral contexts associated with sequences compared with a rate code alone. These findings demonstrate that the BLA contains multiple robust representations of vocal stimuli that can provide the basis for emotional/physiological responses to these stimuli.

Frame Based Postprocessor for Speech Recognition Based on Augmented Conditional Random Fields

In this paper, we present a novel postprocessor for speech recognition using the Augmented Conditional Random Field (ACRF) framework. In this framework, a primary acoustic model is used to generate state posterior scores per frame. These output scores are fed to the ACRF postprocessor for further frame based acoustic modeling. Since ACRF explicitly integrates acoustic context modeling, the postprocessor has the ability to discover new context information and to improve the recognition accuracy. The results on the TIMIT phone recognition task show that the proposed postprocessor can lead to significant improvements especially when Hidden Markov Models (HMMs) were used as primary acoustic model.

Within- and between-talker variability in voice quality in normal speaking situations

Increasing evidence suggests that voices are best thought of as complex auditory patterns, and that listeners perceive and remember voices with reference to a “prototype” or “average” for that talker. Little is known about how, and how much, individual talkers vary their voice quality across situations that arise in every-day speaking, so the nature and extent of variability underlying these abstract averages, and thus the nature of the averages themselves, is unclear. The theoretical relationship between acoustic similarity and confusability in the context of a prototype model also remains unclear. In this preliminary study, 9 tokens of the vowel /a/ were recorded from 5 females on three dates. Measures of F0, spectral slope, HNR, and formant frequencies and their variability were gathered for all voice samples and acoustic distances between talkers were calculated under the assumption that all acoustic variables were equally important perceptually. Perceptual confusability was assessed in a same/different task, and predictions under the equal perceptual weight assumption were tested. Discussion will focus on how much variability is required before a voice sample no longer sounds like the originating talker, and on how the perceptual importance of each acoustical variable varies across talkers and acoustic contexts. [Work supported by NSF and NIH.]

Reverberation increases perceptual calibration to reliable spectral peaks in speech

Preceding acoustic context influences speech perception, especially when it features a reliable spectral property (i.e., relatively stable or recurring across time). When preceding sounds have a spectral peak matching F2 of the following target vowel, listeners decrease reliance on F2 and increase reliance on changing, more informative cues for vowel identification. This process, known as auditory perceptual calibration, has only been studied in anechoic conditions. Room reverberation smears spectral information across time, presumably giving listeners additional “looks” at reliable spectral peaks, which should increase the degree of perceptual calibration. Listeners identified vowels (varying from [i] to [u] in F2 and spectral tilt) presented in isolation, then following a sentence filtered to share a spectral peak with the target vowel’s F2. Calibration was measured as the change in perceptual weights (standardized logistic regression coefficients) across blocks. Listeners completed sessions where stimuli were diotic or processed by virtual room acoustics to simulate high reverberation (T60 = 2.97 s). Perceptual calibration was significantly greater (larger decreases in F2 weight, larger increases in tilt weight) for reverberant stimuli than diotic stimuli. Interestingly, hearing reverberant stimuli first attenuated calibration in subsequent diotic stimuli. Results demonstrate perceptual sensitivity to the persistence of reliable spectral peaks across time.

피보호자 모니터링 시스템을 위한 환경음 기반 상황 인식

This paper presents a context recognition method using environmental sound signals, which is applied to a mobile-based client monitoring system. Seven acoustic contexts are defined and the corresponding environmental sound signals are obtained for the experiments. To evaluate the performance of the context recognition, MFCC and LPCC method are employed as feature extraction, and statistical pattern recognition method are used employing GMM and HMM as acoustic models, The experimental results show that LPCC and HMM are more effective at improving context recognition accuracy compared to MFCC and GMM respectively. The recognition system using LPCC and HMM obtains 96.03% in recognition accuracy. These results demonstrate that LPCC is effective to represent environmental sounds which contain more various frequency components compared to human speech. They also prove that HMM is more effective to model the time-varying environmental sounds compared to GMM.

Auditory modulation of wind-elicited walking behavior in the cricket Gryllus bimaculatus.

Animals flexibly change their locomotion triggered by an identical stimulus depending on the environmental context and behavioral state. This indicates that additional sensory inputs in different modality from the stimulus triggering the escape response affect the neuronal circuit governing that behavior. However, how the spatio-temporal relationships between these two stimuli effect a behavioral change remains unknown. We studied this question, using crickets, which respond to a short air-puff by oriented walking activity mediated by the cercal sensory system. In addition, an acoustic stimulus, such as conspecific 'song' received by the tympanal organ, elicits a distinct oriented locomotion termed phonotaxis. In this study, we examined the cross-modal effects on wind-elicited walking when an acoustic stimulus was preceded by an air-puff and tested whether the auditory modulation depends on the coincidence of the direction of both stimuli. A preceding 10 kHz pure tone biased the wind-elicited walking in a backward direction and elevated a threshold of the wind-elicited response, whereas other movement parameters, including turn angle, reaction time, walking speed and distance were unaffected. The auditory modulations, however, did not depend on the coincidence of the stimulus directions. A preceding sound consistently altered the wind-elicited walking direction and response probability throughout the experimental sessions, meaning that the auditory modulation did not result from previous experience or associative learning. These results suggest that the cricket nervous system is able to integrate auditory and air-puff stimuli, and modulate the wind-elicited escape behavior depending on the acoustic context.

Divide and conquer: How perceptual contrast sensitivity and perceptual learning cooperate in reducing input variation in speech perception.

Listeners have to overcome variability of the speech signal that can arise, for example, because of differences in room acoustics, differences in speakers' vocal tract properties, or idiosyncrasies in pronunciation. Two mechanisms that are involved in resolving such variation are perceptually contrastive effects that arise from surrounding acoustic context and lexically guided perceptual learning. Although both processes have been studied in great detail, little attention has been paid to how they operate relative to each other in speech perception. The present study set out to address this issue. The carrier parts of exposure stimuli of a classical perceptual learning experiment were spectrally filtered such that the acoustically ambiguous final fricatives sounded relatively more like the lexically intended sound (Experiment 1) or the alternative (Experiment 2). Perceptual learning was found only in the latter case. The findings show that perceptual contrast effects precede lexically guided perceptual learning, at least in terms of temporal order, and potentially in terms of cognitive processing levels as well.

Induction of enhanced acoustic startle response by noise exposure: dependence on exposure conditions and testing parameters and possible relevance to hyperacusis.

There has been a recent surge of interest in the development of animal models of hyperacusis, a condition in which tolerance to sounds of moderate and high intensities is diminished. The reasons for this decreased tolerance are likely multifactorial, but some major factors that contribute to hyperacusis are increased loudness perception and heightened sensitivity and/or responsiveness to sound. Increased sound sensitivity is a symptom that sometimes develops in human subjects after acoustic insult and has recently been demonstrated in animals as evidenced by enhancement of the acoustic startle reflex following acoustic over-exposure. However, different laboratories have obtained conflicting results in this regard, with some studies reporting enhanced startle, others reporting weakened startle, and still others reporting little, if any, change in the amplitude of the acoustic startle reflex following noise exposure. In an effort to gain insight into these discrepancies, we conducted measures of acoustic startle responses (ASR) in animals exposed to different levels of sound, and repeated such measures on consecutive days using a range of different startle stimuli. Since many studies combine measures of acoustic startle with measures of gap detection, we also tested ASR in two different acoustic contexts, one in which the startle amplitudes were tested in isolation, the other in which startle amplitudes were measured in the context of the gap detection test. The results reveal that the emergence of chronic hyperacusis-like enhancements of startle following noise exposure is highly reproducible but is dependent on the post-exposure thresholds, the time when the measures are performed and the context in which the ASR measures are obtained. These findings could explain many of the discrepancies that exist across studies and suggest guidelines for inducing in animals enhancements of the startle reflex that may be related to hyperacusis.

Modest, reliable spectral peaks in preceding sounds influence vowel perception.

When a spectral property is reliable across an acoustic context and subsequent vowel target, perception deemphasizes this cue and shifts toward less predictable, more informative cues. This phenomenon (auditory perceptual calibration) has been demonstrated for reliable spectral peaks +20 dB or larger, but psychoacoustic findings predict sensitivity to more modest spectral peaks. Listeners identified vowel targets following a sentence with a reliable +2 to +15 dB spectral peak centered at F2 of the vowel. Vowel identifications weighted F2 significantly less when reliable peaks were at least +5 dB. Results demonstrate high sensitivity to reliable acoustic properties in the sensory environment.