Temporal Envelope Processing Research Articles

Temporal-envelope constancy of speech in rooms and the perceptual weighting of frequency bands

Three experiments measured constancy in speech perception, using natural-speech messages or noise-band vocoder versions of them. The eight vocoder-bands had equally log-spaced center-frequencies and the shapes of corresponding "auditory" filters. Consequently, the bands had the temporal envelopes that arise in these auditory filters when the speech is played. The "sir" or "stir" test-words were distinguished by degrees of amplitude modulation, and played in the context; "next you'll get _ to click on." Listeners identified test-words appropriately, even in the vocoder conditions where the speech had a "noise-like" quality. Constancy was assessed by comparing the identification of test-words with low or high levels of room reflections across conditions where the context had either a low or a high level of reflections. Constancy was obtained with both the natural and the vocoded speech, indicating that the effect arises through temporal-envelope processing. Two further experiments assessed perceptual weighting of the different bands, both in the test word and in the context. The resulting weighting functions both increase monotonically with frequency, following the spectral characteristics of the test-word's [s]. It is suggested that these two weighting functions are similar because they both come about through the perceptual grouping of the test-word's bands.

Dolphins (Tursiops truncatus) use multiple auditory mechanisms for detecting tonal signals in natural and anthropogenic noise

The dolphin's ability to detect a 10 kHz tone masked by a variety of noise types was measured using a standard band-widening paradigm. Maskers included natural noise (rain, snapping shrimp, and ice squeaks), anthropogenic noise (pile saw and boat propeller cavitation) and statistical noise (Gaussian and comodulated noise). For most noise types, detection thresholds increased as noise bandwidth increased up to 1 kHz (the dolphin's critical bandwidth at 10 kHz). Masking patterns for narrow-band maskers were similar regardless of the masker type. However, for noise bandwidths greater than 1 kHz, masking patterns diverged by as much as 23 dB depending on the noise type. The power spectrum model provided reasonable predictions for Gaussian, rain, pile saw, and boat noise masking patterns. Additional experiments suggested that mechanisms related to temporal envelope processing (across-channel envelope comparison and within-valley listening) determined masking patterns for snapping shrimp and comodulated noise. Thresholds in ice squeak noise, which proved to be the most effective masker, were related to the dolphin's inability to discriminate the signal from the background noise rather than inability to detect the signal. These results suggest that the dolphin auditory system uses multiple mechanisms for signal detection in complex noise [Work supported by the ONR.]

Detection and rate discrimination of amplitude modulation in electrical hearing

Three experiments were designed to examine temporal envelope processing by cochlear implant (CI) listeners. In experiment 1, the hypothesis that listeners' modulation sensitivity would in part determine their ability to discriminate between temporal modulation rates was examined. Temporal modulation transfer functions (TMTFs) obtained in an amplitude modulation detection (AMD) task were compared to threshold functions obtained in an amplitude modulation rate discrimination (AMRD) task. Statistically significant nonlinear correlations were observed between the two measures. In experiment 2, results of loudness-balancing showed small increases in the loudness of modulated over unmodulated stimuli beyond a modulation depth of 16%. Results of experiment 3 indicated small but statistically significant effects of level-roving on the overall gain of the TMTF, but no impact of level-roving on the average shape of the TMTF across subjects. This suggested that level-roving simply increased the task difficulty for most listeners, but did not indicate increased use of intensity cues under more challenging conditions. Data obtained with one subject, however, suggested that the most sensitive listeners may derive some benefit from intensity cues in these tasks. Overall, results indicated that intensity cues did not play an important role in temporal envelope processing by the average CI listener.

Age Effects in Temporal Envelope Processing: Speech Unmasking and Auditory Steady State Responses

The purpose of this study was to determine whether temporal envelope processing is reduced in older listeners. Experiment 1 tested the hypothesis that older listeners exhibit reduced speech unmasking at higher modulation rates. Experiment 2 tested the hypothesis that auditory steady state response (ASSR) amplitudes are reduced in older listeners at high modulation rates. Two groups of observers with relatively normal hearing (younger, mean age = 25.0 years and older, mean age = 68.7 years) participated in two experiments. Experiment 1 examined speech unmasking in modulated noise as a function of masker modulation rate (16 and 32 Hz) and target speech rate (normal and 33% time compressed). Experiment 2 measured ASSR amplitudes as a function of modulation rate (32 and 128 Hz) and carrier frequency (500 and 2000 Hz). Experiment 1 indicated that older listeners show reduced speech unmasking for normal-rate speech and reduced recognition of rapid speech in steady noise. However, for rapid speech, there is no age effect for speech unmasking and no difference in the magnitude of masking release as a function of modulation rate. In general, effects of listener age and masker modulation rate on the magnitude of masking release are observed only for normal-rate speech. Experiment 2 showed that the ASSR amplitudes of older listeners are reduced for a 128-Hz modulation rate but not for a 32-Hz modulation rate, irrespective of carrier frequency. These results suggest that the reduced speech unmasking seen in older listeners for relatively slow modulation rates is not caused by deficits in envelope processing but rather is associated with the more constrained redundancy of the speech material available during the masker minima. Deficits in temporal envelope processing are evident in advanced age but only for relatively high envelope frequencies.

Age Effects in Temporal Envelope Processing: Speech Unmasking and Auditory Steady State Responses

Age Effects in Temporal Envelope Processing: Speech Unmasking and Auditory Steady State Responses

Temporal envelope processing in the human auditory cortex: Response and interconnections of auditory cortical areas

Temporal envelope processing in the human auditory cortex has an important role in language analysis. In this paper, depth recordings of local field potentials in response to amplitude modulated white noises were used to design maps of activation in primary, secondary and associative auditory areas and to study the propagation of the cortical activity between them. The comparison of activations between auditory areas was based on a signal-to-noise ratio associated with the response to amplitude modulation (AM). The functional connectivity between cortical areas was quantified by the directed coherence (DCOH) applied to auditory evoked potentials. This study shows the following reproducible results on twenty subjects: (1) the primary auditory cortex (PAC), the secondary cortices (secondary auditory cortex (SAC) and planum temporale (PT)), the insular gyrus, the Brodmann area (BA) 22 and the posterior part of T1 gyrus (T1Post) respond to AM in both hemispheres. (2) A stronger response to AM was observed in SAC and T1Post of the left hemisphere independent of the modulation frequency (MF), and in the left BA22 for MFs 8 and 16 Hz, compared to those in the right. (3) The activation and propagation features emphasized at least four different types of temporal processing. (4) A sequential activation of PAC, SAC and BA22 areas was clearly visible at all MFs, while other auditory areas may be more involved in parallel processing upon a stream originating from primary auditory area, which thus acts as a distribution hub. These results suggest that different psychological information is carried by the temporal envelope of sounds relative to the rate of amplitude modulation.

Neural Rate and Timing Cues for Detection and Discrimination of Amplitude-Modulated Tones in the Awake Rabbit Inferior Colliculus

Neural responses to amplitude-modulated (AM) tones in the unanesthetized rabbit inferior colliculus (IC) were studied in an effort to establish explicit relationships between physiological and psychophysical measures of temporal envelope processing. Specifically, responses to variations in modulation depth (m) at the cell's best modulation frequency, with and without modulation maskers, were quantified in terms of average rate and synchronization to the envelope over the entire perceptual dynamic range of depths. Statistically significant variations in the metrics were used to define neural AM detection and discrimination thresholds. Synchrony emerged at modulation depths comparable with psychophysical AM detection sensitivities in some neurons, whereas the lowest rate-based neural thresholds could not account for psychoacoustical thresholds. The majority of rate thresholds (85%) were -10 dB or higher (in 20 log m), and 16% of the population exhibited no systematic dependence of average rate on m. Neural thresholds for AM detection did not decrease systematically at higher SPLs (as observed psychophysically): thresholds remained constant or increased with level for most cells tested at multiple sound-pressure levels (SPLs). At depths higher than the rate-based detection threshold, some rate modulation-depth functions were sufficiently steep with respect to the across-trial variability of the rate to predict depth discrimination thresholds as low as 1 dB (comparable with the psychophysics). Synchrony, on the other hand, did not vary systematically with m in many cells at high modulation depths. A simple computational model was extended to reproduce several features of the modulation frequency and depth dependence of both transient and sustained pure-tone responders.

Effects of amplitude compression on first- and second-order modulation detection thresholds in cochlear implant listeners

The aim of this study was to examine the effects of instantaneous non-linear amplitude mapping on the detection of single-component and multicomponent temporal envelopes. To address this issue, first- and secondorder amplitude modulation detection thresholds were measured in four cochlear implant users with the intervention of the compression device of the implant processor. The compression device is set to produce either a strongly or a weakly logarithmic mapping of stimulus amplitude to electrical amplitude. ‘First-order’ modulation detection thresholds indicate the ability of listeners to detect sinusoidal amplitude modulation (SAM) applied to a white noise carrier; they are measured as a function of the rate of that modulation, fm. ‘Second-order’ modulation detection thresholds indicate the ability to detect sinusoidal modulation applied to the depth of a sinusoidally amplitude-modulated signal (here, a 16-Hz sinusoidally amplitude-modulated white noise); they are measured as a function of the rate of the modulation applied to the modulation depth (referred to as fm'). In each task, stimuli are transformed by the implant processor and are presented through one electrode at approximately the same level. The results show that, in cochlear implant listeners, both first- and second-order modulation detection thresholds measured at the lower rates (≤7 Hz) decrease slightly by about 3–6 dB when the stronger compression is used. No effect of compression is observed at higher rates. These results suggest that instantaneous logarithmic amplitude mapping has beneficial— but limited—effects on the detection of single-component and multicomponent temporal envelopes. These results are discussed in light of current models of temporal envelope processing.

Use of temporal envelope cues by children with developmental dyslexia.

This study evaluates the ability to process auditory temporal-envelope cues in a group of 6 children with dyslexia (mean age: 10;10 years;months). To address this issue, we measured (a) temporal modulation transfer functions (TMTFs), that is, the detection thresholds of sinusoidal amplitude modulation (SAM) applied to a white noise carrier, as a function of modulation frequency, fm (fm was 4, 16, 64, 256, and 1,024 Hz) and (b) identification performance for vowel-consonant-vowel (VCV) stimuli over 5 sessions. VCV stimuli were either unprocessed or digitally processed to remove the original spectral information, resulting in a time-varying speech envelope amplitude modulating a noise carrier. The same tests were conducted in 6 normal control children (mean age: 11;6 years;months) and 6 normal control adults (mean age: 24;8 years;months). SAM thresholds were similar in normal children and adults. For both normal groups, TMTFs were low pass in shape and showed low between-listener variability. TMTFs measured in children with dyslexia showed higher between-listener variability: TMTFs were band pass in 2 children, flat in 1 child, and low pass in the 3 others. Overall, SAM thresholds were higher in children with dyslexia than in normal children at fm = 4 and 1,024 Hz. Unprocessed-speech identification performance was nearly perfect in normal children and adults, and impaired in children with dyslexia. "Speech-envelope noise" identification performance was poorer in normal children and children with dyslexia than in normal adults. Performance improved across sessions in normal children and adults, but remained constant in children with dyslexia. Compared to normal children, children with dyslexia showed poorer reception of voicing, manner, and place of articulation for unprocessed speech and poorer reception of voicing for "speech-envelope noise." Taken together, these results support the hypothesis that some children with dyslexia may show abnormal auditory temporal-envelope processing. Such a deficit, in turn, may explain the difficulties of children with dyslexia with speech perception.

Auditory temporal envelope processing in a patient with left-hemisphere damage

Auditory temporal envelope processing in a patient with left-hemisphere damage

Auditory Temporal Envelope Processing in a Patient with Left-hemisphere Damage

Abstract Auditory temporal envelope processing was investigated in a patient showing a mild speech identification impairment following left-hemisphere damage. Three tasks evaluated the patient's ability to: (1) detect a sinusoidal amplitude modulation (SAM) applied to a white noise, as a function of modulation rate (i.e. her ‘temporal modulation transfer function’ or TMTF); (2) discriminate between two white noises amplitude modulated by time-reversed temporally asymmetric envelopes; and (3) identify white noises amplitude modulated by the temporal envelope of speech stimuli. Measurements of intensity discrimination thresholds were performed as a control task. Compared to normal data, the results obtained with the brain-damaged patient showed: (1) increased thresholds for the detection of SAM; (2) increased thresholds for the discrimination of temporal asymmetry; and (3) a deficit in the identification of speechenvelope noise stimuli. In contrast, intensity discrimination thresholds were within the normal range. Taken together, the results indicate a general impairment in auditory temporal acuity, which is now specified as a deficit in the coding of envelope rate and shape, and a deficit in the ability to use temporal envelope cues in speech processing. These results support the hypothesis that left-hemisphere damage is associated with an impairment in time analysis, which may cause, in turn, speech intelligibility disorders.

Temporal envelope processing in developmental dyslexics

This study evaluates temporal-envelope processing in six dyslexic children (mean age: 10.10 years), six normal control children (11.6 years), and six normal control adults (24.8 years) by measuring: (1) temporal modulation transfer functions (TMTFs), i.e., the detection thresholds of sinusoidal amplitude-modulation applied to a white noise carrier, as a function of modulation frequency (4, 16, 64, 256, or 1024 Hz), (2) recognition performance for vowel-consonant-vowel stimuli over five sessions; speech stimuli were either unprocessed or processed to degrade the spectral information. Modulation thresholds were similar in normal children and adults; for both groups, TMTFs were low pass in shape. TMTFs measured in dyslexics were bandpass in four children, low pass and flat in the two others. Overall, modulation thresholds were higher in dyslexics than in normal children at 4 and 1024 Hz. Unprocessed-speech recognition performance was perfect in normal children and adults, and impaired in dyslexics. Processed-speech recognition performance was poorer in normal and dyslexic children compared to normal adults. Performance improved across sessions in normal children and adults, but remained constant in dyslexics. These results support the hypothesis that some dyslexics show abnormal auditory temporal-envelope processing. This may, in turn, explain their difficulties with speech perception. [Work supported by the Cognitique program.]

Amplitude-modulation detection in two patients with left- or right-hemisphere damage

The modulation depth required for the detection of sinusoidal amplitude-modulation applied to a white noise carrier was measured as a function of modulation frequency, giving temporal modulation transfer functions (TMTFs). The modulation frequency was 4, 16, 64, 256, or 1024 Hz. Measurements of intensity discrimination thresholds for white noise were performed as a control task. In both tasks, white noise stimuli had a 2-s duration and were presented monaurally at 75 dB SPL. Two adult listeners with left- or right-hemisphere damage and three normal-control adult listeners were tested in each ear. Lesions encompassed the primary auditory cortex in the two patients. All listeners had normal audiometric thresholds. The modulation detection data showed no sign of ear asymmetry. For all listeners, the TMTFs displayed a typical low-pass characteristic. Modulation detection was 10–14 dB poorer-than-normal at modulation frequencies between 4 and 256 Hz in the patient with left-hemisphere damage. In contrast, modulation detection was normal in the patient with right-hemisphere damage. Intensity discrimination was within the normal range (0.5–1 dB) for both patients. These results support the hypothesis that cortical structures within the left hemisphere play a crucial role in temporal envelope processing. [Work supported by the Fondation de l’Avenir and the Cognitique Program.]

Auditory temporal envelope processing in a patient with left-hemisphere damage

Auditory temporal envelope processing was investigated in a patient showing a mild speech identification impairment following left-hemisphere damage. Three tasks evaluated the patient's ability to: (1) detect a sinusoidal amplitude modulation (SAM) applied to a white noise, as a function of modulation rate (i.e. her ‘temporal modulation transfer function’ or TMTF); (2) discriminate between two white noises amplitude modulated by time-reversed temporally asymmetric envelopes; and (3) identify white noises amplitude modulated by the temporal envelope of speech stimuli. Measurements of intensity discrimination thresholds were performed as a control task. Compared to normal data, the results obtained with the brain-damaged patient showed: (1) increased thresholds for the detection of SAM; (2) increased thresholds for the discrimination of temporal asymmetry; and (3) a deficit in the identification of speechenvelope noise stimuli. In contrast, intensity discrimination thresholds were within the normal range. Taken together, the results indicate a general impairment in auditory temporal acuity, which is now specified as a deficit in the coding of envelope rate and shape, and a deficit in the ability to use temporal envelope cues in speech processing. These results support the hypothesis that left-hemisphere damage is associated with an impairment in time analysis, which may cause, in turn, speech intelligibility disorders.

The effect of amplitude comodulation on auditory object formation in sentence perception.

To comprehend speech in most environments, listeners must combine some but not all sounds from across a wide range of frequencies. Three experiments were conducted to examine the role of amplitude comodulation in performing an essential part of this function: the grouping together of the simultaneous components of a speech signal. Each of the experiments used time-varying sinusoidal (TVS) sentences (Remez, Rubin, Pisoni, & Carrell, 1981) as base stimuli because their component tones are acoustically unrelated. The independence of the three tones reduced the number of confounding grouping cues available compared with those found in natural or computer-synthesized speech (e.g., fundamental frequency and simultaneity of harmonic onset). In each of the experiments, the TVS base stimuli were amplitude modulated to determine whether this modulation would lead to appropriate grouping of the three tones as reflected by sentence intelligibility. Experiment 1 demonstrated that amplitude comodulation at 100 Hz did improve the intelligibility of TVS sentences. Experiment 2 showed that the component tones of a TVS sentence must be comodulated (as opposed to independently modulated) for improvements in intelligibility to be found. Experiment 3 showed that the comodulation rates that led to intelligibility improvements were consistent with the effective rates found in experiments that examined the grouping of complex nonspeech sounds by common temporal envelopes (e.g., comodulation masking release; Hall, Haggard, & Fernandes, 1984). The results of these experiments support the claim that certain basic temporal-envelope processing capabilities of the human auditory system contribute to the perception of fluent speech.

Auditory psychophysical performance without a cochlea

Auditory psychophysical performance has been measured using electrical stimulation of the remaining VIII nerve and of the cochlear nucleus in deaf patients. Psychophysical measures of temporal envelope processing show relatively unimpaired performance in these patients compared to normal hearing, and speech discrimination scores indicate that speech information relating to temporal envelopes can be effectively transmitted and received. This finding also indicates that the cochlea and VIII nerve may play relatively little role in the following tasks: detection of gaps, detection of modulation, recovery from adaptation, nonspectral pitch discrimination, and duration discrimination. Intensity perception is impaired with electrical stimulation: dynamic range of usable loudness is only 10 to 20 dB and intensity discrimination experiments indicate only 10 to 40 discriminable intensity levels. Frequency resolution is completely absent in electrical stimulation and can be only crudely reconstructed by multiple electrodes stimulating discrete neural segments. Psychophysical experiments on multiple electrodes indicate that patients can perceive and discriminate complex dynamic patterns of electrical activity changing in both stimulation frequency and electrode location. The combination of temporal envelope formation and the coarse “frequency” resolution provided by multiple electrodes is adequate to convey a surprising amount of speech information. [Work supported by NIH.]