Role Of Temporal Fine Structure Research Articles

Role of Temporal Fine Structure and Envelope in Identifying Indian Ragas- a Prospective Study for Temporal-Based Raga Classifier

Role of Temporal Fine Structure and Envelope in Identifying Indian Ragas- a Prospective Study for Temporal-Based Raga Classifier

Temporal fine structure influences voicing confusions for consonant identification in multi-talker babble.

To understand the mechanisms of speech perception in everyday listening environments, it is important to elucidate the relative contributions of different acoustic cues in transmitting phonetic content. Previous studies suggest that the envelope of speech in different frequency bands conveys most speech content, while the temporal fine structure (TFS) can aid in segregating target speech from background noise. However, the role of TFS in conveying phonetic content beyond what envelopes convey for intact speech in complex acoustic scenes is poorly understood. The present study addressed this question using online psychophysical experiments to measure the identification of consonants in multi-talker babble for intelligibility-matched intact and 64-channel envelope-vocoded stimuli. Consonant confusion patterns revealed that listeners had a greater tendency in the vocoded (versus intact) condition to be biased toward reporting that they heard an unvoiced consonant, despite envelope and place cues being largely preserved. This result was replicated when babble instances were varied across independent experiments, suggesting that TFS conveys voicing information beyond what is conveyed by envelopes for intact speech in babble. Given that multi-talker babble is a masker that is ubiquitous in everyday environments, this finding has implications for the design of assistive listening devices such as cochlear implants.

Investigating the role of temporal fine structure in everyday hearing

Human listeners can derive substantial masking release when there are discrepancies in pitch or spatial location between the target and masking sounds. While the temporal fine-structure (TFS) in low-frequency sounds can convey information about both pitch and location, a nuanced debate exists in the literature about the role of these TFS cues in masking release. The long-term goal of the present study is to leverage individual differences to understand the role of TFS in everyday hearing. As a first step, we sought to measure individual TFS sensitivity using monaural frequency modulation (FM) and binaural interaural time difference (ITD) detection tasks. Preliminary data show large individual differences in these measures. Moreover, individual differences in ITD sensitivity were correlated with monaural FM sensitivity suggesting that monaural TFS coding can be a primary bottleneck determining binaural sensitivity. Alternately, both FM and ITD sensitivity variations could be reflecting common non-sensory factors (e.g., attention). To disambiguate between these hypotheses, we designed two passive EEG metrics of TFS coding. Follow-up experiments will compare individual differences in these perceptual and EEG measures to each other, and to speech-in-noise perception in complex environments.

A two-path model of auditory modulation detection using temporal fine structure and envelope cues.

A model using temporal-envelope cues was previously developed to explain perceptual interference effects between amplitude modulation and frequency modulation (FM). As that model could not accurately predict FM sensitivity and the interference effects, temporal fine structure (TFS) cues were added to the model. Thus, following the initial stage of the model consisting of a linear filter bank simulating cochlear filtering, processing was split into an 'envelope path' based on envelope power cues and a 'TFS path' based on a measure of the distribution of time intervals between successive zero-crossings. This yielded independent detectability indices for envelope and TFS cues, which were optimally combined to produce a single decision statistic. Independent internal noises in the envelope and TFS paths were adjusted to match the data. Simulations indicate that TFS cues are required to account for FM data for young normal-hearing listeners and that TFS processing is impaired for both older normal-hearing and hearing-impaired listeners. The role of TFS was further assessed by relating the monaural FM sensitivity to measures of interaural phase difference, commonly assumed to rely on binaural TFS sensitivity. The model demonstrates that binaural TFS sensitivity is considerably lower than monaural TFS sensitivity. Similar to FM thresholds, interaural phase difference sensitivity declined with age and hearing loss, although higher degradations were observed in binaural temporal processing compared to monaural processing. Overall, this model provides a novel tool to explore the mechanisms involved in FM processing in the normal auditory system and the degradations in FM sensitivity with ageing and hearing loss.

The role of temporal fine structure in tone recognition and music perception

The sound signal can be decomposed into temporal envelope and temporal fine structure information. The temporal envelope information is crucial for speech perception in quiet environment, and the temporal fine structure information plays an important role in speech perception in noise, Mandarin tone recognition and music perception, especially the pitch and melody perception.

Relating age and hearing loss to monaural, bilateral, and binaural temporal sensitivity.

Older listeners are more likely than younger listeners to have difficulties in making temporal discriminations among auditory stimuli presented to one or both ears. In addition, the performance of older listeners is often observed to be more variable than that of younger listeners. The aim of this work was to relate age and hearing loss to temporal processing ability in a group of younger and older listeners with a range of hearing thresholds. Seventy-eight listeners were tested on a set of three temporal discrimination tasks (monaural gap discrimination, bilateral gap discrimination, and binaural discrimination of interaural differences in time). To examine the role of temporal fine structure in these tasks, four types of brief stimuli were used: tone bursts, broad-frequency chirps with rising or falling frequency contours, and random-phase noise bursts. Between-subject group analyses conducted separately for each task revealed substantial increases in temporal thresholds for the older listeners across all three tasks, regardless of stimulus type, as well as significant correlations among the performance of individual listeners across most combinations of tasks and stimuli. Differences in performance were associated with the stimuli in the monaural and binaural tasks, but not the bilateral task. Temporal fine structure differences among the stimuli had the greatest impact on monaural thresholds. Threshold estimate values across all tasks and stimuli did not show any greater variability for the older listeners as compared to the younger listeners. A linear mixed model applied to the data suggested that age and hearing loss are independent factors responsible for temporal processing ability, thus supporting the increasingly accepted hypothesis that temporal processing can be impaired for older compared to younger listeners with similar hearing and/or amounts of hearing loss.

Dual-carrier vocoder: Evidence of a primary role of temporal fine structure in streaming

Thus far, two possible roles of temporal fine structure (TFS) have been suggested for speech recognition. A first role is to provide acoustic speech information. A second role is to assist in identifying which auditory channels are dominated by the target signal so that the output of these channels can be combined at a later stage to reconstruct the internal representation of that target. Our most recent work has been largely in contradiction with the speech-information hypothesis, as we generally observe that normal-hearing (NH) listeners do not rely on the TFS of the target speech signal to obtain speech information. However, direct evidence that NH listeners do rely on the TFS to extract the target speech signal from the background is still lacking. The present study was designed to provide such evidence. A dual-carrier vocoder was implemented to assess the role of TFS cues in streaming. To our knowledge, this is the only strategy allowing TFS cues to be provided without transmitting speech information. Results showed that NH listeners can achieve sentence recognition scores comparable to that obtain with the original TFS (i.e., unprocessed), suggesting a primary role of TFS cues in streaming. Implications for cochlear implants are discussed.

Role of temporal fine structure in speech recognition: From psychoacoustics to cochlear implants

The path from fundamental research to real-world application is often long and sinuous. This presentation will depict how psychoacoustic studies performed in Sid Bacon's laboratory have engendered our original view of the role of temporal fine structure (TFS) in speech recognition, and how this view has in turn led to the development of an extremely effective speech-processing strategy for cochlear implants. First, psychoacoustic data showing how normal-hearing listeners can detect amplitude modulation when presented with only the TFS will be described. Second, a series of experiments illustrating the “absence” of acoustic speech cues in the TFS will be briefly presented. Finally, the results of a recent study involving a dual-carrier vocoder will be described. These results suggest that TFS cues are primarily used to assist in identifying which auditory channels are dominated by the target signal so that the output of these channels can be combined at a later stage to reconstruct the internal representation of that target. They also indicate that cochlear implants implementing a speech-processing strategy based on the “dual-carrier strategy” have the potential to restore nearly perfect speech intelligibility in noise. [Work supported by NIH.]

Revisiting place and temporal theories of pitch

The nature of pitch and its neural coding have been studied for over a century. A popular debate has revolved around the question of whether pitch is coded via "place" cues in the cochlea, or via timing cues in the auditory nerve. In the most recent incarnation of this debate, the role of temporal fine structure has been emphasized in conveying important pitch and speech information, particularly because the lack of temporal fine structure coding in cochlear implants might explain some of the difficulties faced by cochlear implant users in perceiving music and pitch contours in speech. In addition, some studies have postulated that hearing-impaired listeners may have a specific deficit related to processing temporal fine structure. This article reviews some of the recent literature surrounding the debate, and argues that much of the recent evidence suggesting the importance of temporal fine structure processing can also be accounted for using spectral (place) or temporal-envelope cues.

On the possible influence of spectral- and temporal-envelope cues in tests of sensitivity to temporal fine structure.

The role of temporal fine structure (TFS) in speech and pitch perception has attracted considerable attention. Recent studies have suggested sensitivity to TFS even at high frequencies (8 kHz), well beyond the known limits of phase-locking in mammals, and a reduced ability to use TFS in hearing-impaired listeners. These conclusions were based on tasks involving the discrimination of complex tones or iterated-rippled noise, which had different power spectra but were (a) bandpass-filtered in a way that reduced tonotopic excitation-pattern (EP) cues and (b) had random component phases to eliminate temporal-envelope (TE) cues. In this study, we examine the possibility that residual EP and/or TE cues may have been available to the listeners in these experiments. Analytical and computational analyses indicated that although systematic TE differences were absent at the level of the stimuli, they were likely present at the output of the cochlea. Empirical studies tested whether the available TE and EP information might have influenced performance. Preliminary results suggest that although listeners may not have been sensitive to the available TE cues, performance may have been influenced by EP cues in a way that could also explain the deficits shown by hearing-impaired listeners. [Work supported by NIH R01 DC05216.]

The role of temporal fine structure in harmonic segregation through mistuning

Bernstein and Oxenham [(2008). J. Acoust. Soc. Am. 124, 1653-1667] measured thresholds for discriminating the fundamental frequency, F0, of a complex tone that was passed through a fixed bandpass filter. They found that performance worsened when the F0 was decreased so that only harmonics above the tenth were audible. However, performance in this case was improved by mistuning the odd harmonics by 3%. Bernstein and Oxenham considered whether the results could be explained in terms of temporal fine structure information available at the output of a single auditory filter and concluded that their results did not appear to be consistent with such an explanation. Here, it is argued that such cues could have led to the improvement in performance produced by mistuning the odd harmonics.

Role of temporal fine structure in speech perception

Recent studies have shown that lack of access to the temporal fine structure cue is a major reason for the difficulty in speech perception in noise by hearing‐impaired listeners. To further understand the role of temporal fine structure, we need to define the temporal fine structure and to delineate its relationship to the temporal envelope in both acoustical and perceptual domains. This talk will first examine the relationship between temporal envelope and temporal fine structure in signal processing terms and then relate it to speech production and perception. Acoustically, the temporal fine structure primarily contributes to changes in fundamental frequency, harmonics, and formant transition. Perceptually, while the temporal fine structure can contribute to speech intelligibility via the formant transition cue, it contributes to speech perception in noise by enhancing auditory objection formation rather than increasing speech intelligibility directly.

The role of temporal fine structure in speech source segregation

Normal‐hearing (NH) listeners show better speech recognition when a stationary noise masker is replaced by an opposite‐gender competing talker at the same signal‐to‐noise ratio (SNR). Hearing‐impaired (HI) listeners often do not show this interfering‐talker benefit (ITB). This may be due to a reduced ability to use temporal fine structure (TFS). Consistent with this idea, NH listeners also show little ITB when TFS is removed. We hypothesized that TFS underlies the ITB by providing source‐segregation cues. To test this hypothesis, non‐auditory segregation cues were introduced in the form of a video of the talker's face. Speech intelligibility was estimated in NH listeners as a function of SNR for sentences spoken by a female talker and masked by speech‐spectrum shaped stationary noise or a single male talker. Target and masker were summed before processing by a 15‐channel noise vocoder to remove TFS, and presented with or without accompanying video. Without video, listeners received little ITB, consistent with previous results. Auditory‐visual conditions yielded as much as 9 dB of ITB, supporting the hypothesis that a diminished ability to perceptually segregate sources contributes to the lack of ITB in the absence of TFS. Similar results were obtained for unprocessed speech in HI listeners.

Effects of temporal fine structure on the lateralization of speech and on speech understanding in noise.

This study evaluated the role of temporal fine structure in the lateralization and understanding of speech in six normal-hearing listeners. Interaural time differences (ITDs) were introduced to invoke lateralization. Speech reception thresholds (SRTs) were evaluated in backgrounds of two-talker babble and speech-shaped noise. Two-syllable words with ITDs of 0 and 700 micros were used as targets. A vocoder technique, which systematically randomized fine structure, was used to evaluate the effects of fine structure on these tasks. Randomization of temporal fine structure was found to significantly reduce the ability of normal-hearing listeners to lateralize words, although for many listeners, good lateralization performance was achieved with as much as 80% fine-structure randomization. Most listeners demonstrated some rudimentary ability to lateralize with 100% fine-structure randomization. When ITDs were 0 micros, randomization of fine structure had a much greater effect on SRT in two-talker babble than in speech-shaped noise. Binaural advantages were also observed. In steady noise, the difference in SRT between words with 0- vs 700-micros ITDs was, on average, 6 dB with no fine-structure randomization and 2 dB with 100% fine-structure randomization. In two-talker babble this difference was 1.9 dB and, for most listeners, showed little effect of the degree of fine-structure randomization. These results suggest that (1) improved delivery of temporal fine structure would improve speech understanding in noise for implant recipients, (2) bilateral implant recipients might benefit from temporal envelope ITDs, and (3) improved delivery of temporal information could improve binaural benefits.

Speech and melody recognition in binaurally combined acoustic and electric hearing

Speech recognition in noise and music perception is especially challenging for current cochlear implant users. The present study utilizes the residual acoustic hearing in the nonimplanted ear in five cochlear implant users to elucidate the role of temporal fine structure at low frequencies in auditory perception and to test the hypothesis that combined acoustic and electric hearing produces better performance than either mode alone. The first experiment measured speech recognition in the presence of competing noise. It was found that, although the residual low-frequency (<1000 Hz) acoustic hearing produced essentially no recognition for speech recognition in noise, it significantly enhanced performance when combined with the electric hearing. The second experiment measured melody recognition in the same group of subjects and found that, contrary to the speech recognition result, the low-frequency acoustic hearing produced significantly better performance than the electric hearing. It is hypothesized that listeners with combined acoustic and electric hearing might use the correlation between the salient pitch in low-frequency acoustic hearing and the weak pitch in the envelope to enhance segregation between signal and noise. The present study suggests the importance and urgency of accurately encoding the fine-structure cue in cochlear implants.