F0 Discrimination Research Articles

Implications of Within-Fiber Temporal Coding for Perceptual Studies of F0 Discrimination and Discrimination of Harmonic and Inharmonic Tone Complexes

Recent psychophysical studies suggest that normal-hearing (NH) listeners can use acoustic temporal-fine-structure (TFS) cues for accurately discriminating shifts in the fundamental frequency (F0) of complex tones, or equal shifts in all component frequencies, even when the components are peripherally unresolved. The present study quantified both envelope (ENV) and TFS cues in single auditory-nerve (AN) fiber responses (henceforth referred to as neural ENV and TFS cues) from NH chinchillas in response to harmonic and inharmonic complex tones similar to those used in recent psychophysical studies. The lowest component in the tone complex (i.e., harmonic rank N) was systematically varied from 2 to 20 to produce various resolvability conditions in chinchillas (partially resolved to completely unresolved). Neural responses to different pairs of TEST (F0 or frequency shifted) and standard or reference (REF) stimuli were used to compute shuffled cross-correlograms, from which cross-correlation coefficients representing the degree of similarity between responses were derived separately for TFS and ENV. For a given F0 shift, the dissimilarity (TEST vs. REF) was greater for neural TFS than ENV. However, this difference was stimulus-based; the sensitivities of the neural TFS and ENV metrics were equivalent for equal absolute shifts of their relevant frequencies (center component and F0, respectively). For the F0-discrimination task, both ENV and TFS cues were available and could in principle be used for task performance. However, in contrast to human performance, neural TFS cues quantified with our cross-correlation coefficients were unaffected by phase randomization, suggesting that F0 discrimination for unresolved harmonics does not depend solely on TFS cues. For the frequency-shift (harmonic-versus-inharmonic) discrimination task, neural ENV cues were not available. Neural TFS cues were available and could in principle support performance in this task; however, in contrast to human-listeners' performance, these TFS cues showed no dependence on N. We conclude that while AN-fiber responses contain TFS-related cues, which can in principle be used to discriminate changes in F0 or equal shifts in component frequencies of peripherally unresolved harmonics, performance in these two psychophysical tasks appears to be limited by other factors (e.g., central processing noise).

The perception of prosody and associated auditory cues in early-implanted children: The role of auditory working memory and musical activities

Objective: To study prosodic perception in early-implanted children in relation to auditory discrimination, auditory working memory, and exposure to music. Design: Word and sentence stress perception, discrimination of fundamental frequency (F0), intensity and duration, and forward digit span were measured twice over approximately 16 months. Musical activities were assessed by questionnaire. Study sample: Twenty-one early-implanted and age-matched normal-hearing (NH) children (4–13 years). Results: Children with cochlear implants (CIs) exposed to music performed better than others in stress perception and F0 discrimination. Only this subgroup of implanted children improved with age in word stress perception, intensity discrimination, and improved over time in digit span. Prosodic perception, F0 discrimination and forward digit span in implanted children exposed to music was equivalent to the NH group, but other implanted children performed more poorly. For children with CIs, word stress perception was linked to digit span and intensity discrimination: sentence stress perception was additionally linked to F0 discrimination. Conclusions: Prosodic perception in children with CIs is linked to auditory working memory and aspects of auditory discrimination. Engagement in music was linked to better performance across a range of measures, suggesting that music is a valuable tool in the rehabilitation of implanted children.

Timbre-induced pitch shift from the perspective of Signal Detection Theory: the impact of musical expertise, silence interval, and pitch region

The paradigm of Signal Detection Theory (SDT) was used to analyze the ability of professional pianists (N = 16) and string players (N = 15) to discriminate small F0 differences between consecutive musical tones, presented in pairs, with identical and with different (bright and dull) timbres. The sensitivity (d′) and response bias (c) were heavily dependent on the timbral arrangement of the pairs of tones (the “comparable tones”), which can be interpreted as the influence of timbre-induced pitch shift on F0 discrimination. The participants were somewhat biased to “miss” signals when comparable tones had identical timbres and to make “false alarms” when the tones had different timbres. The d′ was lowest when the tones with a lower F0 in those stimulus-pairs containing tones with different timbres had a brighter timber, and highest when both tones had bright timbre. On average, the string players had a somewhat higher d′ and their perception was slightly less influenced by timbre-induced pitch shift when compared to the pianists. Nevertheless, the dependence of d′ and c on the timbral arrangement of the tones was registered in the case of all the participants at all the investigated pitch regions around D#3, D4, and C#5. Furthermore, the presence of a silence of 3.5 s—a silence interval—between the tones to be compared had an impact on both d′- and c-values as well as on the degree of vulnerability to timbre-induced pitch shift.

Pitch Perception of Medium-Rank Harmonic Complex Tones Based on Temporal Analysis

Fundamental frequency difference limens were measured for a target harmonic complex tone (HCT) in the absence and presence of a masker HCT, which were filtered into the same bandpass frequency region and were gated on and off synchronously. There were three kinds of nominal fundamental frequencies (F0s) for target (200, 400, and 800 Hz), five kinds of F0 separations between target and masker (0, ±3, and ±6 semitones), and four kinds of phase combinations. Results found significant effects of nominal F0, phase combination, and F0 separation between target and masker. Analysis based on temporal profile proved that the significant effect of nominal F0 could be explained by peak height of target, and that the significant effects of F0 separation and phase combination could be explained by the ratio of temporal peak heights between target and masker. Thus it is suggested that F0 discrimination of medium-rank harmonics probably depends on the use of temporal fine structure information.

Distortion products and their influence on representation of pitch-relevant information in the human brainstem for unresolved harmonic complex tones

Pitch experiments aimed at evaluating temporal pitch mechanism(s) often utilize complex sounds with only unresolved harmonic components, and a low-pass noise masker to eliminate the potential contribution of audible distortion products to the pitch percept. Herein we examine how: (i) masker induced reduction of neural distortion products (difference tone: DT; and cubic difference tone: CDT) alters the representation of pitch relevant information in the brainstem; and (ii) the pitch salience is altered when distortion products are reduced and/or eliminated. Scalp recorded brainstem frequency following responses (FFR) were recorded in normal hearing individuals using a complex tone with only unresolved harmonics presented in quiet, and in the presence of a low-pass masker at SNRs of +15, +5, and −5 dB. Difference limen for F0 discrimination (F0 DL) was obtained in quiet and in the presence of low-pass noise. Magnitude of DT components (with the exception of components at F0 and 2F0), and the CDT components decreased with increasing masker level. Neural pitch strength decreased with increasing masker level for both the envelope-related (FFRENV) and spectral-related (FFRSPEC) phase-locked activity. Finally, F0 DLs increased with decreasing SNRs suggesting poorer F0 discrimination with reduction of the distortion products. Collectively, these findings support the notion that both DT and CDT, as reflected in the FFRENV and FFRSPEC, respectively, influence both the brainstem representation of pitch relevant information and the pitch salience of the complex sounds.

Sensitivity of school-aged children to pitch-related cues

Two experiments investigated the ability of 17 school-aged children to process purely temporal and spectro-temporal cues that signal changes in pitch. Percentage correct was measured for the discrimination of sinusoidal amplitude modulation rate (AMR) of broadband noise in experiment 1 and for the discrimination of fundamental frequency (F0) of broadband sine-phase harmonic complexes in experiment 2. The reference AMR was 100 Hz as was the reference F0. A child-friendly interface helped listeners to remain attentive to the task. Data were fitted using a maximum-likelihood technique that extracted threshold, slope, and lapse rate. All thresholds were subsequently standardized to a common d' value equal to 0.77. There were relatively large individual differences across listeners: eight had relatively adult-like thresholds in both tasks and nine had higher thresholds. However, these individual differences did not vary systematically with age, over the span of 6-16 yr. Thresholds were correlated across the two tasks and were about nine times finer for F0 discrimination than for AMR discrimination as has been previously observed in adults.

The effect of hearing loss on the resolution of partials and fundamental frequency discrimination

The relationship between the ability to hear out partials in complex tones, discrimination of the fundamental frequency (F0) of complex tones, and frequency selectivity was examined for subjects with mild-to-moderate cochlear hearing loss. The ability to hear out partials was measured using a two-interval task. Each interval included a sinusoid followed by a complex tone; one complex contained a partial with the same frequency as the sinusoid, whereas in the other complex that partial was missing. Subjects had to indicate the interval in which the partial was present in the complex. The components in the complex were uniformly spaced on the ERB(N)-number scale. Performance was generally good for the two "edge" partials, but poorer for the inner partials. Performance for the latter improved with increasing spacing. F0 discrimination was measured for a bandpass-filtered complex tone containing low harmonics. The equivalent rectangular bandwidth (ERB) of the auditory filter was estimated using the notched-noise method for center frequencies of 0.5, 1, and 2 kHz. Significant correlations were found between the ability to hear out inner partials, F0 discrimination, and the ERB. The results support the idea that F0 discrimination of tones with low harmonics depends on the ability to resolve the harmonics.

The effect of hearing loss on the resolution of partials and fundamental frequency discrimination.

The relationship between the ability to hear out partials in complex tones, discrimination of the fundamental frequency (F0) of complex tones, and frequency selectivity was examined for subjects with mild-to-moderate cochlear hearing loss. The ability to hear out partials was measured using a two-interval task. Each interval included a sinusoid followed by a complex tone; one complex contained a partial with the same frequency as the sinusoid, while in the other complex that partial was missing. Subjects had to indicate the interval in which the partial was present in the complex. The components in the complex were uniformly spaced on the ERBN-number scale. Performance was generally good for the two “edge” partials, but poorer for the inner partials. Performance for the latter improved with increasing spacing. F0 discrimination was measured for a bandpass-filtered complex tone containing low harmonics. The equivalent rectangular bandwidth (ERB) of the auditory filter was estimated using the notched-noise method for center frequencies of 0.5, 1, and 2 kHz. Significant correlations were found between the ability to hear out inner partials, F0 discrimination, and the ERB. The results support the idea that F0 discrimination of tones with low harmonics depends on the ability to resolve the harmonics.

Effects of Age on F0 Discrimination and Intonation Perception in Simulated Electric and Electroacoustic Hearing

Recent research suggests that older listeners may have difficulty processing information related to the fundamental frequency (F0) of voiced speech. In this study, the focus was on the mechanisms that may underlie this reduced ability. We examined whether increased age resulted in decreased ability to perceive F0 using fine-structure cues provided by the harmonic structure of voiced speech sounds or cues provided by high-rate envelope fluctuations (periodicity). Younger listeners with normal hearing and older listeners with normal to near-normal hearing completed two tasks of F0 perception. In the first task (steady state F0), the fundamental frequency difference limen (F0DL) was measured adaptively for synthetic vowel stimuli. In the second task (time-varying F0), listeners relied on variations in F0 to judge intonation of synthetic diphthongs. For both tasks, three processing conditions were created: eight-channel vocoding that preserved periodicity cues to F0; a simulated electroacoustic stimulation condition, which consisted of high-frequency vocoder processing combined with a low-pass-filtered portion, and offered both periodicity and fine-structure cues to F0; and an unprocessed condition. F0 difference limens for steady state vowel sounds and the ability to discern rising and falling intonations were significantly worse in the older subjects compared with the younger subjects. For both older and younger listeners, scores were lowest for the vocoded condition, and there was no difference in scores between the unprocessed and electroacoustic simulation conditions. Older listeners had difficulty using periodicity cues to obtain information related to talker fundamental frequency. However, performance was improved by combining periodicity cues with (low frequency) acoustic information, and that strategy should be considered in individuals who are appropriate candidates for such processing. For cochlear implant candidates, this effect might be achieved by partial electrode insertion providing acoustic stimulation in the low frequencies or by the combination of a traditional implant in one ear and a hearing aid in the opposite ear.

Perceptual grouping affects pitch judgments across time and frequency.

Pitch, the perceptual correlate of fundamental frequency (F0), plays an important role in speech, music, and animal vocalizations. Changes in F0 over time help define musical melodies and speech prosody, while comparisons of simultaneous F0 are important for musical harmony, and for segregating competing sound sources. This study compared listeners' ability to detect differences in F0 between pairs of sequential or simultaneous tones that were filtered into separate, nonoverlapping spectral regions. The timbre differences induced by filtering led to poor F0 discrimination in the sequential, but not the simultaneous, conditions. Temporal overlap of the two tones was not sufficient to produce good performance; instead performance appeared to depend on the two tones being integrated into the same perceptual object. The results confirm the difficulty of comparing the pitches of sequential sounds with different timbres and suggest that, for simultaneous sounds, pitch differences may be detected through a decrease in perceptual fusion rather than an explicit coding and comparison of the underlying F0s.

Perceptual learning of fundamental frequency discrimination: Effects of fundamental frequency, harmonic number, and component phase

Thresholds (F0DLs) were measured for discrimination of the fundamental frequency (F0) of a group of harmonics (group B) embedded in harmonics with a fixed F0. Miyazono and Moore [(2009). Acoust. Sci. & Tech. 30, 383386] found a large training effect for tones with high harmonics in group B, when the harmonics were added in cosine phase. It is shown here that this effect was due to use of a cue related to pitch pulse asynchrony (PPA). When PPA cues were disrupted by introducing a temporal offset between the envelope peaks of the harmonics in group B and the remaining harmonics, F0DLs increased markedly. Perceptual learning was examined using a training stimulus with cosine-phase harmonics, F0 = 50 Hz, and high harmonics in group B, under conditions where PPA was not useful. Learning occurred, and it transferred to other cosine-phase tones, but not to random-phase tones. A similar experiment with F0 = 100 Hz showed a learning effect which transferred to a cosine-phase tone with mainly high unresolved harmonics, but not to cosine-phase tones with low harmonics, and not to random-phase tones. The learning found here appears to be specific to tones for which F0 discrimination is based on distinct peaks in the temporal envelope.

Does fundamental-frequency discrimination measure virtual pitch discrimination?

Studies of pitch perception often involve measuring difference limens for complex tones (DLCs) that differ in fundamental frequency (F0). These measures are thought to reflect F0 discrimination and to provide an indirect measure of subjective pitch strength. However, in many situations discrimination may be based on cues other than the pitch or the F0, such as differences in the frequencies of individual components or timbre (brightness). Here, DLCs were measured for harmonic and inharmonic tones under various conditions, including a randomized or fixed lowest harmonic number, with and without feedback. The inharmonic tones were produced by shifting the frequencies of all harmonics upwards by 6.25%, 12.5%, or 25% of F0. It was hypothesized that, if DLCs reflect residue-pitch discrimination, these frequency-shifted tones, which produced a weaker and more ambiguous pitch than would yield larger DLCs than the harmonic tones. However, if DLCs reflect comparisons of component pitches, or timbre, they should not be systematically influenced by frequency shifting. The results showed larger DLCs and more scattered pitch matches for inharmonic than for harmonic complexes, confirming that the inharmonic tones produced a less consistent pitch than the harmonic tones, and consistent with the idea that DLCs reflect F0 pitch discrimination.

Combining information across frequency regions in fundamental frequency discrimination.

Sensitivity to fundamental frequency (F0) differences was measured for two complex tones, A and B, which had the same F0 but were filtered into two different frequency regions. Tones were presented either alone or together. A signal-detection analysis was used to predict effects of combining F0 information across frequency regions. For 400-ms tones containing only unresolved harmonics, the first experiment showed that performance (in terms of d(')) for the combined presentation was better than for the isolated tones but was not optimal (assuming independent channels and noises) and was independent of the relative timing of pulses in the envelopes of tones A and B (varied by changing the starting phase of components of tone B relative to those of tone A). The nonoptimal performance was shown not to be due to peripheral masking (experiment II), or to listeners paying attention mainly to one frequency region (experiment III), nor was it specific to conditions where all harmonics were unresolved (experiment IV). In contrast, optimal performance in F0 discrimination for combined presentation was observed for 50-ms tones (experiment V). The results may reflect the limited ability of the human auditory system to integrate information simultaneously in the time and the frequency domains.

Can temporal fine structure represent the fundamental frequency of unresolved harmonics?

At least two modes of pitch perception exist: in one, the fundamental frequency (F0) of harmonic complex tones is estimated using the temporal fine structure (TFS) of individual low-order resolved harmonics; in the other, F0 is derived from the temporal envelope of high-order unresolved harmonics that interact in the auditory periphery. Pitch is typically more accurate in the former than in the latter mode. Another possibility is that pitch can sometimes be coded via the TFS from unresolved harmonics. A recent study supporting this third possibility [Moore et al. (2006a). J. Acoust. Soc. Am. 119, 480-490] based its conclusion on a condition where phase interaction effects (implying unresolved harmonics) accompanied accurate F0 discrimination (implying TFS processing). The present study tests whether these results were influenced by audible distortion products. Experiment 1 replicated the original results, obtained using a low-level background noise. However, experiments 2-4 found no evidence for the use of TFS cues with unresolved harmonics when the background noise level was raised, or the stimulus level was lowered, to render distortion inaudible. Experiment 5 measured the presence and phase dependence of audible distortion products. The results provide no evidence that TFS cues are used to code the F0 of unresolved harmonics.

Fundamental frequency discrimination and speech perception in noise in cochlear implant simulations

Increasing the number of channels at low frequencies improves discrimination of fundamental frequency (F0) in cochlear implants (Geurts, L., Wouters, J., 2004. Better place-coding of the fundamental frequency in cochlear implants. J. Acoust. Soc. Am. 115 (2), 844–852). We conducted three experiments to test whether improved F0 discrimination can be translated into increased speech intelligibility in noise in a cochlear implant simulation. The first experiment measured F0 discrimination and speech intelligibility in quiet as a function of channel density over different frequency regions. The results from this experiment showed a tradeoff in performance between F0 discrimination and speech intelligibility with a limited number of channels. The second experiment tested whether improved F0 discrimination and optimizing this tradeoff could improve speech performance with a competing talker. However, improved F0 discrimination did not improve speech intelligibility in noise. The third experiment identified the critical number of channels needed at low frequencies to improve speech intelligibility in noise. The result showed that, while 16 channels below 500 Hz were needed to observe any improvement in speech intelligibility in noise, even 32 channels did not achieve normal performance. Theoretically, these results suggest that without accurate spectral coding, F0 discrimination and speech perception in noise are two independent processes. Practically, the present results illustrate the need to increase the number of independent channels in cochlear implants.

Phase effects in F0 discrimination and temporal fine structure

Pitch discrimination with only high-order (unresolved) harmonics is dependent on their relative phases and is thought to be based on the temporal envelope of the waveform after peripheral filtering. In contrast, when low-order harmonics are present, phase effects disappear, and pitch may be derived from the estimated frequencies of the individual resolved harmonics. The dichotomy between the envelope-based periodicity from unresolved harmonics and the temporal fine-structure (or spectrally) based estimates of individual frequencies from resolved harmonics forms the basis of the well-known two-mechanism theory of pitch. Recently, the early idea that listeners use temporal fine-structure cues in unresolved harmonics to estimate the overall F0 has been revived as a third intermediate mechanism to account for why accurate F0 discrimination, normally associated with resolved harmonics, can sometimes be achieved in conditions that produce phase effects. Here we show that it is not necessary to resort to a third mechanism. Instead, we show that the relative component phases affect the instantaneous frequency distributions in ways that could influence the coding of frequencies for individual, partially resolved, harmonics. Implications for the interpretation of temporal fine structure in pitch and other auditory functions, such as speech perception, are discussed. [Work supported by NIH R01DC05216.]

Detection and F discrimination of harmonic complex tones in the presence of competing tones or noise

Normal-hearing listeners' ability to "hear out" the pitch of a target harmonic complex tone (HCT) was tested with simultaneous HCT or noise maskers, all bandpass-filtered into the same spectral region (1200-3600 Hz). Target-to-masker ratios (TMRs) necessary to discriminate fixed fundamental-frequency (F0) differences were measured for target F0s between 100 and 400 Hz. At high F0s (400 Hz), asynchronous gating of masker and signal, presenting the masker in a different F0 range, and reducing the F0 rove of the masker, all resulted in improved performance. At the low F0s (100 Hz), none of these manipulations improved performance significantly. The findings are generally consistent with the idea that the ability to segregate sounds based on cues such as F0 differences and onset/offset asynchronies can be strongly limited by peripheral harmonic resolvability. However, some cases were observed where perceptual segregation appeared possible, even when no peripherally resolved harmonics were present in the mixture of target and masker. A final experiment, comparing TMRs necessary for detection and F0 discrimination, showed that F0 discrimination of the target was possible with noise maskers at only a few decibels above detection threshold, whereas similar performance with HCT maskers was only possible 15-25 dB above detection threshold.

Effect of noise on the detectability and fundamental frequency discrimination of complex tones

Percent correct performance for discrimination of the fundamental frequency (0) of a complex tone was measured as a function of the level of a background pink noise (using fixed values of the difference in F0, deltaF0) and compared with percent correct performance for detection of the complex tone in noise, again as a function of noise level. The tone included some low, resolvable components, but not the fundamental component. The results were used to test the hypothesis that the worsening in F0 discrimination with increasing noise level was caused by the reduced detectability of the tone rather than by reduced precision of the internal representation of F0. For small values of deltaF0, the hypothesis was rejected because measured performance fell below that predicted by the hypothesis. However, this was true only for high noise levels, within 2-4.5 dB of the level required for masked threshold. The results indicate that the mechanism for extracting the F0 of a complex tone with resolved harmonics is remarkably robust. They also indicate that adding a background noise to a complex tone containing resolved harmonics is not a good means for equating its pitch salience with that of a complex tone containing only unresolved harmonics.

Improved Music Perception with Explicit Pitch Coding in Cochlear Implants

Music perception and appraisal is very poor in cochlear implant (CI) subjects partly because (musical) pitch is inadequately transmitted by the current clinically used sound processors. A new sound processing scheme (F0mod) was designed to optimize pitch perception, and its performance for music and pitch perception was compared in four different experiments to that of the current clinically used sound processing scheme (ACE) in six Nucleus CI24 subjects. In the F0mod scheme, slowly varying channel envelopes are explicitly modulated sinusoidally at the fundamental frequency (F0) of the input signal, with 100% modulation depth and in phase across channels to maximize temporal envelope pitch cues. The results of the four experiments show that: (1) F0 discrimination of single-formant stimuli was not significantly different for the two schemes, (2) F0 discrimination of musical notes of five instruments was three times better with the F0mod scheme for F0 up to 250 Hz, (3) melody recognition of familiar Flemish songs (with all rhythm cues removed) was improved with the F0mod scheme, and (4) estimates of musical pitch intervals, obtained in a musically trained CI subject, matched more closely the presented intervals with the F0mod scheme. These results indicate that explicit F0 modulation of the channel envelopes improves music perception in CI subjects.

Effects of Envelope-Vocoder Processing on F0 Discrimination and Concurrent-Vowel Identification

The aim of this study was to examine the effects of envelope-vocoder sound processing on listeners' ability to discriminate changes in fundamental frequency (F0) in anechoic and reverberant conditions and on their ability to identify concurrent vowels based on differences in F0. In the first experiment, F0 difference limens (F0DLs) were measured as a function of number of envelope-vocoder frequency channels (1, 4, 8, 24, and 40 channels, and unprocessed) in four normal-hearing listeners, with degree of simulated reverberation (no, mild, and severe reverberation) as a parameter. In the second experiment, vowel identification was measured as a function of the F0 difference between two simultaneous vowels in six normal-hearing listeners, with the number of vocoder channels (8 and 24 channels, and unprocessed) as a parameter. Reverberation was detrimental to F0 discrimination in conditions with fewer numbers of vocoder channels. Despite the reasonable F0DLs (<1 semitone) with 24- and 8-channel vocoder processing, listeners were unable to benefit from F0 differences between the competing vowels in the concurrent-vowel paradigm. The overall detrimental effects of vocoder processing are probably are due to the poor spectral representation of the lower-order harmonics. The F0 information carried in the temporal envelope is weak, susceptible to reverberation, and may not suffice for source segregation. To the extent that vocoder processing simulates cochlear implant processing, users of current implant processing schemes are unlikely to benefit from F0 differences between competing talkers when listening to speech in complex environments. The results provide further incentive for finding a way to make the information from low-order, resolved harmonics available to cochlear implant users.