Pitch Cues Research Articles

Discovering Structure in Auditory Input: Evidence From Williams Syndrome

We examined auditory perception in Williams syndrome by investigating strategies used in organizing sound patterns into coherent units. In Experiment 1, we investigated the streaming of sound sequences into perceptual units, on the basis of pitch cues, in a group of children and adults with Williams syndrome compared to typical controls. We showed that individuals with Williams syndrome were sensitive to the same pitch cues as typical children and adults when streaming these patterns. In Experiment 2, we evaluated differences in reliance on pitch and contour cues in unfamiliar melody perception in a group of adults with Williams syndrome relative to typical control children and adults. Unlike controls who demonstrated greater proficiency when contour cues were available, adults with Williams syndrome showed no such advantage.

Do ferrets perceive relative pitch?

The existence of relative pitch perception in animals is difficult to demonstrate, since unlike humans, animals often attend to absolute rather than relative properties of sound elements. However, the results of the present study show that ferrets can be trained using relative pitch to discriminate two-tone sequences (rising vs. falling). Three ferrets were trained using a positive-reinforcement paradigm in which sequences of reference (one to five repeats) and target stimuli were presented, and animals were rewarded only when responding correctly to the target. The training procedure consisted of three training phases that successively shaped the ferrets to attend to relative pitch. In Phase-1 training, animals learned the basic task with sequences of invariant tone-pairs and could use absolute pitch information. During Phase-2 training, in order to emphasize relative cues, absolute pitch was varied each trial within a two-octave frequency range. In Phase-3 training, absolute pitch cues were removed, and only relative cue information was available to solve the task. Two ferrets successfully completed training on all three phases and achieved significant discriminative performance over the trained four-octave frequency range. These results suggest that ferrets can be trained to discern the relative pitch relationship of a sequence of tone-pairs independent of frequency.

The tonal component of Estonian quantity in native and non-native perception

This paper focuses on the role of the pitch cue in the perception of Estonian quantity degrees. The significance of the tonal component is investigated through comparison of native vs. learned identification of the quantities. The study reports on perception tests with manipulated natural speech stimuli given to Estonian native (L1) listeners and non-native (L2) listeners with different language backgrounds. Our earlier results [Lippus, P., Pajusalu, K., & Allik, J. (2007). The tonal component in perception of the Estonian quantity. In J. Trouvain, & Barry, W. J. (Eds.), The proceedings of the 16th international congress of phonetic sciences: 16th international congress of phonetic sciences, Saarbrücken, Germany, 6–10 August 2007 (pp. 1049–1052)] showed that native Estonian listeners use pitch as an important cue for perceiving the overlong quantity (Q3), but the cue plays no significant role for L2 listeners. In this study we analyze more closely the effect of the listeners’ native language on the perception of Estonian quantities. Finnish and Russian L1 listeners successfully learn the Estonian three-way quantity distinction without considering the pitch cue. Latvian L1 listeners show some confusion between the long and overlong quantity degree, which could be explained by reference to the tonal and temporal contrasts in their native language.

Examining the role of pitch, duration, and intensity in word segmentation.

English listeners can make use of stress cues in word segmentation. Stress is associated with a number of phonetic cues, including pitch movements, longer durations, and greater intensity. Studies on stress perception have shown that pitch is the most powerful cue to English stress, followed by increased duration and greater intensity [Fry (1958); Bolinger (1958)], suggesting that pitch alone might be sufficient to cue word boundary. Here, I test whether pitch alone is enough to cue word boundary for English listeners, using an artificial language paradigm. The artificial language used contains no distributional cues to word boundary, so words can only be segmented using pitch. I also pit pitch cues against intensity and durational cues in order to test whether English listeners weight cues for stress in a word segmentation task, or if they require correlates of stress to be bundled together.

Concurrent-vowel and tone recognitions in acoustic and simulated electric hearing

Because of the poor spectral resolution in cochlear implants (CIs), fundamental frequency (F0) cues are not well preserved. Chinese-speaking CI users may have great difficulty understanding speech produced by competing talkers, due to conflicting tones. In this study, normal-hearing listeners' concurrent Chinese syllable recognition was measured with unprocessed speech and CI simulations. Concurrent syllables were constructed by summing two vowels from a male talker (with identical mean F0's) or one vowel from each of a male and a female talker (with a relatively large F0 separation). CI signal processing was simulated using four- and eight-channel noise-band vocoders; the degraded spectral resolution may limit listeners' ability to utilize talker and/or tone differences. The results showed that concurrent speech recognition was significantly poorer with the CI simulations than with unprocessed speech. There were significant interactions between the talker and speech-processing conditions, e.g., better tone and syllable recognitions with the male-female condition for unprocessed speech, and with the male-male condition for eight-channel speech. With the CI simulations, competing tones interfered with concurrent-tone and syllable recognitions, but not vowel recognition. Given limited pitch cues, subjects were unable to use F0 differences between talkers or tones for concurrent Chinese syllable recognition.

Cochlear Implant Melody Recognition as a Function of Melody Frequency Range, Harmonicity, and Number of Electrodes

The primary goal of the present study was to determine how cochlear implant melody recognition was affected by the frequency range of the melodies, the harmonicity of these melodies, and the number of activated electrodes. The secondary goal was to investigate whether melody recognition and speech recognition were differentially affected by the limitations imposed by cochlear implant processing. Four experiments were conducted. In the first experiment, 11 cochlear implant users used their clinical processors to recognize melodies of complex harmonic tones with their fundamental frequencies being in the low (104-262 Hz), middle (207-523 Hz), and high (414-1046 Hz) ranges. In the second experiment, melody recognition with pure tones was compared to melody recognition with complex harmonic tones in four subjects. In the third experiment, melody recognition was measured as a function of the number of electrodes in five subjects. In the fourth experiment, vowel and consonant recognition were measured as a function of the number of electrodes in the same five subjects who participated in the third experiment. Frequency range significantly affected cochlear implant melody recognition, with higher frequency ranges producing better performance. Pure tones produced significantly better performance than complex harmonic tones. Increasing the number of activated electrodes did not affect performance with low- and middle-frequency melodies but produced better performance with high-frequency melodies. Large individual variability was observed for melody recognition, but its source seemed to be different from the source of the large variability observed in speech recognition. Contemporary cochlear implants do not adequately encode either temporal pitch or place pitch cues. Melody recognition and speech recognition require different signal processing strategies in future cochlear implants.

Virtual pitch extraction from harmonic structures by absolute-pitch musicians

The ability of absolute-pitch (AP) musicians to identify or produce virtual pitch from harmonic structures without feedback or an external acoustic referent was examined in three experiments. Stimuli consisted of pure tones, missing-fundamental harmonic complexes, or piano notes highpass filtered to remove their fundamental frequency and lower harmonics. Results of Experiment I showed that relative to control (non-AP) musicians, AP subjects easily (>90%) identified pitch of harmonic complexes in a 12-alternative forced-choice task. Increasing harmonic order (i.e., lowest harmonic number in the complex), however, resulted in a monotonic decline in performance. Results suggest that AP musicians use two pitch cues from harmonic structures: 1) spectral spacing between harmonic components, and 2) octave-related cues to note identification in individually resolved harmonics. Results of Experiment II showed that highpass filtered piano notes are identified by AP subjects at better than 75% accuracy even when the note’s energy is confined to the 4th and higher harmonics. Identification of highpass piano notes also appears to be better than that expected from pure or complex tones, possibly due to contributions from familiar timbre cues to note identity. Results of Experiment III showed that AP subjects can adjust the spectral spacing between harmonics of a missing-fundamental complex to accurately match the expected spacing from a target musical note. Implications of these findings for mechanisms of AP encoding are discussed.

Music Onset Detection Based on Resonator Time Frequency Image

This paper describes a new method for music onset detection. The novelty of the approach consists mainly of two elements: the time-frequency processing and the detection stages. The resonator time frequency image (RTFI) is the basic time-frequency analysis tool. The time-frequency processing part is in charge of transforming the RTFI energy spectrum into more natural energy- change and pitch-change cues that are then used as input elements for the detection of music onsets by detection tools. Two detection algorithms have been developed: an energy-based algorithm and a pitch-based one. The energy-based detection algorithm exploits energy-change cues and performs particularly well for the detection of hard onsets. The pitch-based algorithm successfully exploits stable pitch cues for the onset detection in polyphonic music, and achieves much better performances than the energy-based algorithm when applied to the detection of soft onsets. Results for both the energy-based and pitch-based detection algorithms have been obtained on a large music dataset.

Dynamic encoding of pitch, timing, and timbre.

The perception of music depends on three fundamental elements: pitch, timing, and timbre. It is possible to access the neural transcription of these elements objectively, noninvasively, and with great fidelity with scalp electrodes in humans. The effects of musical experience on subcortical encoding of sound are pervasive and extend beyond music to the domains of language and emotion. The acoustic cues relating to pitch, timing, and timbre have distinct representations, which can be selectively enhanced or degraded in different populations. Musically trained subjects have enhanced subcortical representation of pitch, timing, and timbre, although with increased stimulus complexity, timbre cues become more pronounced while pitch cues are diminished. Musical experience fundamentally shapes auditory processing that occurs early in the sensory processing stream. This top‐down influence is likely mediated by the extensive (corticofugal) circuitry of descending efferent fibers that synapse all along the auditory pathway. Sensory shaping occurs not only with lifelong experience but can be effected by short‐term training. I will discuss a series of experiments demonstrating the lifelong and short‐term malleability of the neural encoding of pitch, timing, and timbre in the auditory brainstem. [Work supported by National Institutes of Health (R01DC001510) and National Science Foundation (NSF 0544846).

Spectral weights and the transition between two auditory processes.

Evidence of broad auditory filters for processing temporal information was obtained with a molecular psychophysical technique. The standard in the 2IFC discrimination task consisted of n equal intensity sinusoids, with frequency separations ranging from 10 to 40 Hz. The signal was an intensity increment of the central sinusoid, which on average was 1000 Hz. Pitch cues were degraded by randomly selecting the center frequency over a 100 Hz range on each stimulus presentation. Single channel intensity cues were degraded with a roving‐level procedure. As tones were added symmetrically to the ends of the complex, thresholds increased until a certain bandwidth was reached, beyond which thresholds decreased. Previous results showed that transition bandwidths ranged from 100 Hz to more than 1000 Hz across individuals [B. G. Berg, J. Acoust. Soc. Am. 121, 3639–3645 (2007)]. This discontinuity presumably represents a change in the underlying auditory processes, with envelope cues at bandwidths narrower than the breakpoint and across channel level comparisons for wider bandwidths. Findings show that the pattern of spectral weight estimates before the breakpoint is consistent with an envelope processing model, whereas the weights for bandwidths greater than the transition point were consistent with an across‐channel comparison. [Work supported by ONR].

Effects of pitch and spatial separation on selective attention in anechoic and reverberant environments

Subjects identified a random, spoken sequence of five monotonized digits (F0 = 100 Hz) presented from 0° azimuth. A monotonized masking sentence (F0 = 84, 89, 94, 100, 106, 112, or 119 Hz) was presented simultaneously from either 0° and +90° azimuth (chosen randomly on each trial). The same talker recorded both target and masker speech. KEMAR-derived transfer functions simulated either a reverberant or anechoic environment. In contrast to previous studies, in the anechoic condition, differences in pitch provided little benefit and differences in location gave improvements explainable by improvements in the target-to- masker ratio at the acoustically better ear. In reverberant conditions, differences in target and masker location improved performance more than differences in pitch; however, performance was best when there were differences in both location and pitch. Results suggest that when a target utterance is easy to segregate and select (such as in anechoic space when the target is a digit sequence embedded in a competing sentence), high-level attributes such as pitch and location do not improve the ability to selectively attend in a speech-on-speech task. However, in more challenging, reverberant conditions, location and pitch cues can aid segregation and/or selection.

An exploration of attentional monitoring of isochronous asynchronous streams in deviant detection and sensorimotor synchronization

Past studies of auditory selective attention focused on location and/or pitch cues in stream segregation. In this study, two perceptually-segregated streams competed for attention in a perceptual-judgment task based on timbre cues and a sensorimotor task (tapping to the sounds of one stream). Both streams shared the same temporal structure (isochronous 100-ms events at 1.67 Hz) but were asynchronous (target-distractor asynchronies of -200, -100, 100, 200 ms), differed in pitch (712 or 1000 Hz) and could share or not their perceived location (diotic vs. dichotic). Deviants were amplitude modulated at 25 or 50 Hz and varied in temporal envelopes. Attentional filtering (distracting stream constant) was done using a discrimination task requiring pressing one of two keys upon hearing one of two deviants (Experiment 1) and a synchronization task requiring selective tapping to the high-pitch or low-pitch sounds (Experiment 3). Attentional monitoring (distracting stream varies) required maintaining attention to the same stream upon hearing slow-modulated deviants and switching attention to the other stream upon hearing the fast-modulated deviants. This was done for the discrimination task (Experiment 2) and synchronization task (Experiment 4). Perceptual and perceptuo-motor results suggest attentional-filtering and attentional-monitoring costs. Surprisingly, the mere presence of a non-concurrent stream in a different frequency band interferes with deviant-discrimination and synchronization-tapping performance.

Role of temporal fine structure cues in speech intelligibility

A number of studies have investigated the role of two temporal features of the narrowband speech signals at the output of auditory filters in speech identification: temporal envelope (E) and temporal fine structure (TFS) cues. To assess the contribution of each feature to speech identification, speech stimuli were split into an array of contiguous analysis bands and processed using several techniques to remove, as far as possible, either E or TFS cues within each band. Overall, the outcome of these studies indicated that, after moderate to subtantial training, high levels of speech identification measured in quiet could be obtained for normal‐hearing listeners on the basis of E cues or TFS cues alone. The results obtained with TFS cues only may appear surprising, because it is generally considered that, at least for non‐tonal languages, E cues carry most of the phonetic information required for speech identification in quiet whereas TFS cues play mainly a role in conveying the pitch cues required for the segregation of speech and background sounds. Further work assessing the extent to which TFS cues alone can convey useful linguistic information in addition to these pitch cues will be reviewed.

Improving source localisation in multi‐source, reverberant conditions: exploiting local spectro‐temporal location cues

This paper examines the use of binaural cues for estimating the location of individual sources in multi‐source acoustic mixtures. In most environments, location cues such as interaural time difference (ITD) and interaural intensity difference (IID) are rendered unreliable by noise due to competing sources and reverberation. We propose a method that examines spectro‐temporal regions where such effects are minimised. Source location is estimated by combining two processing stages. First, following [Christensen et al, Interspeech (2007)], as pitch cues are generally less affected by noise, a multi‐pitch estimation and tracking algorithm is used to identify local spectro‐temporal 'fragments' in which the SNR is high. Second, the confidence of ITD and IID estimates arising from each time‐frequency 'pixel' is estimated based on interaural coherence [Faller and Merimaa, J. Acoust. Soc. Am. 116, 3075‐‐3089 (2004)] and reverberation estimation [Heckmann et al, Int. Conf. Int. Robots and Systems (2006)]. A location estimate is then constructed by integrating suitably weighted cues from each pixel within the fragment. Experiments are carried out on a large corpus of multi‐speaker data, mixed from binaural recordings in a real, reverberant environment. The fragment based processing is shown to provide significant improvements when compared to competitive baseline approaches.

Estimating the effective frequency of cochlear implant electrodes using contralateral residual acoustic hearing

For some cochlear implant (CI) users a contralateral hearing aid provides significantly improved speech perception. Important factors in the bimodal transmission of speech spectral information are likely to include the extent to which the frequency selectivity of residual hearing allows additional place‐coded channels, and mismatches between frequency‐to‐place maps across modalities. When acoustic place coding extends above around 500 Hz an overlap of frequency coverage between acoustic and electric hearing may result in interaural conflicts. However, addressing this issue requires accurate knowledge of CI frequency‐to‐place maps. Effective characteristic frequencies of CI electrodes have previously been estimated using comparisons of the pitch produced by electrical stimulation with that produced by contralateral acoustic sinusoids In the present work, the acoustic stimuli used for pitch comparisons are either sinusoids or 1/3 octave bands of noise. The latter minimize temporal pitch cues and may reduce ...

Masking release based on fine‐grain temporal pitch cues in speech‐like stimuli ‐ mechanisms and implications for the cocktail‐party phenomenon

One of the most important cues for segregating wanted from unwanted sounds when listening in noisy environments is temporal regularity (or harmonicity), which gives rise to the perception of pitch. This study measured masked signal detection thresholds when both the signal and the masker are complex tonal stimuli similar to speech. Detection was considerably easier when the pitch of the masker differed from that of the signal. A control experiment involving non‐simultaneous masking revealed that this pitch‐based release from masking was not due to spectral differences between the signal and the masker. A second experiment investigated the effect of a pitch difference between the signal and the masker on the audibility of certain perceptual features of the signal when the signal is presented at a supra‐threshold level. Two features were used that are known to be particularly important for speech perception: amplitude modulation and spectral profile related to formant frequencies. The supra‐threshold tasks enabled to quantify the unmasking effect of pitch on the processing of these features over and above the effect on the signal detection threshold. The results provide new insights into the low‐level processes involved in the cocktail‐party phenomenon. Acknowledgement: Work supported by the Medical Research Council.

Cochlear Implants Special Issue Article: Vocal Emotion Recognition by Normal-Hearing Listeners and Cochlear Implant Users

The present study investigated the ability of normal-hearing listeners and cochlear implant users to recognize vocal emotions. Sentences were produced by 1 male and 1 female talker according to 5 target emotions: angry, anxious, happy, sad, and neutral. Overall amplitude differences between the stimuli were either preserved or normalized. In experiment 1, vocal emotion recognition was measured in normal-hearing and cochlear implant listeners; cochlear implant subjects were tested using their clinically assigned processors. When overall amplitude cues were preserved, normal-hearing listeners achieved near-perfect performance, whereas listeners with cochlear implant recognized less than half of the target emotions. Removing the overall amplitude cues significantly worsened mean normal-hearing and cochlear implant performance. In experiment 2, vocal emotion recognition was measured in listeners with cochlear implant as a function of the number of channels (from 1 to 8) and envelope filter cutoff frequency (50 vs 400 Hz) in experimental speech processors. In experiment 3, vocal emotion recognition was measured in normal-hearing listeners as a function of the number of channels (from 1 to 16) and envelope filter cutoff frequency (50 vs 500 Hz) in acoustic cochlear implant simulations. Results from experiments 2 and 3 showed that both cochlear implant and normal-hearing performance significantly improved as the number of channels or the envelope filter cutoff frequency was increased. The results suggest that spectral, temporal, and overall amplitude cues each contribute to vocal emotion recognition. The poorer cochlear implant performance is most likely attributable to the lack of salient pitch cues and the limited functional spectral resolution.

Estimating the transition bandwidth between two auditory processes: Evidence for broadband auditory filters

A spectral discrimination task was used to estimate the frequency range over which information about the temporal envelope is consolidated. The standard consisted of n equal intensity, random phase sinusoids, symmetrically placed around a signal component. The signal was an intensity increment of the central sinusoid, which on average was 1000 Hz. Pitch cues were degraded by randomly selecting the center frequency of the complex and single channel energy cues were degraded with a roving-level procedure. Stimulus bandwidth was controlled by varying the number of tones and the frequency separation between tones. For a fixed frequency separation, thresholds increased as n increased until a certain bandwidth was reached, beyond which thresholds decreased. This discontinuity in threshold functions suggests that different auditory processes predominate at different bandwidths, presumably an envelope analysis at bandwidths less than the breakpoint and across channel level comparisons for wider stimulus bandwidths. Estimates of the "transition bandwidth" for 46 listeners ranged from 100 to 1250 Hz. The results are consistent with a peripheral filtering system having multiple filterbanks.

Parsing of ambiguous phrases: Single versus multiple prosodic cues

In English, two prosodic contours that differ acoustically in their pitch and the location of the pauses involved can be used to disambiguate two-word phrases. The list contour, which has a fall-rise in pitch over the first word and a pause (approximately 22 ms) between the words is distinguished from the compound contour, which has a rise-fall in the pitch over both words and no pause. In this experiment target words recorded with relatively flat pitch were spliced together to make phrases with either 10 or 40 ms between each word. Participants treated items containing the 10-ms pauses as if they were compounds and items with 40-ms pauses as lists even without a pitch cue. This was true for lexically ambiguous phrases where either prosodic contour was appropriate and for nonsense phrases where the prosodic contour alone made it possible to select the correct answer. There was no statistical difference between the responses to items with just the pause cue and to items with the full prosodic contour. While many prosodic phenomena have multiple cues, the data suggest that in some cases a single cue can be used in place of multiple cues to the same effect.

Pitch perception in normal and impaired hearing

Despite a century-long debate, there is little consensus on how pitch is coded. This talk will focus on recent psychoacoustic studies that have attempted to shed light on some of the basic principles of pitch coding. The pitch associated with melodies and music is conveyed mainly by lower-numbered harmonics, which are thought to be individually resolved in the peripheral auditory system. When the lower harmonics are removed, the pitch percept becomes weaker and dependent on the phase relations between harmonics, as expected with unresolved harmonics that interact within the passbands of peripheral auditory filters. A number of studies in normal and impaired hearing have shown a connection between harmonic resolvability and pitch strength or salience. However, there are reasons to doubt the causality of this relationship. The evidence for whether resolved harmonics are necessary or sufficient for salient pitch will be reviewed. Finally, some experiments related to the use of pitch cues in segregating speech from competing interferers will be discussed, with particular reference to whether the cues that produce strong pitch salience in psychoacoustic tasks are also those that are most useful in promoting segregation. [Work supported by NIH Grant R01DC05216.]