Inharmonic Stimuli Research Articles

Investigating the neural correlates of the emergence of pitch using harmonic and inharmonic stimuli

The emergence of a pitch percept, referred to as the pitch onset response (POR), has predominantly been studied using transitions from noise to regular interval noise (RIN), click trains and more recently, harmonic complex tones. The POR is typically an asymmetrical change response which shows a larger response for stimuli that go from having no salient pitch to a salient pitch but not vice versa. However, the spectro-temporal differences between the pitch inducing stimulus and noise make it difficult to disentangle whether the response reflects a change in regularity or a true difference in pitch salience. The aim of this experiment is to use spectrally similar stimuli that differ in pitch salience (harmonic versus inharmonic tones) without changes in temporal regularity. We recorded EEG responses in normal-hearing participants to investigate the cortical correlates of pitch emergence in humans. We used the noise to RIN transitions as a baseline measure to replicate the asymmetry in the POR. We also used transitions from harmonic to inharmonic stimuli and vice versa to investigate if the POR asymmetry would hold for stimuli with minimal spectro-temporal differences. Results from this study will help us further understand the parameters affecting this response.

Time-dependent discrimination advantages for harmonic sounds suggest efficient coding for memory

Perceptual systems have finite memory resources and must store incoming signals in compressed formats. To explore whether representations of a sound's pitch might derive from this need for compression, we compared discrimination of harmonic and inharmonic sounds across delays. In contrast to inharmonic spectra, harmonic spectra can be summarized, and thus compressed, using their fundamental frequency (f0). Participants heard two sounds and judged which was higher. Despite being comparable for sounds presented back-to-back, discrimination was better for harmonic than inharmonic stimuli when sounds were separated in time, implicating memory representations unique to harmonic sounds. Patterns of individual differences (correlations between thresholds in different conditions) indicated that listeners use different representations depending on the time delay between sounds, directly comparing the spectra of temporally adjacent sounds, but transitioning to comparing f0s across delays. The need to store sound in memory appears to determine reliance on f0-based pitch and may explain its importance in music, in which listeners must extract relationships between notes separated in time.

The function of F0-based pitch

Pitch is traditionally described as the perceptual correlate of a sound’s fundamental frequency (F0). The importance of perceiving F0 is often attributed to the need to compare sounds with different timbres, for which spectral comparisons might be error-prone. Alternatively, because the F0 is a summary measure of a set of harmonic frequencies, it could provide a compressed representation of sound for memory storage. To clarify the function of F0-based pitch, we tested its importance for discriminating sounds with different spectra, and sounds separated by delays, using both harmonic and inharmonic stimuli. If a task relies on F0-based pitch, performance should be impaired for inharmonic stimuli. We observed an advantage for harmonic stimuli for discrimination of synthetic tones with extreme (and unnatural) spectral variation. However, no such advantage was observed for instrument or speech fragments (with naturally occurring spectral variation). By contrast, discrimination was better for harmonic tones separated by brief (5–10 s) delays, suggesting that F0-based pitch aided storage across the delay. The results indicate that the spectral variation found in natural pitched sounds does not itself make listeners reliant on F0 estimation. The results raise the possibility that F0 primarily serves to facilitate compression of sounds for memory storage.

Multiple pitch mechanisms revealed by effects of inharmonicity on pitch perception

Pitch is generally defined as the percept corresponding to a sound’s fundamental frequency (F0). However, there is surprisingly little evidence for the importance of F0-specific mechanisms in pitch perception. To investigate the extent to which pitch perception involves estimating F0, we conducted a battery of pitch-related music and speech tasks using harmonic and inharmonic stimuli, the rationale being that inharmonic stimuli lack a clear F0 and should impair F0-dependent mechanisms. Inharmonic speech was generated with a variant of STRAIGHT analysis/synthesis. We found no difference in performance between harmonic and inharmonic stimuli for basic pitch discrimination, melodic contour discrimination, and speech contour discrimination, but substantial inharmonicity-induced deficits for sour-note detection, interval recognition and discrimination, melody recognition, and voice recognition. Collectively, the results indicate that F0 estimation is not necessary to extract pitch contours in speech or music, but does appear critical for accurate interval perception, and for representing voices. These findings suggest that two or three distinct mechanisms may underlie what has conventionally been couched as pitch perception: one which tracks shifts in fine spectral patterns independent of F0, another that estimates F0 for determining precise interval relationships between notes, and perhaps another associated with voice qualities.

Effect of frequency-modulation coherence for inharmonic stimuli: Frequency-modulation phase discrimination and identification of artificial double vowels

The ability to compare patterns of frequency modulation (FM) in separate frequency regions was explored. In experiment 1, listeners had to distinguish whether the FM applied to two nonharmonically related sinusoidal carriers was in phase or out of phase. The FM rate was the same for each carrier. The starting phase of the modulation was randomized for each stimulus in a three alternative, forced-choice (3AFC) trial. Subjects were sensitive to relative FM phase for modulation rates of 2 and 4 Hz, but not for higher rates. In experiment 2, vowel identification was compared for artificial single and double vowels. The vowels were constructed from complex tones with components spaced at 2-ERB(N) (equivalent rectangular bandwidth) intervals, by increasing the levels of three components by 15 dB, to create three "formants." In the double vowels, the components of the two vowels were interleaved, to give 1-ERB(N) spacing. The three "formant" components were frequency modulated at 2, 4, or 8 Hz, with either the same or different rates for the two vowels. The identification of double vowels was not improved by a difference in FM rate across vowels, suggesting that differences in FM rate do not support perceptual segregation of inharmonic stimuli.

Autocorrelation model for pitch processing in the frog’s auditory nerve

Bullfrogs’ reactions to simulated vocalizations are sensitive to disruption of the 100-Hz 1st harmonic periodicity caused by the presence of inharmonic frequency components or changes in component phase. This behavioral preference for a specific range of periods indicates that representation of periodicity ‘‘pitch’’ may be the basis for evaluation of conspecific calls. Anuran auditory-nerve fibers phase-lock to the 1st and sometimes the 2nd or 3rd harmonic of multiple harmonic signals. Interval histograms of fiber responses reflect the complex pitch of both harmonic and inharmonic stimuli. We developed a pitch model that estimates pitch-period autocorrelation functions from all-order interval histograms across frequency channels by implementing model neurons designed to replicate the frog’s auditory nerve. Histograms are aligned in time across channels before being combined into a single pitch image (‘‘strobing’’ in the Patterson pulse-ribbon model; ‘‘dechirping’’ in bat sonar target-ranging). The resultant output yields a sharp pitchlike image, consistent with both behavioral and physiological results. Robust 1st-harmonic pitch images are produced for multiple harmonic signals, and even two-component sounds require only a few parallel fibers to yield a robust 1st-harmonic pitch.

Concurrent vowel identification. II. Effects of phase, harmonicity, and task

Subjects identified concurrent synthetic vowel pairs in four experiments. The first experiment found that improvements in vowel identification with a difference in fundamental frequency (ΔF0) do not depend on component phase. The second investigated more precisely whether phase patterns resulting from ongoing phase shifts in inharmonic stimuli can by themselves produce effects similar to those attributed to differences in harmonic state of component vowels. No such effects were found. The third experiment found that identification was better for harmonic than for inharmonic backgrounds, and that it did not depend on target harmonicity. The first three experiments employed a task in which subjects were free to report one or two vowels for each stimulus. The fourth experiment reproduced several conditions with a more classic task in which subjects had to report two vowels. Compared to the classic task, the new task gave larger effects and provided an additional measure of segregation: the number of vowels reported per stimulus. Overall, results were consistent with the hypothesis that the auditory system segregates targets by a mechanism of harmonic cancellation of competing vowels. They did not support the hypothesis of harmonic enhancement of targets. The lack of a phase effect places strong constraints on models that exploit pitch period asynchrony (PPA) or beats.

Periodicity extraction in the anuran auditory nerve. I. "Pitch-shift" effects.

1. Activity of individual eight nerve fibers in the bullfrog, Rana catesbeiana, was measured in response to complex, multiple-frequency stimuli differing in both frequency composition and harmonic structure. Stimuli were chosen to parallel types of stimuli producing "pitch-shift" effects in humans. 2. The fundamental frequency of harmonic stimuli can be extracted from the autocorrelation of fiber firing, whether the fundamental is physically present in the stimulus or is a "missing" fundamental. The spectral fine-structure of harmonic stimuli is not robustly represented in fiber temporal response. These effects are seen in both AP and BP fibers. 3. The pseudoperiod of inharmonic stimuli is represented by synchronization to successive high-amplitude peaks in the stimulus envelope. Temporal responses to stimuli with high center frequencies are similar regardless of whether their frequency components are harmonically or inharmonically related. Responses remain dominated by the envelope periodicity, and no "pitch-shift" is signaled. In response to stimuli with low center frequencies, temporal responses signal a "pitch-shift" between harmonic and inharmonic complexes. Both AP and BP fibers show these effects. 4. These data suggest that bullfrog peripheral fibers extract the periodicity of complex stimuli by time-domain rather than frequency-domain coding.