Inharmonic Complexes Research Articles

Benefits of Harmonicity for Hearing in Noise Are Limited to Detection and Pitch-Related Discrimination Tasks.

Harmonic complex tones are easier to detect in noise than inharmonic complex tones, providing a potential perceptual advantage in complex auditory environments. Here, we explored whether the harmonic advantage extends to other auditory tasks that are important for navigating a noisy auditory environment, such as amplitude- and frequency-modulation detection. Sixty young normal-hearing listeners were tested, divided into two equal groups with and without musical training. Consistent with earlier studies, harmonic tones were easier to detect in noise than inharmonic tones, with a signal-to-noise ratio (SNR) advantage of about 2.5 dB, and the pitch discrimination of the harmonic tones was more accurate than that of inharmonic tones, even after differences in audibility were accounted for. In contrast, neither amplitude- nor frequency-modulation detection was superior with harmonic tones once differences in audibility were accounted for. Musical training was associated with better performance only in pitch-discrimination and frequency-modulation-detection tasks. The results confirm a detection and pitch-perception advantage for harmonic tones but reveal that the harmonic benefits do not extend to suprathreshold tasks that do not rely on extracting the fundamental frequency. A general theory is proposed that may account for the effects of both noise and memory on pitch-discrimination differences between harmonic and inharmonic tones.

Measuring harmonic benefit in musicians and non-musicians in several tasks

Prior work has demonstrated that harmonic tones are easier to detect in noise and yield better F0 discrimination in noise than inharmonic tones. These effects, referred to as harmonic benefit, appear to be approximately the same size in musicians and non-musicians, despite musicians’ overall better pitch discrimination. The present study aimed to replicate these findings and extend them to include measurements of harmonic benefit in other tasks. Non-musicians and musicians were compared in four tasks: detection in noise, F0 discrimination, FM detection, and AM detection. The stimuli in each task were either harmonic or inharmonic complex tones and were presented in threshold-equalizing noise at a range of signal-to-noise ratios. We found that harmonic benefit for F0 discrimination was large and remained large even after accounting for differences in the detectability of harmonic and inahrmonic tones. In contrast, harmonic benefit was small for FM and AM detection and could mostly be accounted for by differences in detectability. In contrast to prior studies, we found that musicians showed a larger harmonic benefit than non-musicians. These results provide insight into how musical training may specialize the auditory system for processing of harmonicity and pitch. [Work supported by NIH grants F31 DC019247 and R01 DC005216.]

Robust Rate-Place Coding of Resolved Components in Harmonic and Inharmonic Complex Tones in Auditory Midbrain.

Harmonic complex tones (HCTs) commonly occurring in speech and music evoke a strong pitch at their fundamental frequency (F0), especially when they contain harmonics individually resolved by the cochlea. When all frequency components of an HCT are shifted by the same amount, the pitch of the resulting inharmonic tone (IHCT) can also shift, although the envelope repetition rate is unchanged. A rate-place code, whereby resolved harmonics are represented by local maxima in firing rates along the tonotopic axis, has been characterized in the auditory nerve and primary auditory cortex, but little is known about intermediate processing stages. We recorded single-neuron responses to HCT and IHCT with varying F0 and sound level in the inferior colliculus (IC) of unanesthetized rabbits of both sexes. Many neurons showed peaks in firing rate when a low-numbered harmonic aligned with the neuron's characteristic frequency, demonstrating "rate-place" coding. The IC rate-place code was most prevalent for F0 > 800 Hz, was only moderately dependent on sound level over a 40 dB range, and was not sensitive to stimulus harmonicity. A spectral receptive-field model incorporating broadband inhibition better predicted the neural responses than a purely excitatory model, suggesting an enhancement of the rate-place representation by inhibition. Some IC neurons showed facilitation in response to HCT relative to pure tones, similar to cortical "harmonic template neurons" (Feng and Wang, 2017), but to a lesser degree. Our findings shed light on the transformation of rate-place coding of resolved harmonics along the auditory pathway.SIGNIFICANCE STATEMENT Harmonic complex tones are ubiquitous in speech and music and produce strong pitch percepts when they contain frequency components that are individually resolved by the cochlea. Here, we characterize a "rate-place" code for resolved harmonics in the auditory midbrain that is more robust across sound levels than the peripheral rate-place code and insensitive to the harmonic relationships among frequency components. We use a computational model to show that inhibition may play an important role in shaping the rate-place code. Our study fills a major gap in understanding the transformations in neural representations of resolved harmonics along the auditory pathway.

Exploring the Role of Medial Olivocochlear Efferents on the Detection of Amplitude Modulation for Tones Presented in Noise.

The medial olivocochlear reflex has been hypothesized to improve the detection and discrimination of dynamic signals in noisy backgrounds. This hypothesis was tested here by comparing behavioral outcomes with otoacoustic emissions. The effects of a precursor on amplitude-modulation (AM) detection were measured for a 1- and 6-kHz carrier at levels of 40, 60, and 80dB SPL in a two-octave-wide noise masker with a level designed to produce poor, but above-chance, performance. Three types of precursor were used: a two-octave noise band, an inharmonic complex tone, and a pure tone. Precursors had the same overall level as the simultaneous noise masker that immediately followed the precursor. The noise precursor produced a large improvement in AM detection for both carrier frequencies and at all three levels. The complex tone produced a similarly large improvement in AM detection at the highest level but had a smaller effect for the two lower carrier levels. The tonal precursor did not significantly affect AM detection in noise. Comparisons of behavioral thresholds and medial olivocochlear efferent effects on stimulus frequency otoacoustic emissions measured with similar stimuli did not support the hypothesis that efferent-based reduction of cochlear responses contributes to the precursor effects on AM detection.

Context effects in modulation masking

Neal Viemeister is especially known for his contributions to our understanding of perception of amplitude modulation. Modulation masking and signal-to-noise ratio in the modulation domain have been shown to be strong predictors of speech intelligibility in noise. This study uses 40-ms amplitude-modulated tones embedded in a simultaneous 100-ms noise masker to show that for a given type of simultaneous masker, modulation masking can vary significantly depending on the type of stimulus that precedes the masked target. Three types of precursor were used to investigate these context effects, a 2-octave noise centered on the signal frequency (the same as the masker), a 7-component inharmonic complex tone, and a pure tone with a frequency equal to that of the target. All precursors had a duration of 400 ms and were presented in close temporal proximity at levels equal to those of the noise masker. The noise precursor produced on average a 7-8 dB [20log(m)] release from modulation masking. In contrast no change in the amount of masking and a small release of masking were observed for the pure-tone and complex-tone precursors, respectively. Mechanisms that may underlie the observed context effects will be discussed. [Work supported by NIH grant R01 DC015462.]

A Neuronal Network Model for Pitch Selectivity and Representation.

Pitch is a perceptual correlate of periodicity. Sounds with distinct spectra can elicit the same pitch. Despite the importance of pitch perception, understanding the cellular mechanism of pitch perception is still a major challenge and a mechanistic model of pitch is lacking. A multi-stage neuronal network model is developed for pitch frequency estimation using biophysically-based, high-resolution coincidence detector neurons. The neuronal units respond only to highly coincident input among convergent auditory nerve fibers across frequency channels. Their selectivity for only very fast rising slopes of convergent input enables these slope-detectors to distinguish the most prominent coincidences in multi-peaked input time courses. Pitch can then be estimated from the first-order interspike intervals of the slope-detectors. The regular firing pattern of the slope-detector neurons are similar for sounds sharing the same pitch despite the distinct timbres. The decoded pitch strengths also correlate well with the salience of pitch perception as reported by human listeners. Therefore, our model can serve as a neural representation for pitch. Our model performs successfully in estimating the pitch of missing fundamental complexes and reproducing the pitch variation with respect to the frequency shift of inharmonic complexes. It also accounts for the phase sensitivity of pitch perception in the cases of Schroeder phase, alternating phase and random phase relationships. Moreover, our model can also be applied to stochastic sound stimuli, iterated-ripple-noise, and account for their multiple pitch perceptions.

Effects of auditory enhancement on the loudness of masker and target components

Auditory enhancement refers to the observation that the salience of one spectral region (the “signal”) of a broadband sound can be enhanced and can “pop out” from the remainder of the sound (the “masker”) if it is preceded by the broadband sound without the signal. The present study investigated auditory enhancement as an effective change in loudness, to determine whether it reflects a change in the loudness of the signal, the masker, or both. In the first experiment, the 500-ms precursor, an inharmonic complex with logarithmically spaced components, was followed after a 50-ms gap by the 100-ms signal or masker alone, the loudness of which was compared with that of the same signal or masker presented 2 s later. In the second experiment, the loudness of the signal embedded in the masker was assessed with and without a precursor using the same method, as was the loudness of the entire signal-plus-masker complex. The results suggest that the precursor does not affect the loudness of the signal or the masker alone, but enhances the loudness of the signal in the presence of the masker, while leaving the loudness of the surrounding masker unaffected. The results are consistent with an explanation based on “adaptation of inhibition” [Viemeister and Bacon (1982). J. Acoust. Soc. Am. 71, 1502–1507].

On the search of the optimal range and number of lags for an autocorrelation model of pitch

Autocorrelation-based pitch models account for a number of pitch phenomena. Previous work has demonstrated their successfulness in response to different stimuli, when combined with a model of the auditory nerve activity and summed across channels [Balaguer-Ballester, Denham & Meddis, J. Acoust. Soc. Am. 124, 2186–2195 (2008)]. However, autocorrelation models are generally thought of as biologically unrealistic because they require many calculations and might not tally with the neurobiological principle of economy. The present work consists of a series of simulations that aim to determine whether all operations required to compute autocorrelation are strictly necessary to account for the psychophysical data or not. To investigate this, different ranges and densities of lags are used, and the produced implementations are tested with stimuli comprising pure tones, resolved and unresolved complex tones, inharmonic complex tones, and click trains. As a result, the optimal range and number of lags is found, as the most economical setup that agrees with psychophysical evidence.

Implications of Within-Fiber Temporal Coding for Perceptual Studies of F0 Discrimination and Discrimination of Harmonic and Inharmonic Tone Complexes

Recent psychophysical studies suggest that normal-hearing (NH) listeners can use acoustic temporal-fine-structure (TFS) cues for accurately discriminating shifts in the fundamental frequency (F0) of complex tones, or equal shifts in all component frequencies, even when the components are peripherally unresolved. The present study quantified both envelope (ENV) and TFS cues in single auditory-nerve (AN) fiber responses (henceforth referred to as neural ENV and TFS cues) from NH chinchillas in response to harmonic and inharmonic complex tones similar to those used in recent psychophysical studies. The lowest component in the tone complex (i.e., harmonic rank N) was systematically varied from 2 to 20 to produce various resolvability conditions in chinchillas (partially resolved to completely unresolved). Neural responses to different pairs of TEST (F0 or frequency shifted) and standard or reference (REF) stimuli were used to compute shuffled cross-correlograms, from which cross-correlation coefficients representing the degree of similarity between responses were derived separately for TFS and ENV. For a given F0 shift, the dissimilarity (TEST vs. REF) was greater for neural TFS than ENV. However, this difference was stimulus-based; the sensitivities of the neural TFS and ENV metrics were equivalent for equal absolute shifts of their relevant frequencies (center component and F0, respectively). For the F0-discrimination task, both ENV and TFS cues were available and could in principle be used for task performance. However, in contrast to human performance, neural TFS cues quantified with our cross-correlation coefficients were unaffected by phase randomization, suggesting that F0 discrimination for unresolved harmonics does not depend solely on TFS cues. For the frequency-shift (harmonic-versus-inharmonic) discrimination task, neural ENV cues were not available. Neural TFS cues were available and could in principle support performance in this task; however, in contrast to human-listeners' performance, these TFS cues showed no dependence on N. We conclude that while AN-fiber responses contain TFS-related cues, which can in principle be used to discriminate changes in F0 or equal shifts in component frequencies of peripherally unresolved harmonics, performance in these two psychophysical tasks appears to be limited by other factors (e.g., central processing noise).

The salience of enhanced components within inharmonic complexes

A subjective listening task was used to measure the salience of enhanced components using typical intensity-enhancement stimuli, time-reversed versions of those stimuli, and stimuli which contained a frequency shift of the target component. Twenty-five listeners judged whether or not a pitch "stood out" within an inharmonic complex. For comparison, judgments also were made for stimuli with a single segment that consisted of a simultaneously masked target. The results indicate that the perceived salience of enhanced components is greater than might be predicted by the effective magnitude of those components, and that informational masking is likely involved.

Effects of pulsing of the target tone on the audibility of partials in inharmonic complex tones

The audibility of partials was measured for complex tones with partials uniformly spaced on an ERB(N)-number scale. On each trial, subjects heard a sinusoidal "probe" followed by a complex tone. The probe was mistuned downwards or upwards (at random) by 3% or 4.5% from the frequency of one randomly selected partial in the complex (the "target"). The subject indicated whether the target was higher or lower in frequency than the probe. The probe and the target were pulsed on and off and the ramp times and inter-pulse intervals were systematically varied. Performance was better for longer ramp times and longer inter-pulse intervals. In a second experiment, the ability to detect which of two complex tones contained a pulsed partial was measured. The pattern of results was similar to that for experiment 1. A model of auditory processing including an adaptation stage was able to account for the general pattern of the results of experiment 2. The results suggest that the improvement in ability to hear out a partial in a complex tone produced by pulsing that partial is partly mediated by a release from adaptation produced by the pulsing, and does not result solely from reduction of perceptual confusion.

Effects of level and frequency on the audibility of partials in inharmonic complex tones

The effect of level and frequency on the audibility of partials was measured for complex tones with partials uniformly spaced on an equivalent rectangular bandwidth (ERB(N)) number scale. On each trial, subjects heard a sinusoidal "probe" followed by a complex tone. The probe was mistuned downwards or upwards (at random) by 4.5% from the frequency of one randomly selected partial in the complex. The subject indicated whether the probe was higher or lower in frequency than the nearest partial in the complex. The frequencies were roved from trial to trial, keeping frequency ratios fixed. In experiment 1, the level per partial, L, was 40 or 70 dB SPL and the mean frequency of the central partial, f(c), was 1201 Hz. Scores for the highest and lowest partials in the complexes were generally high for all spacings. Scores for the inner partials were close to chance at 0.75-ERB(N) spacing, and improved as the spacing was increased up to 2 ERB(N). For intermediate spacings, performance was better for the lower level used. In experiment 2, L was 70 dB SPL and f(c) was 3544 Hz. Performance worsened markedly for partial frequencies above 3544 Hz, consistent with a role of phase locking.

Perception of the envelope-beat frequency of inharmonic complex temporal envelopes

Listeners can hear slow sinusoidal variations in the depth of sinusoidally amplitude-modulated (SAM) stimuli. Here, the SAM stimulus of frequency f(m) acts as the carrier, and the slow variation in depth of frequency f'm (referred to as "second-order" amplitude modulation) corresponds to a beat in the temporal envelope. Recent studies have suggested that second-order amplitude modulation perception is based on a modulation-distortion component or the "venelope" (the Hilbert envelope of the ac-coupled Hilbert envelope), both occurring at the envelope-beat frequency f'm. This was tested by transposing to the modulation domain the matching paradigm used by Schouten et al. [J. Acoust. Soc. Am. 34, 1418-1424 (1962)]. Listeners estimated the envelope-beat frequency evoked by a 5-Hz, second-order SAM white noise with f(m) either an integer multiple of f'm or shifted in frequency to make the complex envelope inharmonic. The results indicate that the perception of the envelope-beat frequency was affected by these shifts when f(m) < or = 20 Hz. This suggests that, at least at low modulation frequencies, the perceived envelope beat is not determined by a modulation-distortion or venelope component, but rather relies on the time intervals between the main peaks of the first-order envelope.

Distortion product otoacoustic emissions generated by mistuned harmonic stimuli

Psychoacoustic research in humans [Lee and Green, J. Acoust. Soc. Am. 95, 718–725 (1994)] suggests that the detection of the mistuning of an harmonic when the harmonic complex and a mistuned component were presented simultaneously to the same ear stems, at least in part, from the resulting envelope interactions on the basilar membrane. Neurophysiological research in chinchillas [Sinex et al., Hear. Res. 168, 150–162 (2002)], provides evidence that such envelope interactions can be detected at the level of the inferior colliculus. The potential role of cochlear nonlinearites in determining the nature of the signal on the basilar membrane is explored by evaluating the DPOAE generated by multicomponent harmonic and inharmonic complexes. When harmonic complexes are used, the DPOAE all fall at harmonic frequencies. When inharmonic complexes are used, many nonharmonic DPOAE are detected.

Evoked potentials of the human auditory cortex: sensitive to the harmonic series?

Cortically generated auditory evoked potentials (N1 and P2) were recorded to frequency changes of harmonic and inharmonic complex tones comprising four sinusoidal components. The responses obtained when the frequencies suddenly became stationary after a period of 16/s changes were significantly shorter in latency when the frequencies were harmonically related, possibly implying a process of periodicity detection.

Sensitivity of human auditory evoked potentials to the harmonicity of complex tones: evidence for dissociated cortical processes of spectral and periodicity analysis.

A strong subjective tendency exists for simultaneous sound frequencies forming an harmonic series (integer multiples of the fundamental) to "group" together into a unified auditory percept whose pitch is similar to that of the fundamental. The aim of the study was to determine whether cortical auditory evoked potentials (AEPs) to complex tones differ according to whether the component frequencies of the stimuli are harmonically related or not. AEPs were recorded to continuous complex tones comprising four or more sinusoids. The vertex-maximal "change-potentials" (CP1, CN1, CP2), recorded to a stimulus cycle comprising one harmonic and five inharmonic complexes changing every second, showed no sensitivity to harmonicity, although an additional mismatch negativity was possibly present to the harmonic complex. In a second study the CP2 was significantly attenuated when an harmonic complex changed to a new one in the presence of an unchanging sinusoidal background tone, harmonically related to the first complex but not to the second, and thus becoming perceptually distinct. This, however, might be caused by lateral inhibitory effects not related to harmonicity. In a third experiment, when four concurrent sinusoidal tones came to rest on steady frequencies after a 5-s period of 16/s pseudo-random frequency changes, fronto-centrally maximal "mismatch-potentials" (MN1, MP2), were recorded. Both the MN1 and the MP2 were significantly shorter in latency when the steady frequencies formed an harmonic complex. Since the harmonic complex had a short overall periodicity, equal to that of the fundamental, while that of the inharmonic complex was much longer, the effect might be explained if the latencies of the mismatch-potential are related to periodicity. The perceptual grouping of harmonically related frequencies appears not to be a function of spectral domain analysis, reflected in the change-potentials, but of periodicity analysis, reflected in the mismatch-potentials

Inharmonic Sounds: Implications as to «Pitch», «Timbre» and «Consonance»

This paper relates the structure of certain types of inharmonic complex sounds such as produced by idiophones (gongs, bells, etc.) to perceptual observations as well as to models of pitch and timbre perception. Since most of the current models operate in the time domain focussing on the concept of periodicity, it is argued that perception of pitch of inharmonic sounds, in general, is more ambiguous because (quasi-)periodicity in many such signals is more difficult to detect. Periodicity typically deteriorates with increasing inharmonicity of spectral components while it is maximum in perfectly harmonic spectra. Thereby a clear relationship between the temporal structure of a signal as reflected by its time function and wave-shape, and its spectral composition can be established that is of relevance also as to perception of timbre and consonance.

Influence of source and echo separation on echo threshold of natural complex sounds

Most studies of the precedence effect are based on stimuli that differ dramatically from ‘‘realistic’’ stimuli. Important differences include the following. (a) Stimulus types used in studies are often clicks, noise bursts or sinusoids; realistic sounds are mostly harmonic or inharmonic complex tones. (b) Durations of stimuli used in studies often avoid a temporal overlap between lead and lag; durations of realistic sounds are normally such that first and subsequent wave fronts overlap most of the time. (c) Lead and lag stimuli used in studies to simulate a primary sound and its reflection from a nearby surface are often signals of equal amplitude and identical wave form; reflections of realistic sounds differ normally from the primary sound in amplitude and wave form. As a first step toward understanding the precedence effect and its significance in ‘‘real world’’ situations, experiments with synthetic piano tones of durations longer than 100 ms (longer than echo delays in typical rooms) were conducted in a simulated reverberant environment. The dependence of echo threshold on the direction of sound incidence was studied using measurements of fusion and localization dominance. The results show a decrease of echo threshold with an increase of spatial separation between primary source and reflection.

Perceptual segregation and pitch shifts of mistuned components in harmonic complexes and in regular inharmonic complexes.

It is unclear whether the perceptual segregation of a mistuned harmonic from a periodic complex tone depends specifically on harmonic relations between the other components. A procedure used previously for harmonic complexes [W. M. Hartmann et al., J. Acoust. Soc. Am. 88, 1712-1724 (1990)] was adapted and extended to regular inharmonic complexes. On each trial, subjects heard a 12-component complex followed by a pure tone in a continuous loop. In experiment 1, a mistuning of +/- 4% was applied to one of the components 2-11. The complex was either harmonic, frequency shifted, or spectrally stretched. Subjects adjusted the pure tone to match the pitch of the mistuned component. Near matches were taken to indicate segregation, and were almost as frequent in the inharmonic conditions as in the harmonic case. Also, small but consistent mismatches, pitch shifts, were found in all conditions. These were similar in direction and size to earlier findings for harmonic complexes. Using a range of mistunings, experiment 2 showed that the segregation of components from regular inharmonic complexes could be sensitive to mistunings of 1.5% or less. These findings are consistent with the proposal that aspects of spectral regularity other than harmonic relations can also influence auditory grouping.

Perceptual segregation and pitch shifts of mistuned components in harmonic complexes and in regular inharmonic complexes

It is unclear whether the perceptual segregation of a mistuned harmonic from a periodic complex tone depends specifically on harmonic relations between the other components. A procedure used previously for harmonic complexes [W. M. Hartmann, S. McAdams, and B. K. Smith, J. Acoust. Soc. Am. 88, 1712–1724 (1990)] was adapted and extended to regular inharmonic complexes. On each trial, subjects heard a 12-component complex followed by a pure tone in a continuous loop. A mistuning of ±4% was applied to one of the components 2–11. Subjects adjusted the pure tone to match the pitch of the mistuned component. Near matches were taken to indicate segregation. The complexes were either harmonic, frequency shifted, or spectrally stretched. Performance in the inharmonic conditions was generally as accurate as in the harmonic condition. Small but consistent mismatches, pitch shifts, were found in all conditions. Their direction and size were similar to earlier findings for harmonic complexes. Subsequent tests using a range of mistunings showed that the perceptual segregation of components from regular inharmonic complexes could be sensitive to mistunings of 1.5% or less. These findings suggest that the effects of spectral regularity on auditory grouping are more general than those of harmonic relations.