Harmonic Complex Research Articles

Musical expertise and concurrent sound segregation

There is growing evidence suggesting that musical training can improve performance in various auditory perceptual tasks. These improvements can be paralleled by changes in scalp recorded auditory evoked potentials (AEPs). The present study examined whether musical training modulates the ability to segregate concurrent auditory objects using behavioral measures and AEPs. Expert musicians and nonmusicians were presented with complex sounds comprised of six harmonics (220, 440, 660 Hz, etc.). The third harmonic was either tuned or mistuned by 1%–16% of its original value. Mistuning a component of a harmonic complex results in the precept of a second auditory object. Stimuli were presented passively (no response) and actively (participants responded by indicating if they heard one sound or two sounds). Behaviorally both musicians and nonmusicians perceived a second auditory object at similar levels of mistuning. In both groups, complex sounds generated N1 and P2 waves at fronto-central scalp regions. The perception of concurrent auditory objects was paralleled by an increased negativity around 150 ms post-stimulus onset. This increased negativity is referred to as object-related negativity (ORN). Small differences between musicians and nonmusicians were noted in the ORN. The implication of these results will be discussed in terms of current auditory scene analysis theory.

Pitch cue learning in chinchillas: The role of spectral region in the training stimulus

Chinchillas were trained to discriminate a cosine-phase harmonic tone complex (COS) from wideband noise (WBN) and tested in a stimulus generalization paradigm with tone complexes in which phase differed between frequency regions. In this split-phase condition, responses to complexes made of random-phase low frequencies, cosine-phase high frequencies were similar to responses to the COS-training stimulus. However, responses to complexes made of cosine-phase low frequencies, random-phase high frequencies were generally lower than their responses to the COS-training stimulus. When tested with sine-phase (SIN) and random-phase (RND) tone complexes, responses were large for SIN, but were small for RND. Chinchillas were then trained to discriminate infinitely-iterated rippled noise (IIRN) from WBN and tested with noises in which the spectral ripple differed between frequency regions. In this split-spectrum condition, responses were large to noises made of rippled-spectrum low frequencies, flat-spectrum high frequencies, whereas responses were generally lower to noises made of flat-spectrum low frequencies, rippled-spectrum high frequencies. The results suggest that chinchillas listen across all frequencies, but attend to high frequencies when discriminating COS from WBN and attend to low frequencies when discriminating IIRN from WBN.

Detection and F discrimination of harmonic complex tones in the presence of competing tones or noise

Normal-hearing listeners' ability to "hear out" the pitch of a target harmonic complex tone (HCT) was tested with simultaneous HCT or noise maskers, all bandpass-filtered into the same spectral region (1200-3600 Hz). Target-to-masker ratios (TMRs) necessary to discriminate fixed fundamental-frequency (F0) differences were measured for target F0s between 100 and 400 Hz. At high F0s (400 Hz), asynchronous gating of masker and signal, presenting the masker in a different F0 range, and reducing the F0 rove of the masker, all resulted in improved performance. At the low F0s (100 Hz), none of these manipulations improved performance significantly. The findings are generally consistent with the idea that the ability to segregate sounds based on cues such as F0 differences and onset/offset asynchronies can be strongly limited by peripheral harmonic resolvability. However, some cases were observed where perceptual segregation appeared possible, even when no peripherally resolved harmonics were present in the mixture of target and masker. A final experiment, comparing TMRs necessary for detection and F0 discrimination, showed that F0 discrimination of the target was possible with noise maskers at only a few decibels above detection threshold, whereas similar performance with HCT maskers was only possible 15-25 dB above detection threshold.

Testing the role of harmonic number in the contribution of a nonsimultaneous mistuned harmonic to residue pitch

Ciocca and Darwin [J. Acoust. Soc. Am. 105, 2421–2430 (1999)] reported a surprising finding: Mistuning the fourth harmonic produced the same residue pitch shift when presented after the remainder of the complex as when it was simultaneous. They concluded that partials are integrated over a period of 170–250 ms for the calculation of pitch. Gockel et al. [J. Acoust. Soc. Am. 118, 3783–3793 (2005)] failed to replicate this result, but mistuned the third harmonic. Here, subjects adjusted a matching sound (harmonic complex) so that its pitch equaled that of a subsequent 90-ms complex tone (12 harmonics of a 155-Hz F0) containing a mistuned (±3%) third or fourth harmonic. Pitch shifts were significantly larger when: (i) the mistuned harmonic was presented simultaneously with, rather than after, the remainder of the target complex; (ii) the fourth rather than the third harmonic was mistuned; (iii) the nominally mistuned harmonic was present in, rather than absent from, the matching sound. The results again contrast with Ciocca and Darwin’s findings and contradict the idea of obligatory integration over a long time period for the formation of residue pitch. [Work supported by EPSRC Grant EP/D501571/1.]

Acoustic cues for improved speech recognition in combined acoustic and electric hearing

Speech recognition in noise improves with combined acoustic and electric hearing compared to electric hearing alone. Here, we investigate the contribution of four low-frequency hearing cues (fundamental frequency (F0), first formant (F1), voicing, and glimpsing) to speech recognition in combined hearing. Normal-hearing listeners heard vocoded speech in one ear and low-pass (LP) filtered speech in the other. Three conditions (vocode alone, LP alone, and combined) were tested. Target speech with an average F0 of 120 Hz was mixed with a time-reversed masker sentence (average F0=172 Hz) at three SNRs (5,10,15 dB). The LP speech aided performance at all three SNRs. F1 cues were then removed by replacing the LP speech with an LP equal-amplitude harmonic complex with the same F0 contour as the target speech, and an amplitude that was modulated by that of the target’s voiced portions. The benefits of combined hearing disappeared at 10- and 15-dB SNR, but persisted at SNR=5 dB. The same happened when, additionally, F0 cues were removed by fixing the F0 of the harmonic complex at 150 Hz. The results are consistent with a role for the F1 cue and of voicing and/or glimpsing cues, but not with a combination of F0 information between the two ears.

Responses of cochlear nucleus units recorded from chinchillas trained to discriminate iterated rippled noise from flat-spectrum noise

Behavioral responses obtained from chinchillas using a stimulus generalization paradigm show that the perception of pitch strength is dependent on the stimuli used for training. When animals are trained to discriminate a cosine-phase harmonic tone complex (COS) from a flat-spectrum, wideband noise (WBN), generalization gradients indicate that animals rely primarily on temporal information in the envelope. Gradients from animals retrained to discriminate infinitely iterated rippled noise (IIRN) from WBN indicate animals rely more on information in the fine structure. Thus, chinchillas learn to use the information in the fine structure when trained with an appropriate stimulus. In order to begin to examine whether there exists a neural correlate of this learning at the level of the cochlear nucleus, single-unit responses to IIRNs and WBN were recorded from three anesthetized chinchillas that had received considerable behavioral training in discriminating IIRNs from WBN. Preliminary data were obtained for 19 single units consisting of primarylike, chopper, and onset response types. All-order interspike-interval histograms obtained for IIRNs and WBN were qualitatively similar to those obtained from naive, untrained animals for each unit type. Quantitative comparisons among units recorded from trained and untrained animals will be presented. [Work supported by NIDCD R01 DC005596.]

Phase effects in masking by harmonic complexes in birds

Masking by harmonic complexes depends on the frequency content of the masker and its phase spectrum. Harmonic complexes created with negative Schroeder phases (component phases decreasing with increasing frequency) produce more masking than those with positive Schroeder phases (increasing phase) in humans, but not in birds. The masking differences in humans have been attributed to interactions between the masker phase spectrum and the phase characteristic of the basilar membrane. In birds, the similarity in masking by positive and negative Schroeder maskers, and reduced masking by cosine-phase maskers (constant phase), suggests a phase characteristic that does not change much along the basilar papilla. To evaluate this possibility, the rate of phase change across masker bandwidth was varied by systematically altering the Schroeder algorithm. Humans and three species of birds detected tones added in phase to a single component of a harmonic complex. As observed in earlier studies, the minimum amount of masking in humans occurred for positive phase gradients. However, minimum masking in birds occurred for a shallow negative phase gradient. These results suggest a cochlear delay in birds that is reduced compared to that found in humans, probably related to the shorter avian basilar epithelia.

Effects of Relative Phases on Discrimination of Harmonic Complex Tones

청각시스템의 심리음향학적 기능에 대해서 실제 자연 현상과 유사한 조건에서 측정하기 위해서는 순음(pure tone)보다 복합음(complex stimuli), 특히 기본주파수(fundamental frequency)를 포함하는 harmonic complex tone을 이용하는 것이 적절하다고 제안되어왔다.1)9)10) 이러한 harmonic complex tone은 일반적으로 관찰되는 음의 복합성(complexity)을 지니며, 정밀하게 디지털 합성을 할 수가 있고 그 변수조작이 용이하다는 장점이 있다. 그러나 harmonic complex tone의 구성성분들이 동일하더라도 심리적으로 느끼는 음의 고저(pitch), 음의 크기(loudness), 음질(timber) 이 대상마다 다를 수가 있다. 이러한 harmonic complextone에 대한 심리적 인지에 영향을 주는 요인으로 fundamental frequency,6)7) mistuned frequency,3) unequalled-level,2) stimulus duration3) 등이 있다는 여러 연구들이 보고 된 적은 있으나 구성성분 위상의 역할에 대한 연구는 아직 미흡한 실정이다. Helmholtz는‘청각시스템은 harmonic complex tone의 상대적 위상변화에 민감하지 않는다’고 하였고, Goldstein은‘청각시스템은‘phase-deaf’는 아니며 phase는 제한된 범위 내에서 인지적으로 중요하다’고 제안했었다.6) 그러나 일부 논문에 의하면 청각시스템은 phase 에 민감하며 특히 높은 차수의 배음(higher harmonics)의 변화에 민감하다는 보고도 있었다.6)7) 현재까지 주파수와 강도가 인지적인 부분에서 서로 영향을 준다는 보고는 잘 알려져 왔으나 앞서 언급된 것 같이 구성성분 위상의 영향에 대해서 behavioral 검사를 통한 연구는 상대적으로 부족한 실정이다. 따라서 본 연구에서는 정상 청력인을 대상으로 구성성분 위상이 harmonic complex tone을 변별하는데 있어서 중요한 변수로서 작용할 수 있는지를 체계적으로 분석하고자 한다.

Reduced contribution of a nonsimultaneous mistuned harmonic to residue pitch

Ciocca and Darwin [V. Ciocca and C. J. Darwin, J. Acoust. Soc. Am. 105, 2421-2430 (1999)] reported that the shift in residue pitch caused by mistuning a single harmonic (the fourth out of the first 12) was the same when the mistuned harmonic was presented after the remainder of the complex as when it was simultaneous, even though subjects were asked to ignore the pure-tone percept. The present study tried to replicate this result, and investigated the role of the presence of the nominally mistuned harmonic in the matching sound. Subjects adjusted a "matching" sound so that its pitch equaled that of a subsequent 90-ms complex tone (12 harmonics of a 155-Hz F0), whose mistuned (+/-3%) third harmonic was presented either simultaneously with or after the remaining harmonics. In experiment 1, the matching sound was a harmonic complex whose third harmonic was either present or absent. In experiments 2A and 2B, the target and matching sound had nonoverlapping spectra. Pitch shifts were reduced both when the mistuned component was nonsimultaneous, and when the third harmonic was absent in the matching sound. The results indicate a shorter than originally estimated time window for obligatory integration of nonsimultaneous components into a virtual pitch.

Structural, Functional, and Perceptual Differences in Heschl's Gyrus and Musical Instrument Preference

The musical pitch of harmonic complex sounds, such as instrumental sounds, is perceived primarily by decoding either the fundamental pitch (keynote) or spectral aspects of the stimuli, for example, single harmonics. We divided 334 professional musicians, including symphony orchestra musicians, 75 amateur musicians, and 54 nonmusicians, into either fundamental pitch listeners or spectral pitch listeners. We observed a strong correlation between pitch perception preference and asymmetry of brain structure and function in the pitch-sensitive lateral areas of Heschl's gyrus (HG), irrespective of musical ability. In particular, fundamental pitch listeners exhibited both larger gray matter volume measured using magnetic resonance imaging (MRI) and enhanced P50m activity measured using magnetoencephalography (MEG) in the left lateral HG, which is sensitive to rapid temporal processing. Their chosen instruments were percussive or high-pitched instruments that produce short, sharp, or impulsive tones (e.g., drums, guitar, piano, trumpet, or flute). By contrast, spectral pitch listeners exhibited a dominant right lateral HG, which is known to be sensitive to slower temporal and spectral processing. Their chosen instruments were lower-pitched melodic instruments that produce rather sustained tones with characteristic changes in timbre (e.g., bassoon, saxophone, french horn, violoncello, or organ). Singers also belonged to the spectral pitch listeners. Furthermore, the absolute size of the neural HG substrate depended strongly on musical ability. Overall, it is likely that both magnitude and asymmetry of lateral HG, and the related perceptual mode, may have an impact on preference for particular musical instruments and on musical performance.

Listening experience with iterated rippled noise alters the perception of ‘pitch’ strength of complex sounds in the chinchilla

Behavioral responses obtained from chinchillas trained to discriminate a cosine-phase harmonic tone complex from wideband noise indicate that the perception of 'pitch' strength in chinchillas is largely influenced by periodicity information in the stimulus envelope. The perception of 'pitch' strength was examined in chinchillas in a stimulus generalization paradigm after animals had been retrained to discriminate infinitely iterated rippled noise from wideband noise. Retrained chinchillas gave larger behavioral responses to test stimuli having strong fine structure periodicity, but weak envelope periodicity. That is, chinchillas learn to use the information in the fine structure and consequently, their perception of 'pitch' strength is altered. Behavioral responses to rippled noises having similar periodicity strengths, but large spectral differences were also tested. Responses to these rippled noises were similar, suggesting a temporal analysis can be used to account for the behavior. Animals were then retested using the cosine-phase harmonic tone complex as the expected signal stimulus. Generalization gradients returned to those obtained originally in the naïve condition, suggesting that chinchillas do not remain "fine structure listeners," but rather revert back to being "envelope listeners" when the periodicity strength in the envelope of the expected stimulus is high.

Pitch circularity produced by varying the amplitudes of odd and even harmonics

Pitch circularities have been produced using tones whose components stand in octave relation (Shepard, 1964; Risset, 1969). This paper describes two circular pitch scales produced by a new algorithm. Each scale consists of a bank of harmonic complex tones, each tone 500 ms in duration. As the scale descends in semitone steps, the relative amplitudes of the odd‐numbered harmonics decrease in 5‐dB steps, so that at the bottom of the scale they are 55 dB lower than the even‐numbered harmonics. The entire bank of tones is then low‐pass filtered. The lowest tone of such a scale is therefore heard as though its fundamental frequency were displaced up an octave. For each scale all possible (i.e., 132) ordered tone pairs were presented, and 15 subjects judged for each pair whether the second tone was higher or lower than the first. The data derived from these pairwise comparisons were subjected to Kruskal’s nonmetric multidimensional scaling (MDS). For both scales, two‐dimensional plots yielded approximately circular configurations; Stress1 values for the two scales were 0.016 and 0.034. When the tones were presented in semitone steps, the impression of infinitely ascending and descending scales was obtained. These circular scales are demonstrated with sound examples.

Tonal strength of harmonic complex tones in machinery noise

The sounds of machines often contain families of harmonically related sine waves that are referred to as harmonic complex tones. The perceived tonal strength of these types of sounds can adversely influence people’s impressions of the sound. While a complex tone is comprised of many sine waves, usually only one prominent pitch sensation is produced. It can be argued that harmonic complex tones are perceived as a single entities, not as a sum of individual tones. A series of psychoacoustics tests was conducted to evaluate tonal prominence of harmonic complex tones. Two sounds of equal loudness were played to subjects. One was a harmonic complex tone in noise and the other was a single tone in noise. Subjects were asked to equalize the perceived tonalness of the two sounds by adjusting the tone to noise ratio of the single tone in noise. Tonalness and Terhardt’s pitch perception models were applied to the pairs of sounds used in each test. The feasibility of replacing harmonic complex tones with a tonally equivalent simple sound was investigated, and strong correlations between Aures’ tonality for the simple and complex tones were found.

Structural and functional asymmetry of lateral Heschl's gyrus reflects pitch perception preference

The relative pitch of harmonic complex sounds, such as instrumental sounds, may be perceived by decoding either the fundamental pitch (f0) or the spectral pitch (fSP) of the stimuli. We classified a large cohort of 420 subjects including symphony orchestra musicians to be either f0 or fSP listeners, depending on the dominant perceptual mode. In a subgroup of 87 subjects, MRI (magnetic resonance imaging) and magnetoencephalography studies demonstrated a strong neural basis for both types of pitch perception irrespective of musical aptitude. Compared with f0 listeners, fSP listeners possessed a pronounced rightward, rather than leftward, asymmetry of gray matter volume and P50m activity within the pitch-sensitive lateral Heschl's gyrus. Our data link relative hemispheric lateralization with perceptual stimulus properties, whereas the absolute size of the Heschl's gyrus depends on musical aptitude.

Prevalence of Stereotypical Responses to Mistuned Complex Tones in the Inferior Colliculus

The human auditory system has an exceptional ability to separate competing sounds, but the neural mechanisms that underlie this ability are not understood. Responses of inferior colliculus (IC) neurons to "mistuned" complex tones were measured to investigate possible neural mechanisms for spectral segregation. A mistuned tone is a harmonic complex tone in which the frequency of one component has been changed; that component may be heard as a separate sound source, suggesting that the mistuned tone engages the same mechanisms that contribute to the segregation of natural sounds. In this study, the harmonic tone consisted of eight harmonics of 250 Hz; in the mistuned tone, the frequency of the fourth harmonic was increased by 12% (120 Hz). The mistuned tone elicited a stereotypical discharge pattern, consisting of peaks separated by about 8 ms and a response envelope modulated with a period of 100 ms, which bore little resemblance to the discharge pattern elicited by the harmonic tone or to the stimulus waveform. Similar discharge patterns were elicited from many neurons with a range of characteristic frequencies, especially from neurons that exhibited short-latency sustained responses to pure tones. In contrast, transient and long-latency neurons usually did not exhibit the stereotypical discharge pattern. The discharge pattern was generally stable when the stimulus level or component phase was varied; the major effect of these manipulations was to shift the phase of the response envelope. Simulation of IC responses with a computational model suggested that off-frequency inhibition could produce discharge patterns with these characteristics.

Perception of Spectral Contrast by Hearing-Impaired Listeners

The ability to discriminate the spectral shapes of complex sounds is critical to accurate speech perception. Part of the difficulty experienced by listeners with hearing loss in understanding speech sounds in noise may be related to a smearing of the internal representation of the spectral peaks and valleys because of the loss of sensitivity and an accompanying reduction in frequency resolution. This study examined the discrimination by hearing-impaired listeners of highly similar harmonic complexes with a single spectral peak located in 1 of 3 frequency regions. The minimum level difference between peak and background harmonics required to discriminate a small change in the spectral center of the peak was measured for peaks located near 2, 3, or 4 kHz. Component phases were selected according to an algorithm thought to produce either highly modulated (positive Schroeder) or very flat (negative Schroeder) internal waveform envelopes in the cochlea. The mean amplitude difference between a spectral peak and the background components required for discrimination of pairs of harmonic complexes (spectral contrast threshold) was from 4 to 19 dB greater for listeners with hearing impairment than for a control group of listeners with normal hearing. In normal-hearing listeners, improvements in threshold were seen with increasing stimulus level, and there was a strong effect of stimulus phase, as the positive Schroeder stimuli always produced lower thresholds than the negative Schroeder stimuli. The listeners with hearing loss showed no consistent spectral contrast effects due to stimulus phase and also showed little improvement with increasing stimulus level, once their sensitivity loss was overcome. The lack of phase and level effects may be a result of the more linear processing occurring in impaired ears, producing poorer-than-normal frequency resolution, a loss of gain for low amplitudes, and an altered cochlear phase characteristic in regions of damage.

The neuronal representation of pitch in primate auditory cortex

Pitch perception is critical for identifying and segregating auditory objects, especially in the context of music and speech. The perception of pitch is not unique to humans and has been experimentally demonstrated in several animal species. Pitch is the subjective attribute of a sound's fundamental frequency (f(0)) that is determined by both the temporal regularity and average repetition rate of its acoustic waveform. Spectrally dissimilar sounds can have the same pitch if they share a common f(0). Even when the acoustic energy at f(0) is removed ('missing fundamental') the same pitch is still perceived. Despite its importance for hearing, how pitch is represented in the cerebral cortex is unknown. Here we show the existence of neurons in the auditory cortex of marmoset monkeys that respond to both pure tones and missing fundamental harmonic complex sounds with the same f(0), providing a neural correlate for pitch constancy. These pitch-selective neurons are located in a restricted low-frequency cortical region near the anterolateral border of the primary auditory cortex, and is consistent with the location of a pitch-selective area identified in recent imaging studies in humans.

Perception of dissonance by people with normal hearing and sensorineural hearing loss

The purpose of this study was to determine whether the perceived sensory dissonance of pairs of pure tones (PT dyads) or pairs of harmonic complex tones (HC dyads) is altered due to sensorineural hearing loss. Four normal-hearing (NH) and four hearing-impaired (HI) listeners judged the sensory dissonance of PT dyads geometrically centered at 500 and 2000 Hz, and of HC dyads with fundamental frequencies geometrically centered at 500 Hz. The frequency separation of the members of the dyads varied from 0 Hz to just over an octave. In addition, frequency selectivity was assessed at 500 and 2000 Hz for each listener. Maximum dissonance was perceived at frequency separations smaller than the auditory filter bandwidth for both groups of listners, but maximum dissonance for HI listeners occurred at a greater proportion of their bandwidths at 500 Hz than at 2000 Hz. Further, their auditory filter bandwidths at 500 Hz were significantly wider than those of the NH listeners. For both the PT and HC dyads, curves displaying dissonance as a function of frequency separation were more compressed for the HI listeners, possibly reflecting less contrast between their perceptions of consonance and dissonance compared with the NH listeners.

An autocorrelation model with place dependence to account for the effect of harmonic number on fundamental frequency discrimination

Fundamental frequency (f0) difference limens (DLs) were measured as a function of f0 for sine- and random-phase harmonic complexes, bandpass filtered with 3-dB cutoff frequencies of 2.5 and 3.5 kHz (low region) or 5 and 7 kHz (high region), and presented at an average 15 dB sensation level (approximately 48 dB SPL) per component in a wideband background noise. Fundamental frequencies ranged from 50 to 300 Hz and 100 to 600 Hz in the low and high spectral regions, respectively. In each spectral region, f0 DLs improved dramatically with increasing f0 as approximately the tenth harmonic appeared in the passband. Generally, f0 DLs for complexes with similar harmonic numbers were similar in the two spectral regions. The dependence of f0 discrimination on harmonic number presents a significant challenge to autocorrelation (AC) models of pitch, in which predictions generally depend more on spectral region than harmonic number. A modification involving a "lag window"is proposed and tested, restricting the AC representation to a limited range of lags relative to each channel's characteristic frequency. This modified unitary pitch model was able to account for the dependence of f0 DLs on harmonic number, although this correct behavior was not based on peripheral harmonic resolvability.

A New Technique for Unbalance Current and Voltage Estimation With Neural Networks

In this paper, a new measurement procedure based on neural networks for the estimation of harmonic powers and current/voltage-symmetrical components is presented. The theory foundation is the Park vectors representation of a three-phase voltage/current. The measurement system scheme is built with three neural network blocks. The first block is a feedforward neural network that computes the Park vectors and the zero-phase sequence components. The second block is an adaptive linear neuron (ADALINE) that estimates the harmonic complex coefficients of the current/voltage Park vectors. A third block is another feedforward neural network that obtains symmetrical components of current/voltage harmonics and harmonic active/reactive powers. Finally, to check the measurement method performance, the digital simulation results of a practical case are presented.