Sound Segregation Research Articles

Concurrent Sound Segregation in Electric and Acoustic Hearing

We investigated potential cues to sound segregation by cochlear implant (CI) and normal-hearing (NH) listeners. In each presentation interval of experiment 1a, CI listeners heard a mixture of four pulse trains applied concurrently to separate electrodes, preceded by a "probe" applied to a single electrode. In one of these two intervals, which the subject had to identify, the probe electrode was the same as a "target" electrode in the mixture. The pulse train on the target electrode had a higher level than the others in the mixture. Additionally, it could be presented either with a 200-ms onset delay, at a lower rate, or with an asynchrony produced by delaying each pulse by about 5 ms re those on the nontarget electrodes. Neither the rate difference nor the asynchrony aided performance over and above the level difference alone, but the onset delay produced a modest improvement. Experiment 1b showed that two subjects could perform the task using the onset delay alone, with no level difference. Experiment 2 used a method similar to that of experiment 1, but investigated the onset cue using NH listeners. In one condition, the mixture consisted of harmonics 5 to 40 of a 100-Hz fundamental, with the onset of either harmonics 13 to 17 or 26 to 30 delayed re the rest. Performance was modest in this condition, but could be improved markedly by using stimuli containing a spectral gap between the target and nontarget harmonics. The results suggest that (a) CI users are unlikely to use temporal pitch differences between adjacent channels to separate concurrent sounds, and that (b) they can use onset differences between channels, but the usefulness of this cue will be compromised by the spread of excitation along the nerve-fiber array. This deleterious effect of spread-of-excitation can also impair the use of onset cues by NH listeners.

Toward a neurophysiological theory of auditory stream segregation.

Auditory stream segregation (or streaming) is a phenomenon in which 2 or more repeating sounds differing in at least 1 acoustic attribute are perceived as 2 or more separate sound sources (i.e., streams). This article selectively reviews psychophysical and computational studies of streaming and comprehensively reviews more recent neurophysiological studies that have provided important insights into the mechanisms of streaming. On the basis of these studies, segregation of sounds is likely to occur beginning in the auditory periphery and continuing at least to primary auditory cortex for simple cues such as pure-tone frequency but at stages as high as secondary auditory cortex for more complex cues such as periodicity pitch. Attention-dependent and perception-dependent processes are likely to take place in primary or secondary auditory cortex and may also involve higher level areas outside of auditory cortex. Topographic maps of acoustic attributes, stimulus-specific suppression, and competition between representations are among the neurophysiological mechanisms that likely contribute to streaming. A framework for future research is proposed.

Multiple fundamental frequency estimation assisted by audio watermarking

The success of computer segregation of sound sources from a single‐channel mixture often relies on the estimation of multiple fundamental frequencies. Instead of solving the problem directly, this presented work utilizes audio watermarks to assist sound source segregation. Here, individual sources are assumed to be available for watermark embedding before mixing. Each source’s short‐time spectral peaks are aligned to frequency gridpoints labeled with a digital ID [Liu and Smith, ‘‘Audio Watermarking Based on Sinusoidal Analysis and Synthesis,’’ International Symposium on Musical Acoustics, Nara, Japan (2004)]. A modified sound source is sinusoidally synthesized afterwards. In this sense, each source is embedded with a watermark. In an experiment, two sound sources are watermarked with two sets of gridpoints P and Q, respectively. Points in P and Q interleave and are spaced 10 cents of a semitone apart. The watermarked sources are then linearly added. To demix them, sinusoidal trajectories found in the mix...

Musical expertise and concurrent sound segregation

There is growing evidence suggesting that musical training can improve performance in various auditory perceptual tasks. These improvements can be paralleled by changes in scalp recorded auditory evoked potentials (AEPs). The present study examined whether musical training modulates the ability to segregate concurrent auditory objects using behavioral measures and AEPs. Expert musicians and nonmusicians were presented with complex sounds comprised of six harmonics (220, 440, 660 Hz, etc.). The third harmonic was either tuned or mistuned by 1%–16% of its original value. Mistuning a component of a harmonic complex results in the precept of a second auditory object. Stimuli were presented passively (no response) and actively (participants responded by indicating if they heard one sound or two sounds). Behaviorally both musicians and nonmusicians perceived a second auditory object at similar levels of mistuning. In both groups, complex sounds generated N1 and P2 waves at fronto-central scalp regions. The perception of concurrent auditory objects was paralleled by an increased negativity around 150 ms post-stimulus onset. This increased negativity is referred to as object-related negativity (ORN). Small differences between musicians and nonmusicians were noted in the ORN. The implication of these results will be discussed in terms of current auditory scene analysis theory.

Sequential Auditory Scene Analysis Is Preserved in Normal Aging Adults

Normal aging is accompanied by speech perception difficulties, especially in adverse listening situations such as a cocktail party. To assess whether such difficulties might be related to impairments in sequential auditory scene analysis, event-related brain potentials were recorded from normal-hearing young, middle-aged, and older adults during presentation of low (A) tones, high (B) tones, and silences (--) in repeating 3 tone triplets (ABA--). The likelihood of reporting hearing 2 streams increased as a function of the frequency difference between A and B tones (Delta f) to the same extent for all 3 age groups and was paralleled by enhanced sensory-evoked responses over the frontocentral scalp regions. In all 3 age groups, there was also a progressive buildup in brain activity from the beginning to the end of the sequence of triplets, which was characterized by an enhanced positivity that peaked at about 200 ms after the onset of each ABA--triplet. Similar Delta f- and buildup-related activity also occurred over the right temporal cortex, but only for young adults. We conclude that age-related difficulties in separating competing speakers are unlikely to arise from deficits in streaming and might instead reflect less efficient concurrent sound segregation.

Effects of bilateral versus unilateral hearing aid fitting on abilities measured by the Speech, Spatial, and Qualities of Hearing scale (SSQ)

The Speech, Spatial and Qualities of Hearing Scale (Gatehouse & Noble, ) was applied to three independent clinical groups: 144 people prior to being fitted with amplification; 118 people with six months experience with unilateral amplification; and 42 people with six months experience with bilateral amplification. For traditional speech hearing contexts (one-on-one, in groups, in quiet, in noise) there was benefit with one aid, and no further benefit with two. By contrast, hearing speech in demanding contexts (divided or rapidly switching attention) showed benefit with one aid and further benefit with two. In the spatial domain, directional hearing showed some benefit with one hearing aid, and particular further benefit in distance and movement discrimination from fitting with two. There was some benefit from unilateral fitting for elements of the qualities domains (clarity, naturalness, recognisability, segregation of sounds), with no consistent sign of further benefit from two. Bilateral fitting added benefit with respect to listening effort. Two hearing aids offer advantage in demanding and dynamic contexts; these contexts are argued as significant in the maintenance of social competence and emotional wellbeing. The present results go toward establishing the real-world advantages of bilateral hearing aid fitting and suggest that previous, inconclusive clinical findings reflect inquiry limited to more traditional areas of hearing function.SumarioSe aplicó la escala de cualidades auditiva, espacial y del lenguaje (Gatehouse & Noble, ) a tres grupos clínicos independientes: 144 personas, antes de adaptarles auxiliares auditivos; 118 personas con 6 meses de experiencia con amplificación unilateral y 42 personas con 6 meses de experiencia con amplificación bilateral. Se encontró beneficio de la amplificación unilateral en el contexto de habla tradicional (uno a uno, en grupo, en silencio y en ruido) y no hubo beneficio posterior en el modo bilateral. En contraste, bajo condiciones de discriminación del lenguaje en contexto demandante (atención dividida o cambio de rápido de atención) hubo beneficio con un solo un auxiliar, y mayor beneficio con dos. En el dominio espacial, hubo cierto beneficio en la audición direccional con un solo auxiliar auditivo y particularmente resultó benéfica la condición bilateral en la discriminación de distancia y movimiento. Hubo cierto beneficio de la condición unilateral para elementos de cualidad (claridad, naturalidad, fácil discriminación, segregación de sonidos) sin signos consistentes de beneficio con amplificación bilateral. La amplificación bilateral benefició el menor esfuerzo para escuchar. Hay ventajas con dos auxiliares auditivos en contexto demandante o dinámico; estos contextos son considerados como significativos en el mantenimiento de la competencia social y el bienestar emocional. Estos resultados se encaminan a establecer las ventajas de la amplificación bilateral en el mundo real y sugieren que los hallazgos clínicos previos e inconclusos, reflejan información limitada a áreas más tradicionales de la función auditiva.

Effects of Attention on Neuroelectric Correlates of Auditory Stream Segregation

A general assumption underlying auditory scene analysis is that the initial grouping of acoustic elements is independent of attention. The effects of attention on auditory stream segregation were investigated by recording event-related potentials (ERPs) while participants either attended to sound stimuli and indicated whether they heard one or two streams or watched a muted movie. The stimuli were pure-tone ABA--patterns that repeated for 10.8 sec with a stimulus onset asynchrony between A and B tones of 100 msec in which the A tone was fixed at 500 Hz, the B tone could be 500, 625, 750, or 1000 Hz, and--was a silence. In both listening conditions, an enhancement of the auditory-evoked response (P1-N1-P2 and N1c) to the B tone varied with Deltaf and correlated with perception of streaming. The ERP from 150 to 250 msec after the beginning of the repeating ABA- patterns became more positive during the course of the trial and was diminished when participants ignored the tones, consistent with behavioral studies indicating that streaming takes several seconds to build up. The N1c enhancement and the buildup over time were larger at right than left temporal electrodes, suggesting a right-hemisphere dominance for stream segregation. Sources in Heschl's gyrus accounted for the ERP modulations related to Deltaf-based segregation and buildup. These findings provide evidence for two cortical mechanisms of streaming: automatic segregation of sounds and attention-dependent buildup process that integrates successive tones within streams over several seconds.

Spatial unmasking of birdsong in human listeners: Energetic and informational factors

Spatial unmasking describes the improvement in the detection or identification of a target sound afforded by separating it spatially from simultaneous masking sounds. This effect has been studied extensively for speech intelligibility in the presence of interfering sounds. In the current study, listeners identified zebra finch song, which shares many acoustic properties with speech but lacks semantic and linguistic content. Three maskers with the same long-term spectral content but different short-term statistics were used: (1) chorus (combinations of unfamiliar zebra finch songs), (2) song-shaped noise (broadband noise with the average spectrum of chorus), and (3) chorus-modulated noise (song-shaped noise multiplied by the broadband envelope from a chorus masker). The amount of masking and spatial unmasking depended on the masker and there was evidence of release from both energetic and informational masking. Spatial unmasking was greatest for the statistically similar chorus masker. For the two noise maskers, there was less spatial unmasking and it was wholly accounted for by the relative target and masker levels at the acoustically better ear. The results share many features with analogous results using speech targets, suggesting that spatial separation aids in the segregation of complex natural sounds through mechanisms that are not specific to speech.

Perceptual Organization of Tone Sequences in the Auditory Cortex of Awake Macaques

Acoustic sequences such as speech and music are generally perceived as coherent auditory "streams," which can be individually attended to and followed over time. Although the psychophysical stimulus parameters governing this "auditory streaming" are well established, the brain mechanisms underlying the formation of auditory streams remain largely unknown. In particular, an essential feature of the phenomenon, which corresponds to the fact that the segregation of sounds into streams typically takes several seconds to build up, remains unexplained. Here, we show that this and other major features of auditory-stream formation measured in humans using alternating-tone sequences can be quantitatively accounted for based on single-unit responses recorded in the primary auditory cortex (A1) of awake rhesus monkeys listening to the same sound sequences.

Right hemispheric predominance in the segregation of mistuned partials

To elucidate the central mechanisms of sound segregation, we compared responses to a harmonic sound and a mistuned sound using a whole-head magnetoencephalography system. The harmonic sound was composed of a 200-Hz tone and its 2nd to 12th harmonics. The mistuned sound had, instead of the 600-Hz harmonic, a 696-Hz tone. In the right hemisphere, the amplitude of N100m responses evoked by the mistuned sound was significantly larger and the peak latency significantly longer than that evoked by the harmonic sound, suggesting that the right hemisphere plays a more important role than the left in detecting mistuned partials.

Contribution of harmonicity and location to auditory object formation in free field: Evidence from event-related brain potentials

The contribution of location and harmonicity cues in sound segregation was investigated using behavioral reports and source waveforms derived from the scalp-recorded evoked potentials. Participants were presented with sounds composed of multiple harmonics in a free-field environment. The third harmonic was either tuned or mistuned and could be presented from the same or different location from the remaining harmonics. Presenting the third harmonic at a different location than the remaining harmonics increased the likelihood of hearing the tuned or slightly (i.e., 2%) mistuned harmonic as a separate object. Partials mistuned by 16% of their original value "pop out" of the complex and were paralleled by an object-related negativity (ORN) that superimposed the N1 and P2 components. For the 2% mistuned stimuli, the ORN was present only when the mistuned harmonic was presented at a different location than the remaining harmonics. Presenting the tuned harmonic at a different location also yielded changes in neural activity between 150 and 250 ms after sound onset. The behavioral and electrophysiological results indicate that listeners can segregate sounds based on harmonicity or location alone. The results also indicate that a conjunction of harmonicity and location cues contribute to sound segregation primarily when harmonicity is ambiguous.

Prevalence of Stereotypical Responses to Mistuned Complex Tones in the Inferior Colliculus

The human auditory system has an exceptional ability to separate competing sounds, but the neural mechanisms that underlie this ability are not understood. Responses of inferior colliculus (IC) neurons to "mistuned" complex tones were measured to investigate possible neural mechanisms for spectral segregation. A mistuned tone is a harmonic complex tone in which the frequency of one component has been changed; that component may be heard as a separate sound source, suggesting that the mistuned tone engages the same mechanisms that contribute to the segregation of natural sounds. In this study, the harmonic tone consisted of eight harmonics of 250 Hz; in the mistuned tone, the frequency of the fourth harmonic was increased by 12% (120 Hz). The mistuned tone elicited a stereotypical discharge pattern, consisting of peaks separated by about 8 ms and a response envelope modulated with a period of 100 ms, which bore little resemblance to the discharge pattern elicited by the harmonic tone or to the stimulus waveform. Similar discharge patterns were elicited from many neurons with a range of characteristic frequencies, especially from neurons that exhibited short-latency sustained responses to pure tones. In contrast, transient and long-latency neurons usually did not exhibit the stereotypical discharge pattern. The discharge pattern was generally stable when the stimulus level or component phase was varied; the major effect of these manipulations was to shift the phase of the response envelope. Simulation of IC responses with a computational model suggested that off-frequency inhibition could produce discharge patterns with these characteristics.

Combining energetic and informational masking for speech identification

This study examined combinations of energetic and informational maskers in speech identification. Speech targets and maskers (speech or noise) were processed and filtered into sets of 15 narrow frequency bands. The target was the sum of eight randomly selected bands. More masking occurred for speech maskers than for spectrally matched noise maskers regardless of whether the masker bands overlapped the target bands. The greater effect of the speech maskers was interpreted as due to informational masking. When the masker was comprised of nonoverlapping bands of speech, the addition of bands of noise overlapping the speech masker, but not the speech target, reduced the overall amount of masking. Surprisingly, presenting the noise to the ear contralateral to the target and masker produced an even greater release from masking. The contralateral noise was apparently sufficient to cause a slight change in the image of the ipsilateral speech masker, possibly pulling it away from the target enough to allow the focus of attention on the target. This finding is consistent with the interpretation that in some conditions small binaural differences may be sufficient to cause, or significantly strengthen, the perceptual segregation of sounds.

6.27 A paradigm to assess kinematic responses during anteroposterior translations in sitting following whiplash injuries

Natural soundscapes often contain multiple sound sources at any given time. Numerous studies have reported that in human observers, the perception and identification of concurrent sounds is paralleled by specific changes in cortical event-related potentials (ERPs). Although these studies provide a window into the cerebral mechanisms governing sound segregation, little is known about the subcortical neural architecture and hierarchy of neurocomputations that lead to this robust perceptual process. Using computational modeling, scalp-recorded brainstem/cortical ERPs, and human psychophysics, we demonstrate that a primary cue for sound segregation, i.e., harmonicity, is encoded at the auditory nerve level within tens of milliseconds after the onset of sound and is maintained, largely untransformed, in phase-locked activity of the rostral brainstem. As then indexed by auditory cortical responses, (in)harmonicity is coded in the signature and magnitude of the cortical object-related negativity (ORN) response (150–200 ms). The salience of the resulting percept is then captured in a discrete, categorical-like coding scheme by a late negativity response (N5; ~500 ms latency), just prior to the elicitation of a behavioral judgment. Subcortical activity correlated with cortical evoked responses such that weaker phase-locked brainstem responses (lower neural harmonicity) generated larger ORN amplitude, reflecting the cortical registration of multiple sound objects. Studying multiple brain indices simultaneously helps illuminate the mechanisms and time-course of neural processing underlying concurrent sound segregation and may lead to further development and refinement of physiologically driven models of auditory scene analysis.

Age-related changes in neural activity associated with concurrent vowel segregation

Older adults exhibit degraded speech comprehension in complex sound environments, which may be related to overall age-related declines in low-level sound segregation. This hypothesis was tested by measuring event-related potentials (ERPs) while listeners identified two different vowels presented simultaneously. Older adults were less accurate than young adults at identifying the two vowels, although both groups improved similarly with increasing fundamental frequency differences (Δ ƒ 0) between vowels. Reaction time data showed that older adults took more time to process stimuli, especially those with smaller Δ ƒ 0. A negative ERP wave indexing the automatic registration of Δ ƒ 0 (the object-related negativity) was reduced in older adults. In contrast, young and older adults showed a similar pattern of neural activity indexing attentive processing of Δ ƒ 0. The results suggest that aging affects the ability to automatically segregate speech sounds.

Left thalamo-cortical network implicated in successful speech separation and identification

The separation of concurrent sounds is paramount to human communication in everyday settings. The primary auditory cortex and the planum temporale are thought to be essential for both the separation of physical sound sources into perceptual objects and the comparison of those representations with previously learned acoustic events. To examine the role of these areas in speech separation, we measured brain activity using event-related functional Magnetic Resonance Imaging (fMRI) while participants were asked to identify two phonetically different vowels presented simultaneously. The processing of brief speech sounds (200 ms in duration) activated the thalamus and superior temporal gyrus bilaterally, left anterior temporal lobe, and left inferior temporal gyrus. A comparison of fMRI signals between trials in which participants successfully identified both vowels as opposed to when only one of the two vowels was recognized revealed enhanced activity in left thalamus, Heschl's gyrus, superior temporal gyrus, and the planum temporale. Because participants successfully identified at least one of the two vowels on each trial, the difference in fMRI signal indexes the extra computational work needed to segregate and identify successfully the other concurrently presented vowel. The results support the view that auditory cortex in or near Heschl's gyrus as well as in the planum temporale are involved in sound segregation and reveal a link between left thalamo-cortical activation and the successful separation and identification of simultaneous speech sounds.

Integration and segregation in auditory scene analysis

Assessment of the neural correlates of auditory scene analysis, using an index of sound change detection that does not require the listener to attend to the sounds [a component of event-related brain potentials called the mismatch negativity (MMN)], has previously demonstrated that segregation processes can occur without attention focused on the sounds and that within-stream contextual factors influence how sound elements are integrated and represented in auditory memory. The current study investigated the relationship between the segregation and integration processes when they were called upon to function together. The pattern of MMN results showed that the integration of sound elements within a sound stream occurred after the segregation of sounds into independent streams and, further, that the individual streams were subject to contextual effects. These results are consistent with a view of auditory processing that suggests that the auditory scene is rapidly organized into distinct streams and the integration of sequential elements to perceptual units takes place on the already formed streams. This would allow for the flexibility required to identify changing within-stream sound patterns, needed to appreciate music or comprehend speech.

Understanding response proclivity and the limits of sensory capability: What do we hear and what can we hear?

Charles Watson’s studies of informational masking and the effects of stimulus uncertainty on auditory perception have had a profound impact on auditory research. His series of seminal studies in the mid-1970s on the detection and discrimination of target sounds in sequences of brief tones with uncertain properties addresses the fundamental problem of extracting target signals from background sounds. As conceptualized by Chuck and others, informational masking results from more central (even ‘‘cogneetive’’) processes as a consequence of stimulus uncertainty, and can be distinguished from ‘‘energetic’’ masking, which primarily arises from the auditory periphery. Informational masking techniques are now in common use to study the detection, discrimination, and recognition of complex sounds, the capacity of auditory memory and aspects of auditory selective attention, the often large effects of training to reduce detrimental effects of uncertainty, and the perceptual segregation of target sounds from irrelevant context sounds. This paper will present an overview of past and current research on informational masking, and show how Chuck’s work has been expanded in several directions by other scientists to include the effects of informational masking on speech perception and on perception by listeners with hearing impairment. [Work supported by NIDCD.]

Representation of concurrent acoustic objects in primary auditory cortex.

Auditory scene analysis involves the simultaneous grouping and parsing of acoustic data into separate mental representations (i.e., objects). Over two experiments, we examined the sequence of neural processes underlying concurrent sound segregation by means of recording of human middle latency auditory evoked responses. Participants were presented with complex sounds comprising several harmonics, one of which could be mistuned such that it was not an integer multiple of the fundamental frequency. In both experiments, Na (approximately 22 ms) and Pa (approximately 32 ms) waves were reliably generated for all classes of stimuli. For stimuli with a fundamental frequency of 200 Hz, the mean Pa amplitude was significantly larger when the third harmonic was mistuned by 16% of its original value, relative to when it was tuned. The enhanced Pa amplitude was related to an increased likelihood in reporting the presence of concurrent auditory objects. Our results are consistent with a low-level stage of auditory scene analysis in which acoustic properties such as mistuning act as preattentive segregation cues that can subsequently lead to the perception of multiple auditory objects.

Discriminating harmonicity.

Simultaneous tones that are harmonically related tend to be grouped perceptually to form a unitary auditory image. A partial that is mistuned stands out from the other tones, and harmonic complexes with different fundamental frequencies can readily be perceived as separate auditory objects. These phenomena are evidence for the strong role of harmonicity in perceptual grouping and segregation of sounds. This study measured the discriminability of harmonicity directly. In a two interval, two alternative forced-choice (2I2AFC) paradigm, the listener chose which of two sounds, signal or foil, was composed of tones that more closely matched an exact harmonic relationship. In one experiment, the signal was varied from perfectly harmonic to highly inharmonic by adding frequency perturbation to each component. The foil always had 100% perturbation. Group mean performance decreased from greater than 90% correct for 0% signal perturbation to near chance for 80% signal perturbation. In the second experiment, adding a masker presented simultaneously with the signals and foils disrupted harmonicity. Both monaural and dichotic conditions were tested. Signal level was varied relative to masker level to obtain psychometric functions from which slopes and midpoints were estimated. Dichotic presentation of these audible stimuli improved performance by 3-10 dB, due primarily to a release from "informational masking" by the perceptual segregation of the signal from the masker.