Sound Segregation Research Articles

Auditory Stream Segregation Can Be Modeled by Neural Competition in Cochlear Implant Listeners.

Auditory stream segregation is a perceptual process by which the human auditory system groups sounds from different sources into perceptually meaningful elements (e.g., a voice or a melody). The perceptual segregation of sounds is important, for example, for the understanding of speech in noisy scenarios, a particularly challenging task for listeners with a cochlear implant (CI). It has been suggested that some aspects of stream segregation may be explained by relatively basic neural mechanisms at a cortical level. During the past decades, a variety of models have been proposed to account for the data from stream segregation experiments in normal-hearing (NH) listeners. However, little attention has been given to corresponding findings in CI listeners. The present study investigated whether a neural model of sequential stream segregation, proposed to describe the behavioral effects observed in NH listeners, can account for behavioral data from CI listeners. The model operates on the stimulus features at the cortical level and includes a competition stage between the neuronal units encoding the different percepts. The competition arises from a combination of mutual inhibition, adaptation, and additive noise. The model was found to capture the main trends in the behavioral data from CI listeners, such as the larger probability of a segregated percept with increasing the feature difference between the sounds as well as the build-up effect. Importantly, this was achieved without any modification to the model's competition stage, suggesting that stream segregation could be mediated by a similar mechanism in both groups of listeners.

Auditory Streaming and Prediction in Tinnitus Sufferers

The aim of this study was to determine whether auditory streaming (segregation of incoming sounds into separate sound sources) and the ability to anticipate future auditory events are affected in tinnitus sufferers compared with nontinnitus controls with matched levels of hearing. It was hypothesized that tinnitus would result in abnormal electroencephalography responses to tone deviants and tone omissions compared to controls for frequencies near the pitch of tinnitus, and this should correspond with increased levels of cortical γ and θ oscillatory rhythms. Sixteen individuals with tinnitus (10 men and 6 women; age, 53.44; SD, 12.92 years) and 14 control participants (8 men and 6 women; age, 50.25; SD, 18.54 years) took part in the study. A modified version of the ABA streaming paradigm, with repeating triplet pattern of two frequencies (A and B) presented as A-B-A, was used to examine deviant-related prediction error. Omission-related prediction errors were examined using a modified version of a tone-omission paradigm. Regions of interest were frontocentral, left frontal, right frontal, and temporal lobes. A larger N1c waveform was elicited in the absence of any tone deviation within the left primary auditory cortex of tinnitus participants. No differences were present between groups for omissions. The only difference in oscillatory band activity between the two groups in this study was in response to tones 7 semitones different from tinnitus pitch, with significantly lower β-2 band activity present for the tinnitus group, correlating most with activity within the right inferior occipital gyrus. The findings from this study imply that cortical-level auditory stream segregation is altered among individuals with tinnitus.

Perceptual fusion of musical notes suggests universal representations of dissonance despite culture-dependent aesthetic associations

Music varies enormously across cultures, but some traits are widespread. Cross-cultural consistency in music could be driven by universal perceptual mechanisms adapted to natural sounds, but supporting evidence has been circumstantial due to the dearth of cross-cultural research. Here, we explore whether such perceptual mechanisms impose universal similarity relations on musical structure, potentially dissociating from culture-specific aesthetic judgments about music. We measured one possible signature of these similarity relations—the extent to which concurrent notes are perceived as a single sound—in members of a small-scale Amazonian society and Western listeners. We also measured aesthetic responses to the same stimuli. Unlike Westerners, Amazonian listeners were aesthetically indifferent to whether note combinations were canonically consonant (with aggregate frequency spectra resembling the harmonic series). However, Amazonians were nonetheless more likely to hear consonant combinations as a single sound, with fusion judgments that qualitatively resembled those of Western listeners. Thus, even in a culture with little exposure to Western harmony, reliance on harmonic frequency relations for sound segregation evidently induces consistent perceptual structure in note combinations. The results suggest that perceptual mechanisms for representing music can be shared across cultures, even though the perceptual equivalences that result give rise to culture-specific aesthetic associations.

Age-related differences in modulation detection interference and interference release

Age-related declines have been observed in auditory tasks related to temporal processing, but less work has been conducted in middle-aged listeners and in auditory sound segregation tasks. This experiment addresses these issues by evaluating modulation detection interference (MDI) and release from MDI using streaming in young, middle-aged and older listeners. Using a standard MDI paradigm, we measured amplitude modulation detection thresholds of a high-frequency tone in different conditions: in quiet, in the presence of a low-impact unmodulated interferer, and in the presence of a high-impact modulated interferer. Then, we measured streaming-based release from MDI using four modulated precursors designed to perceptually capture the interferer and therefore reduce the amount of interference. Preliminary data suggest that middle-aged normal-hearing listeners experience similar MDI to young listeners and also receive a large release from interference when precursors are present. Additional data will be presented from older listeners, who experience slightly greater MDI than younger listeners.

Free-field evoked auditory brainstem responses in cochlear implant users.

The importance of binaural cues in auditory stream formation and sound source segregation is widely accepted. When treating one ear with a cochlear implant (CI) the peripheral auditory system gets partially replaced and processing delays get added potentially, thus important interaural time differences get altered. However, these effects are not fully understood, leaving a lack of systematic binaural fitting strategies with respect to an optimal binaural fusion. To get new insights into such alterations, we suggest a novel method of free-field auditory brainstem evoked responses (ABRs) analysis in CI users. This method does not bypass the technically induced intrinsic delays of the sound processor while leaving the whole electrode array active, thus the most natural way of stimulation is provided. We compared the ABRs collected of 12 CI users and 12 normal hearing listeners using two different stimuli (chirp, click) at four different intensities each. We analyzed the ABRs using the average of 2000 trials as well as a single trial analysis and found consistent results in the ABRs’ amplitudes and latencies, as well as in single trial relationships between both groups. This method provides a new perspective into the natural CI users’ ABRs and can be useful in future research regarding binaural interaction and fusion.

Effect of harmonic rank on sequential sound segregation

The ability to segregate sounds from different sound sources is thought to depend on the perceptual salience of differences between the sounds, such as differences in frequency or fundamental frequency (F0). F0 discrimination of complex tones is better for tones with low harmonics than for tones that only contain high harmonics, suggesting greater pitch salience for the former. This leads to the expectation that the sequential stream segregation (streaming) of complex tones should be better for tones with low harmonics than for tones with only high harmonics. However, the results of previous studies are conflicting about whether this is the case. The goals of this study were to determine the effect of harmonic rank on streaming and to establish whether streaming is related to F0 discrimination. Thirteen young normal-hearing participants were tested. Streaming was assessed for pure tones and complex tones containing harmonics with various ranks using sequences of ABA triplets, where A and B differed in frequency or in F0. The participants were asked to try to hear two streams and to indicate when they heard one and when they heard two streams. F0 discrimination was measured for the same tones that were used as A tones in the streaming experiment. Both streaming and F0 discrimination worsened significantly with increasing harmonic rank. There was a significant relationship between streaming and F0 discrimination, indicating that good F0 discrimination is associated with good streaming. This supports the idea that the extent of stream segregation depends on the salience of the perceptual difference between successive sounds.

Inharmonic speech reveals the role of harmonicity in the cocktail party problem

The “cocktail party problem” requires us to discern individual sound sources from mixtures of sources. The brain must use knowledge of natural sound regularities for this purpose. One much-discussed regularity is the tendency for frequencies to be harmonically related (integer multiples of a fundamental frequency). To test the role of harmonicity in real-world sound segregation, we developed speech analysis/synthesis tools to perturb the carrier frequencies of speech, disrupting harmonic frequency relations while maintaining the spectrotemporal envelope that determines phonemic content. We find that violations of harmonicity cause individual frequencies of speech to segregate from each other, impair the intelligibility of concurrent utterances despite leaving intelligibility of single utterances intact, and cause listeners to lose track of target talkers. However, additional segregation deficits result from replacing harmonic frequencies with noise (simulating whispering), suggesting additional grouping cues enabled by voiced speech excitation. Our results demonstrate acoustic grouping cues in real-world sound segregation.

A FPGA Implementation of the CAR-FAC Cochlear Model

This paper presents a digital implementation of the Cascade of Asymmetric Resonators with Fast-Acting Compression (CAR-FAC) cochlear model. The CAR part simulates the basilar membrane's (BM) response to sound. The FAC part models the outer hair cell (OHC), the inner hair cell (IHC), and the medial olivocochlear efferent system functions. The FAC feeds back to the CAR by moving the poles and zeros of the CAR resonators automatically. We have implemented a 70-section, 44.1 kHz sampling rate CAR-FAC system on an Altera Cyclone V Field Programmable Gate Array (FPGA) with 18% ALM utilization by using time-multiplexing and pipeline parallelizing techniques and present measurement results here. The fully digital reconfigurable CAR-FAC system is stable, scalable, easy to use, and provides an excellent input stage to more complex machine hearing tasks such as sound localization, sound segregation, speech recognition, and so on.

Effect of age and hearing loss on auditory stream segregation of speech sounds

Segregating and understanding speech in complex environments is a major challenge for hearing-impaired (HI) listeners. It remains unclear to what extent these difficulties are dominated by direct interference, such as simultaneous masking, or by a failure of the mechanisms of stream segregation. This study compared older HI listeners' performance with that of young and older normal-hearing (NH) listeners in stream segregation tasks involving speech sounds. Listeners were presented with sequences of speech tokens, each consisting of a fricative consonant and a voiced vowel (CV). The CV tokens were concatenated into interleaved sequences that alternated in fundamental frequency (F0) and/or simulated vocal tract length (VTL). Each pair of interleaved sequences was preceded by a “word” consisting of two random tokens. The listeners were asked to indicate whether the word was present in the following interleaved sequences. The word, if present, occurred within one of the interleaved sequences, so that performance improved if the listeners were able to perceptually segregate the two sequences. Although HI listeners' identification of the speech tokens in isolation was poorer than that of the NH listeners, HI listeners were generally able to use both F0 and VTL cues to segregate the interleaved sequences. The results suggest that the difficulties experienced by HI listeners in complex acoustic environments cannot be explained by a loss of basic stream segregation abilities.

Sequential stream segregation based on spatial cues: Behavioral and neural measurements

Differences in simulated spatial cues in the horizontal plane have been shown to enhance both voluntary and obligatory stream segregation of sounds with realistic spectro-temporal variations, such as sequences of syllables. In this experiment, listeners were presented with sequences of speech tokens, each consisting of a fricative consonant and a voiced vowel. The CV tokens were concatenated into interleaved sequences that alternated in simulated spatial positions. The interleaved sequences lasted 1 min. The listeners had to press a button each time they heard a repeated token. In the selective attention task, the listeners were asked to attend only one of the two interleaved sequences; in the global attention task, the listeners had to perceive the interleaved sequences as single stream to detect a repetition between the sequences. Simultaneous EEG measurements were made. The behavioral results confirmed that listeners were able to either attend selectively or globally, depending on the task requirements. The EEG waveforms differed between the two tasks, despite identical physical stimuli, reflecting the difference between global and selective attention. Both behavioral and EEG results reflected the effects of increasing spatial separation in enhancing selective attention and making global attention to the sequences more difficult.

The Role of Place Cues in Voluntary Stream Segregation for Cochlear Implant Users.

Sequential stream segregation by cochlear implant (CI) listeners was investigated using a temporal delay detection task composed of a sequence of regularly presented bursts of pulses on a single electrode (B) interleaved with an irregular sequence (A) presented on a different electrode. In half of the trials, a delay was added to the last burst of the regular B sequence, and the listeners were asked to detect this delay. As a jitter was added to the period between consecutive A bursts, time judgments between the A and B sequences provided an unreliable cue to perform the task. Thus, the segregation of the A and B sequences should improve performance. In Experiment 1, the electrode separation and the sequence duration were varied to clarify whether place cues help CI listeners to voluntarily segregate sounds and whether a two-stream percept needs time to build up. Results suggested that place cues can facilitate the segregation of sequential sounds if enough time is provided to build up a two-stream percept. In Experiment 2, the duration of the sequence was fixed, and only the electrode separation was varied to estimate the fission boundary. Most listeners were able to segregate the sounds for separations of three or more electrodes, and some listeners could segregate sounds coming from adjacent electrodes.

Attentional Resources Are Needed for Auditory Stream Segregation in Aging.

The ability to select sound streams from background noise becomes challenging with age, even with normal peripheral auditory functioning. Reduced stream segregation ability has been reported in older compared to younger adults. However, the reason why there is a difference is still unknown. The current study investigated the hypothesis that automatic sound processing is impaired with aging, which then contributes to difficulty actively selecting subsets of sounds in noisy environments. We presented a simple intensity oddball sequence in various conditions with irrelevant background sounds while recording EEG. The ability to detect the oddball tones was dependent on the ability to automatically or actively segregate the sounds to frequency streams. Listeners were able to actively segregate sounds to perform the loudness detection task, but there was no indication of automatic segregation of background sounds while watching a movie. Thus, our results indicate impaired automatic processes in aging that may explain more effortful listening, and that tax attentional systems when selecting sound streams in noisy environments.

Neural representations of concurrent sounds with overlapping spectra in rat inferior colliculus: Comparisons between temporal-fine structure and envelope

Perceptual segregation of multiple sounds, which overlap in both time and spectra, into individual auditory streams is critical for hearing in natural environments. Some cues such as interaural time disparities (ITDs) play an important role in the segregation, especially when sounds are separated in space. In this study, we investigated the neural representation of two uncorrelated narrowband noises that shared the identical spectrum in the rat inferior colliculus (IC) using frequency-following-response (FFR) recordings, when the ITD for each noise stimulus was manipulated. The results of this study showed that recorded FFRs exhibited two distinctive components: the fast-varying temporal fine structure (TFS) component (FFRTFS) and the slow-varying envelope component (FFRENV). When a single narrowband noise was presented alone, the FFRTFS, but not the FFRENV, was sensitive to ITDs. When two narrowband noises were presented simultaneously, the FFRTFS took advantage of the ITD disparity that was associated with perceived spatial separation between the two concurrent sounds, and displayed a better linear synchronization to the sound with an ipsilateral-leading ITD. However, no effects of ITDs were found on the FFRENV. These results suggest that the FFRTFS and FFRENV represent two distinct types of signal processing in the auditory brainstem and contribute differentially to sound segregation based on spatial cues: the FFRTFS is more critical to spatial release from masking.

Situational awareness assessment of hearing protection

Uncompromised situational awareness has become a critical component of hearing protection devices (HPDs). Situational awareness is a complex psychological phenomenon, which consists of several perceptual and cognitive factors. This study presents data from three experiments designed to analyze the performance of HPDs on measures of situational awareness and determine how multiple factors may impact in-the-field performance. Data are presented for four hearing protection devices on measures of sound localization, distance perception, spatial segregation, speech intelligibility, and dynamic scene analysis. Baseline (open-ear) data is also presented to contrast with the HPDs. Small differences in fine localization-discrimination and the number of large quadrant errors were observed, but these differences between devices were not observed in the more complex scene analysis task. A performance index was developed to determine overall performance of each HPD compared to open-ear performance. This index combined...

Neural correlates of attention and streaming in a perceptually multistable auditory illusion

In a complex acoustic environment, acoustic cues and attention interact in the formation of streams within the auditory scene. In this study, a variant of the "octave illusion" [Deutsch (1974). Nature 251, 307-309] was used to investigate the neural correlates of auditory streaming, and to elucidate the effects of attention on the interaction between sequential and concurrent sound segregation in humans. By directing subjects' attention to different frequencies and ears, it was possible to elicit several different illusory percepts with the identical stimulus. The first experiment tested the hypothesis that the illusion depends on the ability of listeners to perceptually stream the target tones from within the alternating sound sequences. In the second experiment, concurrent psychophysical measures and electroencephalography recordings provided neural correlates of the various percepts elicited by the multistable stimulus. The results show that the perception and neural correlates of the auditory illusion can be manipulated robustly by attentional focus and that the illusion is constrained in much the same way as auditory stream segregation, suggesting common underlying mechanisms.

Promoting the perception of two and three concurrent sound objects: An event-related potential study

The auditory environment typically comprises several simultaneously active sound sources. In contrast to the perceptual segregation of two concurrent sounds, the perception of three simultaneous sound objects has not yet been studied systematically. We conducted two experiments in which participants were presented with complex sounds containing sound segregation cues (mistuning, onset asynchrony, differences in frequency or amplitude modulation or in sound location), which were set up to promote the perceptual organization of the tonal elements into one, two, or three concurrent sounds. In Experiment 1, listeners indicated whether they heard one, two, or three concurrent sounds. In Experiment 2, participants watched a silent subtitled movie while EEG was recorded to extract the object-related negativity (ORN) component of the event-related potential. Listeners predominantly reported hearing two sounds when the segregation promoting manipulations were applied to the same tonal element. When two different tonal elements received manipulations promoting them to be heard as separate auditory objects, participants reported hearing two and three concurrent sounds objects with equal probability. The ORN was elicited in most conditions; sounds that included the amplitude- or the frequency-modulation cue generated the smallest ORN amplitudes. Manipulating two different tonal elements yielded numerically and often significantly smaller ORNs than the sum of the ORNs elicited when the same cues were applied on a single tonal element. These results suggest that ORN reflects the presence of multiple concurrent sounds, but not their number. The ORN results are compatible with the horse-race principle of combining different cues of concurrent sound segregation.

Theta oscillations accompanying concurrent auditory stream segregation

The ability to isolate a single sound source among concurrent sources is crucial for veridical auditory perception. The present study investigated the event-related oscillations evoked by complex tones, which could be perceived as a single sound and tonal complexes with cues promoting the perception of two concurrent sounds by inharmonicity, onset asynchrony, and/or perceived source location difference of the components tones. In separate task conditions, participants performed a visual change detection task (visual control), watched a silent movie (passive listening) or reported for each tone whether they perceived one or two concurrent sounds (active listening). In two time windows, the amplitude of theta oscillation was modulated by the presence vs. absence of the cues: 60–350ms/6–8Hz (early) and 350–450ms/4–8Hz (late). The early response appeared both in the passive and the active listening conditions; it did not closely match the task performance; and it had a fronto-central scalp distribution. The late response was only elicited in the active listening condition; it closely matched the task performance; and it had a centro-parietal scalp distribution. The neural processes reflected by these responses are probably involved in the processing of concurrent sound segregation cues, in sound categorization, and response preparation and monitoring. The current results are compatible with the notion that theta oscillations mediate some of the processes involved in concurrent sound segregation.

Arming Organizations to Detect and Respond to Stealthy APTS

In today’s digital dependent world, organizations struggle to mitigate a stealthy, well-resourced, and tenacious advanced persistent threat (APT) attacks by nefarious actors, organizations, and even nation-states with intent on gaining a foothold into an organization’s IT infrastructure. This onslaught of advanced attacks requires far more than baseline security practices. While most security professionals are APT-aware, many lack the experience, requisite skills, and the ability to integrate technology to counter APT attacks. The problem is exacerbated by a widening cybersecurity skills gap. Recent research by ISACA, the world’s largest information security professional association, reported more than 60% of applicants for entry level cybersecurity positions lack the skill and ability to perform the tasks associated with their potential new roles. Success against the APT is predicated on insight into APT attack stages and the integration of technology to enable organizational resilience; however, this is not possible in organizations do not have the workforce with the requisite knowledge, skills, and abilities to perform the technical tasks related to their functional roles. This article addresses a customized response strategy executed by a skilled workforce that mitigates and even counters attacks. The strategy recommends that a coordinated response based on organization risk management policies be implemented. In addition, it requires organizational insight into their information assets, control of administrator privileges, implementation of sound network segregation architecture, and a commitment to a balanced vulnerability management program. It is critical that a further discussion occur to outline skills acquisition based on skills-based training and performance-based assessments.

The role of harmonic spectral structure in the cocktail party problem

Harmonicity is believed to provide an important acoustic grouping cue underlying sound segregation, though the mechanisms by which this occur, and its importance in real-world conditions, remain unclear. To test the role of harmonicity in the segregation of speech, we used a modified version of the STRAIGHT methodology for speech resynthesis to manipulate the fine-grained spectral structure of otherwise natural-sounding speech tokens. We then measured the ability of human listeners on two tasks: speech comprehension—of individual speech tokens in isolation, of speech in noise, and of speech presented concurrently with other tokens—and detection of a mistuned harmonic. We tested the importance of harmonicity in both tasks by jittering harmonic frequencies up or down by a small amount, rendering the speech inharmonic. We tested the importance of familiar spectral structure by deleting even-numbered harmonics. This latter manipulation left only the odd harmonics, a spectral pattern that, while harmonic, does...

Sound segregation via embedded repetition is robust to inattention.

The segregation of sound sources from the mixture of sounds that enters the ear is a core capacity of human hearing, but the extent to which this process is dependent on attention remains unclear. This study investigated the effect of attention on the ability to segregate sounds via repetition. We utilized a dual task design in which stimuli to be segregated were presented along with stimuli for a “decoy” task that required continuous monitoring. The task to assess segregation presented a target sound 10 times in a row, each time concurrent with a different distractor sound. McDermott, Wrobleski, and Oxenham (2011) demonstrated that repetition causes the target sound to be segregated from the distractors. Segregation was queried by asking listeners whether a subsequent probe sound was identical to the target. A control task presented similar stimuli but probed discrimination without engaging segregation processes. We present results from 3 different decoy tasks: a visual multiple object tracking task, a rapid serial visual presentation (RSVP) digit encoding task, and a demanding auditory monitoring task. Load was manipulated by using high- and low-demand versions of each decoy task. The data provide converging evidence of a small effect of attention that is nonspecific, in that it affected the segregation and control tasks to a similar extent. In all cases, segregation performance remained high despite the presence of a concurrent, objectively demanding decoy task. The results suggest that repetition-based segregation is robust to inattention.