Auditory Object Formation Research Articles

Auditory objects in working memory include task-irrelevant features

Object-based attention operates both in perception and visual working memory. While the efficient perception of auditory stimuli also requires the formation of auditory objects, little is known about their role in auditory working memory (AWM). To investigate whether attention to one object feature in AWM leads to the involuntary maintenance of another, task-irrelevant feature, we conducted four experiments. Stimuli were abstract sounds that differed on the dimensions frequency and location, only one of which was task-relevant in each experiment. The first two experiments required a match–nonmatch decision about a probe sound whose irrelevant feature value could either be identical to or differ from the memorized stimulus. Matches on the relevant dimension were detected more accurately when the irrelevant feature matched as well, whereas for nonmatches on the relevant dimension, performance was better for irrelevant feature nonmatches. Signal-detection analysis showed that changes of irrelevant frequency reduced the sensitivity for sound location. Two further experiments used continuous report tasks. When location was the target feature, changes of irrelevant sound frequency had an impact on both recall error and adjustment time. Irrelevant location changes affected adjustment time only. In summary, object-based attention led to a concurrent maintenance of task-irrelevant sound features in AWM.

Early Years: Creating Perceptual Space Where Music and Language Can Meet

ABSTRACT Auditory perception and auditory cognition development in early years are foundational for social and emotional development and language acquisition. The ability to recognize sound patterns around us and the delicate balance between their acoustic features determines our understanding of the intrinsic nature of language, socio-emotional interactions, and music. Both acoustic feature recognition and an ability to effectively form a perceptual unit – an auditory object – are fundamental for developing an ability to make sense of sound. Since both music and language rely on the auditory system to perceive and produce sound, the potential of using music in interventions for children with language acquisition difficulties like Autism Spectrum Disorders (ASD) has been raised by numerous scientists. Based on personal experience, the current paper discusses opportunities for supporting auditory perception development in music-based interactions using instrumental sound patterns. It details how manipulation of acoustic features like pitch, rhythm, timbre, duration, and intensity can help develop auditory skills like auditory pattern recognition, categorization of sounds by timbre, pitch, auditory object formation, and symbolization – the prerequisites for recognizing vocal identity, understanding spoken language, and articulating spoken messages.

Towards audiovisual scene analysis and object formation

The neural mechanisms underpinning auditory scene analysis and object formation have been of intense research interest in the past two decades. Fundamentally, however, we live in a multisensory environment. Even Cherry in his original paper posited that “lip reading” as a way for us to solve the cocktail party problem. Yet, how different aspects of visual cues (e.g., timing, linguistic information) help listeners follow conversation in a complex acoustic scene is still not well understood. In this talk, we present a theoretical framework to study audiovisual scene analysis that has been extrapolated from the unisensory object-based attention literature and posit the following questions: How do we define a multi-modal object? What are the predictions from unisensory object-based attention theory when we apply to the audiovisual domain? What are the conceptual models to test the different neural mechanisms that underpin audiovisual scene analysis? Answering these questions would move us closer to addressing the cocktail party problem in the real-world setting as well as help us create, de novo, audiovisual scenes that are more engaging in the augmented/virtual reality world.

Central Representation of Cervical Vestibular Evoked Myogenic Potentials.

Sensitivity of vestibular system to sounds (SVSS) can be measureable by cervical vestibular evoked myogenic potentials (cVEMPs). The aim of this study is to investigate central representation of vestibular system sensitivity to sound. The research was conducted in 2022-2023 by searching English language databases. The criterion for selecting documents was their overlap with the aim of this work. The animals studies were not included. The saccule is stimulated by sounds, that are transmitted through air and bone conduction. Utricle and semicircular canals are activated only by the vibrations. The afferent nerve fibers of the vestibular system project to the temporal, frontal, parietal, primary visual cortex, insula and the cingulate cortex. There is a relationship between normal results of the cVEMPs and these parameters. Improved phonemes recognition scores and word recognition scores in white noise, the efficiency of auditory training, incraed amplitude of the auditory brainstem responses to 500 HZ tone burst. Learning the first words is not only based on the hearing and other senses participate. The auditory object is a three-dimensional imaging in people's minds, when they hear a word. The words expressed by a speaker create different auditory objects in people's minds. Each of these auditory objects has its own color, shape, aroma and characteristics. For the formation of the auditory objects, all senses and whole areas of the brain contribute. Like other senses, central representation of vestibular system sensitivity to sound are also involved in the formation of auditory objects.

Selective auditory attention modulates cortical responses to sound location change for speech in quiet and in babble.

Listeners use the spatial location or change in spatial location of coherent acoustic cues to aid in auditory object formation. From stimulus-evoked onset responses in normal-hearing listeners using electroencephalography (EEG), we have previously shown measurable tuning to stimuli changing location in quiet, revealing a potential window into the cortical representations of auditory scene analysis. These earlier studies used non-fluctuating, spectrally narrow stimuli, so it was still unknown whether previous observations would translate to speech stimuli, and whether responses would be preserved for stimuli in the presence of background maskers. To examine the effects that selective auditory attention and interferers have on object formation, we measured cortical responses to speech changing location in the free field with and without background babble (+6 dB SNR) during both passive and active conditions. Active conditions required listeners to respond to the onset of the speech stream when it occurred at a new location, explicitly indicating 'yes' or 'no' to whether the stimulus occurred at a block-specific location either 30 degrees to the left or right of midline. In the aggregate, results show similar evoked responses to speech stimuli changing location in quiet compared to babble background. However, the effect of the two background environments diverges somewhat when considering the magnitude and direction of the location change and where the subject was attending. In quiet, attention to the right hemifield appeared to evoke a stronger response than attention to the left hemifield when speech shifted in the rightward direction. No such difference was found in babble conditions. Therefore, consistent with challenges associated with cocktail party listening, directed spatial attention could be compromised in the presence of stimulus noise and likely leads to poorer use of spatial cues in auditory streaming.

Attention effects on cortical neural processing in spatial listening

Speech understanding in complex acoustic scenes relies on the formation of and attention to auditory objects. Spatial information is an essential cue in the formation of auditory objects and may facilitate attention-based neural enhancement and suppression of target and non-target speech, respectively. Previous studies have used cortical auditory evoked potentials (CAEP) to measure how attention modulates neural representation of spatially separated auditory objects. The current study further extends this work to investigate how expected and unexpected shifts in spatial location of speech signals affect neural processing of these signals. In the experiment, listeners were directed where to listen with a spatial cue presented from one of five locations (±90, ±45, or 0 deg azimuth); the following target sequence was either presented in the cued location (congruent trials) or from a non-cued location (incongruent trials). We hypothesized that neural responses to target sequences would be enhanced when presented from the cued location, whereas neural responses to incongruent sequences would initially be suppressed before spatial attention was shifted toward the target. Further, we expected the size of this suppression to be proportional to the size of the relative shift between cue and incongruent target.

The role of reliable interaural time difference cues in ambiguous binaural signals for the intelligibility of multitalker speech.

When listening to speech in the presence of concurrent talkers, listeners can benefit from glimpses that occur as a result of spectro-temporal modulations in the speech signals. These glimpses are characterized by a high local signal-to-noise ratio and allow listeners to collect relatively undistorted and reliable information on target speech features. A series of experiments was designed to measure the spatial advantage for binaurally presented speech when useful interaural time difference (ITD) information was provided only in glimpses of speech signals with otherwise ambiguous ITDs. For interaurally coherent signals, ITD information provided by target glimpses contributed substantially to the spatial advantage, but consistent target ITDs overall appeared to be of minor importance to speech intelligibility. For interaurally incoherent signals, a similarly large contribution of coherent ITD information in glimpses to the spatial advantage was not observed. Rather, target speech intelligibility depended on the interaural coherence of the interfering speech signals. While the previous observation conforms with models of auditory object formation, and the latter is consistent with equalization-cancellation theory modeling the spatial advantage, the two seem to be at odds for the presented set of experiments. A conceptual framework employing different strategies to process the perceptual foreground and background may solve this issue.

Hierarchical Encoding of Attended Auditory Objects in Multi-talker Speech Perception

Humans can easily focus on one speaker in a multi-talker acoustic environment, but how different areas of the human auditory cortex (AC) represent the acoustic components of mixed speech is unknown. We obtained invasive recordings from the primary and nonprimary AC in neurosurgical patients as they listened to multi-talker speech. We found that neural sites in the primary AC responded to individual speakers in the mixture and were relatively unchanged by attention. In contrast, neural sites in the nonprimary AC were less discerning of individual speakers but selectively represented the attended speaker. Moreover, the encoding of the attended speaker in the nonprimary AC was invariant to the degree of acoustic overlap with the unattended speaker. Finally, this emergent representation of attended speech in the nonprimary AC was linearly predictable from the primary AC responses. Our results reveal the neural computations underlying the hierarchical formation of auditory objects in human AC during multi-talker speech perception.

Spectrotemporal window of binaural integration in auditory object formation

Binaural integration of interaural temporal information is essential for sound source localization and segregation. Current models of binaural interaction have shown that accurate sound localization in the horizontal plane depends on the resolution of phase ambiguous information by across-frequency integration. However, as such models are mostly static, it is not clear how proximate in time binaural events in different frequency channels should occur to form an auditory object with a unique lateral position. The present study examined the spectrotemporal window required for effective integration of binaural cues across frequency to form the perception of a stationary position. In Experiment 1, listeners judged whether dichotic frequency-modulated (FM) sweeps with a constant large nominal interaural delay (1500 μs), whose perceived laterality was ambiguous depending on the sweep rate (1500, 3000, 6000, and 12,000 Hz/s), produced a percept of continuous motion or a stationary image. Motion detection performance, indexed by d-prime (d') values, showed a clear effect of sweep rate, with auditory motion effects most pronounced for low sweep rates, and a punctate stationary image at high rates. Experiment 2 examined the effect of modulation rate (0.5, 3, 20, and 50 Hz) on lateralizing sinusoidally frequency-modulated (SFM) tones to confirm the effect of sweep rate on motion detection, independent of signal duration. Lateralization accuracy increased with increasing modulation rate up to 20 Hz and saturated at 50 Hz, with poorest performance occurring below 3 Hz depending on modulator phase. Using the transition point where percepts changed from motion to stationary images, we estimated a spectrotemporal integration window of approximately 150 ms per octave required for effective integration of interaural temporal cues across frequency channels. A Monte Carlo simulation based on a cross-correlation model of binaural interaction predicted 90% of the variance on perceptual motion detection performance as a function of FM sweep rate. Findings suggest that the rate of frequency channel convergence of binaural cues is essential to binaural lateralization.

Atypical brain responses to auditory spatial cues in adults with autism spectrum disorder.

The auditory processing atypicalities experienced by many individuals on the autism spectrum disorder might be understood in terms of difficulties parsing the sound energy arriving at the ears into discrete auditory 'objects'. Here, we asked whether autistic adults are able to make use of two important spatial cues to auditory object formation - the relative timing and amplitude of sound energy at the left and right ears. Using electroencephalography, we measured the brain responses of 15 autistic adults and 15 age- and verbal-IQ-matched control participants as they listened to dichotic pitch stimuli - white noise stimuli in which interaural timing or amplitude differences applied to a narrow frequency band of noise typically lead to the perception of a pitch sound that is spatially segregated from the noise. Responses were contrasted with those to stimuli in which timing and amplitude cues were removed. Consistent with our previous studies, autistic adults failed to show a significant object-related negativity (ORN) for timing-based pitch, although their ORN was not significantly smaller than that of the control group. Autistic participants did show an ORN to amplitude cues, indicating that they do not experience a general impairment in auditory object formation. However, their P400 response - thought to indicate the later attention-dependent aspects of auditory object formation - was missing. These findings provide further evidence of atypical auditory object processing in autism with potential implications for understanding the perceptual and communication difficulties associated with the condition.

Auditory object formation from background noise improves speech perception

Auditory object formation allows listeners to isolate a speech stream from a complex acoustic environment. While most studies focus on foregrounded speech, here we focus on whether listeners can form auditory objects from background noise to aid in speech perception. We use an open-set word recognition task with fluctuating speech-shaped-noise maskers, created by randomly interrupting the noise in each 125 ms window with a long (100 ms) or short (25 ms) silence. On each trial, three maskers were played, with the word occurring during the third. We previously showed that participants perform significantly better when the patterns repeat, compared to when the first two maskers differed from the third. Only the ordering of long or short within preceding patterns differs. To examine whether this advantage is due to auditory object formation, we leverage previous studies showing that inserted silences interfere with object formation. We insert 433 ms silences between patterns and observe defeat of the advantage we previously observed. We therefore attribute the advantage in the repeated masker condition to formation of an auditory object from background noise. Moving forward, we will manipulate other parameters affecting object formation and observe their affect on speech perception in noise.

The impact of asymmetric rates on interaural time difference lateralization and auditory object formation in bilateral cochlear implant and normal hearing listeners

Normal hearing (NH) listeners are able to accurately identify and locate sound sources using auditory object formation (AOF) and interaural time differences (ITDs). Temporal cues can further facilitate AOF and ITD sensitivity: this includes within- and across-ear stimulus’ rate, envelope-, and onset-symmetry. Bilateral cochlear implant (BiCI) listeners are not guaranteed to receive symmetrical or complementing temporal information across ears. Cochlear-Nucleus devices undergo “peak-picking” where stimulation of electrodes could yield asymmetric rates across the ears, interrupting good AOF and ITD sensitivity. We investigated the impact of asymmetric rates and ITDs, on AOF and ITD lateralization. BiCI and NH listeners were presented with diotic and dichotic pulsatile stimulus rates, combined with an ITD. Rate was fixed in one ear, and varied in the contralateral ear. In a single interval, six-alternative forced-choice task, listeners reported where and how many sounds they heard. We hypothesized that with ...

65. Neurophysiology of Auditory Object Formation Deficits in the First-Episode Schizophrenia Spectrum

65. Neurophysiology of Auditory Object Formation Deficits in the First-Episode Schizophrenia Spectrum

Timing predictability enhances regularity encoding in the human subcortical auditory pathway.

The encoding of temporal regularities is a critical property of the auditory system, as short-term neural representations of environmental statistics serve to auditory object formation and detection of potentially relevant novel stimuli. A putative neural mechanism underlying regularity encoding is repetition suppression, the reduction of neural activity to repeated stimulation. Although repetitive stimulation per se has shown to reduce auditory neural activity in animal cortical and subcortical levels and in the human cerebral cortex, other factors such as timing may influence the encoding of statistical regularities. This study was set out to investigate whether temporal predictability in the ongoing auditory input modulates repetition suppression in subcortical stages of the auditory processing hierarchy. Human auditory frequency–following responses (FFR) were recorded to a repeating consonant–vowel stimuli (/wa/) delivered in temporally predictable and unpredictable conditions. FFR amplitude was attenuated by repetition independently of temporal predictability, yet we observed an accentuated suppression when the incoming stimulation was temporally predictable. These findings support the view that regularity encoding spans across the auditory hierarchy and point to temporal predictability as a modulatory factor of regularity encoding in early stages of the auditory pathway.

Lateralization of interaural timing differences with multi-electrode stimulation in bilateral cochlear-implant users.

Bilateral cochlear implant (BiCI) users have shown variability in interaural time difference (ITD) sensitivity at different places along the cochlea. This paper investigates perception of multi-electrode binaural stimulation to determine if auditory object formation (AOF) and lateralization are affected by variability in ITD sensitivity when a complex sound is encoded with multi-channel processing. AOF and ITD lateralization were compared between single- and multi-electrode configurations. Most (7/8) BiCI users perceived a single auditory object with multi-electrode stimulation, and the range of lateralization was comparable to single-electrode stimulation, suggesting that variability in single-electrode ITD sensitivity does not compromise AOF with multi-electrode stimulation.

Interrupted speech with competing talkers: Benefits of temporal envelope and periodicity cues for younger and older adults

Recognition of interrupted speech requires connecting speech fragments over time and across gaps of missing information. Intelligibility improves when silent intervals are filled with noise; the effect is enhanced when noise contains rudimentary speech information, such as the temporal envelope of the missing speech. In multiple-talker environments, recognition of interrupted speech is more difficult, particularly for older adults. In these cases, temporal envelope cues may provide an important scaffold for auditory object formation. Other basic speech cues, such as F0-related periodicity, may help listeners segregate multiple voices and provide additional benefit. The relative, and potentially additive, benefit of temporal envelope and periodicity cues, and their use by older adults, remain unclear. To address these questions, younger and older adults with normal hearing listened to sentences in quiet and competing talker backgrounds. Sentences were periodically interrupted with (1) silence, (2) envelope-modulated noise, (3) steady-state pulse trains, which contained periodicity information from the missing speech, or (4) envelope-modulated pulse trains, which provided both envelope and periodicity information. Results are discussed in terms of contributions of temporal envelope and periodicity cues to perceptual organization for younger and older adults listening in complex environments. [Work supported by NIH/NIDCD and a AAA Student Investigator Research Grant.]

Attention is critical for spatial auditory object formation.

The precedence effect provides a novel way to examine the role of attention in auditory object formation. When presented with two identical sounds from different locations separated by a short stimulus onset asynchrony (SOA), listeners report a single auditory object at the location of the lead sound. When the SOA is above the echo threshold, listeners report hearing two auditory objects with different locations. Event-related potential (ERP) studies have shown that the number of perceived auditory objects is reflected in an object-related negativity (ORN) 100-250ms after onset and in a posterior late positivity (LP) 300-500ms after onset. In the present study, we tested whether these ERP effects are modulated by attention by presenting lead/lag click pairs at and around listeners' echo thresholds, while in separate blocks the listeners (1) attended to the sounds and reported whether the lag sound was a separate source, and (2) performed a two-back visual task. When attention was directed away from the sounds, neither the ORN nor the LP observed in the attend condition was evident. Instead, unattended click pairs above the echo threshold elicited an anterior positivity 250-450ms after onset. However, an effect resembling an ORN was found in comparing the ERPs elicited by unattended click pairs with SOAs below the attended echo threshold, indicating that the echo threshold may have been lowered when attention was directed away from the sounds. These results suggest that attention modulates early perceptual processes that are critical for auditory object formation.

Animal models of spontaneous activity in the healthy and impaired auditory system.

Spontaneous neural activity in the auditory nerve fibers and in auditory cortex in healthy animals is discussed with respect to the question: Is spontaneous activity noise or information carrier? The studies reviewed suggest strongly that spontaneous activity is a carrier of information. Subsequently, I review the numerous findings in the impaired auditory system, particularly with reference to noise trauma and tinnitus. Here the common assumption is that tinnitus reflects increased noise in the auditory system that among others affects temporal processing and interferes with the gap-startle reflex, which is frequently used as a behavioral assay for tinnitus. It is, however, more likely that the increased spontaneous activity in tinnitus, firing rate as well as neural synchrony, carries information that shapes the activity of downstream structures, including non-auditory ones, and leading to the tinnitus percept. The main drivers of that process are bursting and synchronous firing, which facilitates transfer of activity across synapses, and allows formation of auditory objects, such as tinnitus.

Using auditory pre-information to solve the cocktail-party problem: electrophysiological evidence for age-specific differences.

Speech understanding in complex and dynamic listening environments requires (a) auditory scene analysis, namely auditory object formation and segregation, and (b) allocation of the attentional focus to the talker of interest. There is evidence that pre-information is actively used to facilitate these two aspects of the so-called “cocktail-party” problem. Here, a simulated multi-talker scenario was combined with electroencephalography to study scene analysis and allocation of attention in young and middle-aged adults. Sequences of short words (combinations of brief company names and stock-price values) from four talkers at different locations were simultaneously presented, and the detection of target names and the discrimination between critical target values were assessed. Immediately prior to speech sequences, auditory pre-information was provided via cues that either prepared auditory scene analysis or attentional focusing, or non-specific pre-information was given. While performance was generally better in younger than older participants, both age groups benefited from auditory pre-information. The analysis of the cue-related event-related potentials revealed age-specific differences in the use of pre-cues: Younger adults showed a pronounced N2 component, suggesting early inhibition of concurrent speech stimuli; older adults exhibited a stronger late P3 component, suggesting increased resource allocation to process the pre-information. In sum, the results argue for an age-specific utilization of auditory pre-information to improve listening in complex dynamic auditory environments.

Differential activation of human core, non-core and auditory-related cortex during speech categorization tasks as revealed by intracranial recordings.

Speech perception requires that sounds be transformed into speech-related objects with lexical and semantic meaning. It is unclear at what level in the auditory pathways this transformation emerges. Primary auditory cortex has been implicated in both representation of acoustic sound attributes and sound objects. While non-primary auditory cortex located on the posterolateral superior temporal gyrus (PLST) is clearly involved in acoustic-to-phonetic pre-lexical representations, it is unclear what role this region plays in auditory object formation. Additional data support the importance of prefrontal cortex in the formation of auditory objects, while other data would implicate this region in auditory object selection. To help clarify the respective roles of auditory and auditory-related cortex in the formation and selection of auditory objects, we examined high gamma activity simultaneously recorded directly from Heschl's gyrus (HG), PLST and prefrontal cortex, while subjects performed auditory semantic detection tasks. Subjects were patients undergoing evaluation for treatment of medically intractable epilepsy. We found that activity in posteromedial HG and early activity on PLST was robust to sound stimuli regardless of their context, and minimally modulated by tasks. Later activity on PLST could be strongly modulated by semantic context, but not by behavioral performance. Activity within prefrontal cortex also was related to semantic context, and did co-vary with behavior. We propose that activity in posteromedial HG and early activity on PLST primarily reflect the representation of spectrotemporal sound attributes. Later activity on PLST represents a pre-lexical processing stage and is an intermediate step in the formation of word objects. Activity in prefrontal cortex appears directly involved in word object selection. The roles of other auditory and auditory-related cortical areas in the formation of word objects remain to be explored.