Auditory Object Perception Research Articles

Listening to trees in the forest: Attentional set influences how semantic and acoustic factors interact in auditory perception.

Studies of auditory object perception claim that semantic properties dominate acoustic properties in determining identification accuracy. Yet the direction of the semantic effect is mixed, with some studies showing an advantage for detecting incongruent sounds and others reporting a congruent sound advantage. Here we examine the role of the participant's attentional set when identifying auditory objects in naturalistic soundscapes. We varied the acoustic and semantic properties of the sounds orthogonally in two experiments. In Experiment 1 participants tuned their attention broadly to detect any change between two successive soundscapes (e.g., two restaurant soundscapes, with and without a child coughing). In Experiment 2 they tuned attention more narrowly to a probe presented after a soundscape (e.g., a restaurant soundscape with a child coughing, followed by the coughing sound alone). In both experiments, semantic relations between the objects and backgrounds helped to disambiguate objects that blended acoustically with the background. When attending globally (Experiment 1), objects that were acoustically similar yet semantically incongruent tended to be missed (e.g., bouncing basketball on a construction site), as though camouflaged by the gist of the soundscape. When attending locally (Experiment 2), semantically congruent foil objects led to false positive reports under acoustically similar conditions (hammering sounds on a construction site), as though the gist of the soundscape contributed to their plausible inclusion. In summary, although attentional set had a strong influence on the specific kinds of errors made, both results pointed to participants using a semantically congruent high-level schema to report the sounds they heard.

Neuroimaging evidence for the direct role of auditory scene analysis in object perception.

Auditory Scene Analysis (ASA) refers to the grouping of acoustic signals into auditory objects. Previously, we have shown that perceived musicality of auditory sequences varies with high-level organizational features. Here, we explore the neural mechanisms mediating ASA and auditory object perception. Participants performed musicality judgments on randomly generated pure-tone sequences and manipulated versions of each sequence containing low-level changes (amplitude; timbre). Low-level manipulations affected auditory object perception as evidenced by changes in musicality ratings. fMRI was used to measure neural activation to sequences rated most and least musical, and the altered versions of each sequence. Next, we generated two partially overlapping networks: (i) a music processing network (music localizer) and (ii) an ASA network (base sequences vs. ASA manipulated sequences). Using Representational Similarity Analysis, we correlated the functional profiles of each ROI to a model generated from behavioral musicality ratings as well as models corresponding to low-level feature processing and music perception. Within overlapping regions, areas near primary auditory cortex correlated with low-level ASA models, whereas right IPS was correlated with musicality ratings. Shared neural mechanisms that correlate with behavior and underlie both ASA and music perception suggests that low-level features of auditory stimuli play a role in auditory object perception.

Manipulation of low-level features modulates grouping strength of auditory objects.

A central challenge of auditory processing involves the segregation, analysis, and integration of acoustic information into auditory perceptual objects for processing by higher order cognitive operations. This study explores the influence of low-level features on auditory object perception. Participants provided perceived musicality ratings in response to randomly generated pure tone sequences. Previous work has shown that music perception relies on the integration of discrete sounds into a holistic structure. Hence, high (versus low) ratings were viewed as indicative of strong (versus weak) object formation. Additionally, participants rated sequences in which random subsets of tones were manipulated along one of three low-level dimensions (timbre, amplitude, or fade-in) at one of three strengths (low, medium, or high). Our primary findings demonstrate how low-level acoustic features modulate the perception of auditory objects, as measured by changes in musicality ratings for manipulated sequences. Secondarily, we used principal component analysis to categorize participants into subgroups based on differential sensitivities to low-level auditory dimensions, thereby highlighting the importance of individual differences in auditory perception. Finally, we report asymmetries regarding the effects of low-level dimensions; specifically, the perceptual significance of timbre. Together, these data contribute to our understanding of how low-level auditory features modulate auditory object perception.

Acoustic Correlates of Auditory Object and Event Perception: Speakers, Musical Timbres, and Environmental Sounds.

Human listeners must identify and orient themselves to auditory objects and events in their environment. What acoustic features support a listener’s ability to differentiate the great variety of natural sounds they might encounter? Studies of auditory object perception typically examine identification (and confusion) responses or dissimilarity ratings between pairs of objects and events. However, the majority of this prior work has been conducted within single categories of sound. This separation has precluded a broader understanding of the general acoustic attributes that govern auditory object and event perception within and across different behaviorally relevant sound classes. The present experiments take a broader approach by examining multiple categories of sound relative to one another. This approach bridges critical gaps in the literature and allows us to identify (and assess the relative importance of) features that are useful for distinguishing sounds within, between and across behaviorally relevant sound categories. To do this, we conducted behavioral sound identification (Experiment 1) and dissimilarity rating (Experiment 2) studies using a broad set of stimuli that leveraged the acoustic variability within and between different sound categories via a diverse set of 36 sound tokens (12 utterances from different speakers, 12 instrument timbres, and 12 everyday objects from a typical human environment). Multidimensional scaling solutions as well as analyses of item-pair-level responses as a function of different acoustic qualities were used to understand what acoustic features informed participants’ responses. In addition to the spectral and temporal envelope qualities noted in previous work, listeners’ dissimilarity ratings were associated with spectrotemporal variability and aperiodicity. Subsets of these features (along with fundamental frequency variability) were also useful for making specific within or between sound category judgments. Dissimilarity ratings largely paralleled sound identification performance, however the results of these tasks did not completely mirror one another. In addition, musical training was related to improved sound identification performance.

Acoustic correlates of auditory object perception: Speakers, musical timbres, and environmental sounds

Human listeners must identify and orient themselves to auditory objects in their environment. What acoustic features support a listener’s ability to differentiate the variety of sound sources they might encounter? Typical studies of auditory object perception obtain dissimilarity ratings between pairs of objects, often within a single category of sound. However, such an approach precludes an understanding of general acoustic features that might be used to differentiate sounds across categories. The present experiment takes a broader approach to the analysis of dissimilarity ratings by leveraging the acoustic variability within and between different sound categories as characterized by a large, diverse set of 36 sound tokens (12 speech utterances from different speakers, 12 instrument timbres, and 12 everyday objects from a typical human environment). We analyze multidimensional scaling results as well as models of trial- level dissimilarity ratings as a function of different acoustic representations including spectral, temporal and noise features as well as modulation power spectra and cochlear spectrograms. In addition to previously noted differences in spectral and temporal envelopes, results indicate that listener’s dissimilarity ratings are also related to spectral variability and noise, particularly in differentiating sounds between categories. Dissimilarity ratings also appear to closely parallel sound identification performance.Human listeners must identify and orient themselves to auditory objects in their environment. What acoustic features support a listener’s ability to differentiate the variety of sound sources they might encounter? Typical studies of auditory object perception obtain dissimilarity ratings between pairs of objects, often within a single category of sound. However, such an approach precludes an understanding of general acoustic features that might be used to differentiate sounds across categories. The present experiment takes a broader approach to the analysis of dissimilarity ratings by leveraging the acoustic variability within and between different sound categories as characterized by a large, diverse set of 36 sound tokens (12 speech utterances from different speakers, 12 instrument timbres, and 12 everyday objects from a typical human environment). We analyze multidimensional scaling results as well as models of trial- level dissimilarity ratings as a function of different acoustic representations inclu...

Auditory object perception: A neurobiological model and prospective review

Interaction with the world is a multisensory experience, but most of what is known about the neural correlates of perception comes from studying vision. Auditory inputs enter cortex with its own set of unique qualities, and leads to use in oral communication, speech, music, and the understanding of emotional and intentional states of others, all of which are central to the human experience. To better understand how the auditory system develops, recovers after injury, and how it may have transitioned in its functions over the course of hominin evolution, advances are needed in models of how the human brain is organized to process real-world natural sounds and “auditory objects”. This review presents a simple fundamental neurobiological model of hearing perception at a category level that incorporates principles of bottom-up signal processing together with top-down constraints of grounded cognition theories of knowledge representation. Though mostly derived from human neuroimaging literature, this theoretical framework highlights rudimentary principles of real-world sound processing that may apply to most if not all mammalian species with hearing and acoustic communication abilities. The model encompasses three basic categories of sound-source: (1) action sounds (non-vocalizations) produced by ‘living things’, with human (conspecific) and non-human animal sources representing two subcategories; (2) action sounds produced by ‘non-living things’, including environmental sources and human-made machinery; and (3) vocalizations (‘living things’), with human versus non-human animals as two subcategories therein. The model is presented in the context of cognitive architectures relating to multisensory, sensory-motor, and spoken language organizations. The models’ predictive values are further discussed in the context of anthropological theories of oral communication evolution and the neurodevelopment of spoken language proto-networks in infants/toddlers. These phylogenetic and ontogenetic frameworks both entail cortical network maturations that are proposed to at least in part be organized around a number of universal acoustic-semantic signal attributes of natural sounds, which are addressed herein.

Multisensory object perception in infancy: 4-month-olds perceive a mistuned harmonic as a separate auditory and visual object

Infants learn to use auditory and visual information to organize the sensory world into identifiable objects with particular locations. Here we use a behavioural method to examine infants’ use of harmonicity cues to auditory object perception in a multisensory context. Sounds emitted by different objects sum in the air and the auditory system must figure out which parts of the complex waveform belong to different sources (auditory objects). One important cue to this source separation is that complex tones with pitch typically contain a fundamental frequency and harmonics at integer multiples of the fundamental. Consequently, adults hear a mistuned harmonic in a complex sound as a distinct auditory object (Alain, Theunissen, Chevalier, Batty, & Taylor, 2003). Previous work by our group demonstrated that 4-month-old infants are also sensitive to this cue. They behaviourally discriminate a complex tone with a mistuned harmonic from the same complex with in-tune harmonics, and show an object-related event-related potential (ERP) electrophysiological (EEG) response to the stimulus with mistuned harmonics. In the present study we use an audiovisual procedure to investigate whether infants perceive a complex tone with an 8% mistuned harmonic as emanating from two objects, rather than merely detecting the mistuned cue. We paired in-tune and mistuned complex tones with visual displays that contained either one or two bouncing balls. Four-month-old infants showed surprise at the incongruous pairings, looking longer at the display of two balls when paired with the in-tune complex and at the display of one ball when paired with the mistuned harmonic complex. We conclude that infants use harmonicity as a cue for source separation when integrating auditory and visual information in object perception.

40-Hz oscillations underlying perceptual binding in young and older adults.

Auditory object perception requires binding of elementary features of complex stimuli. Synchronization of high-frequency oscillation in neural networks has been proposed as an effective alternative to binding via hard-wired connections because binding in an oscillatory network can be dynamically adjusted to the ever-changing sensory environment. Previously, we demonstrated in young adults that gamma oscillations are critical for sensory integration and found that they were affected by concurrent noise. Here, we aimed to support the hypothesis that stimulus evoked auditory 40-Hz responses are a component of thalamocortical gamma oscillations and examined whether this oscillatory system may become less effective in aging. In young and older adults, we recorded neuromagnetic 40-Hz oscillations, elicited by monaural amplitude-modulated sound. Comparing responses in quiet and under contralateral masking with multitalker babble noise revealed two functionally distinct components of auditory 40-Hz responses. The first component followed changes in the auditory input with high fidelity and was of similar amplitude in young and older adults. The second, significantly smaller in older adults, showed a 200-ms interval of amplitude and phase rebound and was strongly attenuated by contralateral noise. The amplitude of the second component was correlated with behavioral speech-in-noise performance. Concurrent noise also reduced the P2 wave of auditory evoked responses at 200-ms latency, but not the earlier N1 wave. P2 modulation was reduced in older adults. The results support the model of sensory binding through thalamocortical gamma oscillations. Limitation of neural resources for this process in older adults may contribute to their speech-in-noise understanding deficits.

Auditory object feature maps with a hierarchical network of independent components?

Auditory Object representation in the brain is still a controversial question [1,2]. Kumar et al. [3] discuss the hierarchical organization for auditory object perception and observe that the Planum Temporale (PT) area of the cortex encodes invariant representations of the spectral envelops of sounds. Many other studies find maps of representations elsewhere in the brain (Cochlear Nucleus, Inferior Nucleus, etc.). Sparse representations with minimum overlap could be considered, according to Barlow [4]. Griffiths and Warren [5] propose that auditory object representations might be segmented or segregated in the Planum Temporal (PT) by increasing the independence between the neural activities. We therefore explore the potential of a hierarchical neural assembly - with the use of a computer simulation - whose layers increase the feature independence during training, to represent auditory object parts. It is observed that learned features are organized into non-overlapping maps (Figure ​(Figure1)1) and that redundancy of the representation is in fact reduced. Learning was done on three categories of sounds having distinct acoustical statistics: speech, music and natural sounds. We observed that the learned feature maps are very different from one sound category to another and might be, to some extend, comparable to receptive fields measured in the brain. We discuss of their potential similarity with receptive fields measured in the Inferior Colliculus of the Guinea Pig and how they might be part of a representation of auditory objects in the brain. Figure 1 Learned representations illustrated for speech and natural sounds. FastICA training is first done on patches of 80ms x 32 cochlear channels of the envelopes coming from a 128 channels cochleagram (Level L0). Then, patches of 160ms x 64 cochlear channels ...

Beeinflussung objektbildender Leistungen des Hörsinns durch multimodale Interaktion

The task of assigning concurrent sounds to different auditory objects is known to depend on temporal and spectral cues. When tones of high and low frequencies are presented in alternation, they can be perceived as a single (integrated) melody, or as two parallel (segregated) melodic lines, according to the presentation rate and frequency distance between the sounds. At an intermediate distance or stimulation rate, the percept is ambiguous and alternates between segregated and integrated. This work studied whether an ambiguous sound organization could be modulated towards a robust integrated or a segregated percept by the synchronous presentation of visual cues. Two interleaved sets of sounds, one high frequency and one low frequency set were presented with concurrent visual stimuli, synchronized to either a within-set frequency pattern or to the across-set intensity pattern. Elicitation of the mismatch negativity (MMN) component of event-related brain potentials served as indices for the segregated organization, when no task was performed with the sounds. As a result, MMN was elicited only when the visual pattern promoted the segregation of the sounds. By spatial analysis of the distribution of electromagnetic potentials, four separated neuronal sources underlying the obtained MMN response were identified. One pair was located bilaterally in temporal cortical structures and another pair in occipital areas, representing the auditory and visual origin of the MMN response, evoked by inverted triplets as used in this study. Thus, the results demonstrate cross-modal effects of visual information on auditory object perception.

Toward a neurobiology of auditory object perception: What can we learn from the songbird forebrain?

Abstract In the acoustic world, no sounds occur entirely in isolation; they always reach the ears in combination with other sounds. How any given sound is discriminated and perceived as an independent auditory object is a challenging question in neuroscience. Although our knowledge of neural processing in the auditory pathway has expanded over the years, no good theory exists to explain how perception of auditory objects is achieved. A growing body of evidence suggests that the selectivity of neurons in the auditory forebrain is under dynamic modulation, and this plasticity may contribute to auditory object perception. We propose that stimulus-specific adaptation in the auditory forebrain of the songbird (and perhaps in other systems) may play an important role in modulating sensitivity in a way that aids discrimination, and thus can potentially contribute to auditory object perception.

In search for the prerequisites of stable auditory object perception

In search for the prerequisites of stable auditory object perception

Seeing and Hearing Directly

According to Paul Snowdon, one directly perceives an object x iff one is in a position to make a true demonstrative judgement of the form “That is x”. Whenever one perceives an object x indirectly (or dependently, as Snowdon puts it) it is the case that there exists an item y (which is not identical to x) such that one can count as demonstrating x only if one acknowledges that y bears a certain relation to x. In this paper I argue that what we hear directly are sounds, and that material objects (such as violins and goldfinches) are only indirectly heard. However, there are cases of auditory object perception that should count as direct: Some blind persons’ ears are so sensitive to the way sound waves are modified by things in their surroundings that they can detect objects such as other persons, fences or trees. Interestingly, objects localized in this way make themselves felt via a kind of pressure in the perceiver’s face (that is why the phenomenon is commonly called “facial vision”), the perception is phenomenally quite different from hearing. Since, to some degree, most people are able to conclude from the way it sounds that, say, they stand at the foot of a concrete wall (when there is enough traffic noise around), we can imagine situations where two persons perceive the same wall, one indirectly (demonstratively apprehending sounds) and the other directly (demonstratively apprehending nothing but the wall). These cases invite us to discuss the role phenomenology plays in determining whether an object is perceived directly or indirectly.

Object‐related negativity indexes perception of sounds beyond the echo threshold

The ability to isolate a single sound source among concurrent sources and reverberant energy is crucial for understanding the auditory world. The precedence effect describes an experimental finding that when listeners are presented with identical sounds from two locations with a short onset asynchrony, they report hearing a single source with a location dominated by the lead sound. We recently showed that event‐related potentials (ERPs) elicited by click pairs differ for trials on which listeners do and do not report hearing the lag sound as a separate source. Specifically, when participants report hearing two sounds, we observe a negativity between 100 and 250 ms, previously termed the object‐related negativity (ORN). These results indicate that the precedence effect reflects top‐down cognitive influence over early neurosensory processing. Additionally, the study provides support for the claim that the ORN, initially reported for sounds perceived as two concurrent pitches, indexes auditory object perception. The effects on the ORN of repeating identical click pairs, a condition that has previously been shown to increase echo thresholds, will also be discussed in the context of listeners forming complex models of room acoustics based on what they hear in a specific setting.

Auditory temporal grouping in newborn infants

Adults normally perceive auditory scenes in terms of sound patterns emitted by concurrently active sources. Thus pattern formation is an important process of auditory object perception. The aim of the present study was to determine whether neonates group sounds by repeating pitch patterns. Standard ("S"; p=80%) and deviant tones ("D", p=20%) differing only in pitch were delivered either in a randomized order (random condition) or in a repeating SSSSD pattern (grouped condition). Both event-related brain potentials and gamma-band activity differed between the S and D tones in the random condition but not in the grouped condition. These results suggest that in the grouped condition, the S and D tones were processed as part of the same higher order regularity by the neonate auditory system. Also, for the first time, we observed oscillatory gamma-band activity in neonates, which was sensitive to infrequent pitch changes.

Visual cues can modulate integration and segregation of objects in auditory scene analysis

The task of assigning concurrent sounds to different auditory objects is known to depend on temporal and spectral cues. When tones of high and low frequencies are presented in alternation, they can be perceived as a single, integrated melody, or as two parallel, segregated melodic lines, according to the presentation rate and frequency distance between the sounds. At an intermediate distance, in the ‘ambiguous’ range, both percepts are possible. We conducted an electrophysiological experiment to determine whether an ambiguous sound organization could be modulated toward an integrated or segregated percept by the synchronous presentation of visual cues. Two sets of sounds (one high frequency and one low frequency) were interleaved. To promote integration or segregation, visual stimuli were synchronized to either the within-set frequency pattern or to the across-set intensity pattern. Elicitation of the mismatch negativity (MMN) component of event-related brain potentials was used to index the segregated organization, when no task was performed with the sounds. MMN was elicited only when the visual pattern promoted the segregation of the sounds. The results demonstrate cross-modal effects on auditory object perception in that sound ambiguity was resolved by synchronous presentation of visual stimuli, which promoted either an integrated or segregated perception of the sounds.

Task-modulated "what" and "where" pathways in human auditory cortex.

Human neuroimaging studies suggest that localization and identification of relevant auditory objects are accomplished via parallel parietal-to-lateral-prefrontal "where" and anterior-temporal-to-inferior-frontal "what" pathways, respectively. Using combined hemodynamic (functional MRI) and electromagnetic (magnetoencephalography) measurements, we investigated whether such dual pathways exist already in the human nonprimary auditory cortex, as suggested by animal models, and whether selective attention facilitates sound localization and identification by modulating these pathways in a feature-specific fashion. We found a double dissociation in response adaptation to sound pairs with phonetic vs. spatial sound changes, demonstrating that the human nonprimary auditory cortex indeed processes speech-sound identity and location in parallel anterior "what" (in anterolateral Heschl's gyrus, anterior superior temporal gyrus, and posterior planum polare) and posterior "where" (in planum temporale and posterior superior temporal gyrus) pathways as early as approximately 70-150 ms from stimulus onset. Our data further show that the "where" pathway is activated approximately 30 ms earlier than the "what" pathway, possibly enabling the brain to use top-down spatial information in auditory object perception. Notably, selectively attending to phonetic content modulated response adaptation in the "what" pathway, whereas attending to sound location produced analogous effects in the "where" pathway. This finding suggests that selective-attention effects are feature-specific in the human nonprimary auditory cortex and that they arise from enhanced tuning of receptive fields of task-relevant neuronal populations.

The precedence effect in three species of birds (Melopsittacus undulatus, Serinus canaria, and Taeniopygia guttata).

The perceived locations of paired auditory images, simulating direct sounds and their echoes, have been recently studied in budgerigars (Melopsittacus undulatus; M. L. Dent & R. J. Dooling, 2003a, 2003b). In this article, the authors extend those experiments to include measurements of the precedence effect using a discrimination paradigm in two additional bird species: canaries (Serinus canaria) and zebra finches (Taeniopygia guttata). Although time courses of summing localization, localization dominance, and echo thresholds were similar across all species, budgerigars had slightly higher overall levels of discrimination. The results from these experiments add further support that the precedence effect in birds is similar to that found in other animals and that the ability to suppress echoes that might degrade localization and auditory object perception may be a general property of the vertebrate auditory system.

An adaptive tuning procedure for the psychophysical study of complex sounds

Auditory object perception depends on the subjective criteria used by the observer to decompose an ambiguous acoustic field into auditory sources. Results from discrimination or detection experiments based on objective psychophysical procedures can only provide indirect evidence concerning the nature of the observer's source decomposition rules. Standard subjective psychophysical procedures provide direct evidence, but may be influenced by response bias and more importantly by experimenter bias in the choice of stimulus parameters to adjust. In the present paper, an adaptive tuning procedure is introduced which, unlike the method of adjustment, provides an efficient search over the stimulus space while minimizing experimenter bias in selecting that search space. The psychophysical procedure optimizes the stimulus parameter space according to a perceptual criterion adopted by the observer. It is claimed that inferences concerning the cues for source formation can be drawn from the values of the stimulus parameters to which the procedure converges. The procedure is applied to study the enhancement and degradation of single and multicomponent sinusoids in noise and to comodulated multicomponent sinusoids in noise to evaluate this claim. [Work supported by grants from the AFOSR (AFOSR‐87‐0193) and the NIH (NINDCD NS21440).]

Recognition memory for arbitrary, complex waveforms

Recognition memory for arbitrary complex stimuli was measured using the operating characteristic technique developed by Egan [J. P. Egan, AFCRC TN 58‐51, AD 152650 (1958)]. Ten complex stimuli (500 ms in duration), each consisting of six, randomly chosen, equal amplitude sinusoidal components spanning frequencies from 300 to 3000 Hz, were used as a training set. After 15 min of listening to the training set, listeners were presented a 20‐stimulus test set consisting of the ten training stimuli along with ten additional complex stimuli (generated in the same way as the ten training stimuli). In a recognition memory paradigm, listeners used a five‐point rating scale to rate their confidence that a stimulus presented from the test set was from the training set. Different constraints were placed on the selection of the complex waveforms to determine how those constraints affected recognition memory. For one condition there were no constraints except as described above; for a second condition the six‐tonal components were harmonics (randomly chosen) of a common fundamental; and for a third condition the six‐tone complex was sinusoidally amplitude modulated at rates of 4, 16, or 32 Hz. The results will be discussed in terms of auditory object perception and the use of receiver operating techniques to measure recognition memory. [Work supported by a Program Project Grant from the NINCDS and a grant from AFOSR.]