Auditory Perceptual Organization Research Articles

Treefrogs exploit temporal coherence to form perceptual objects of communication signals.

For many animals, navigating their environment requires an ability to organize continuous streams of sensory input into discrete 'perceptual objects' that correspond to physical entities in visual and auditory scenes. The human visual and auditory systems follow several Gestalt laws of perceptual organization to bind constituent features into coherent perceptual objects. A largely unexplored question is whether nonhuman animals follow similar Gestalt laws in perceiving behaviourally relevant stimuli, such as communication signals. We used females of Cope's grey treefrog (Hyla chrysoscelis) to test the hypothesis that temporal coherence-a powerful Gestalt principle in human auditory scene analysis-promotes perceptual binding in forming auditory objects of species-typical vocalizations. According to the principle of temporal coherence, sound elements that start and stop at the same time or that modulate coherently over time are likely to become bound together into the same auditory object. We found that the natural temporal coherence between two spectral components of advertisement calls promotes their perceptual binding into auditory objects of advertisement calls. Our findings confirm the broad ecological validity of temporal coherence as a Gestalt law of auditory perceptual organization guiding the formation of biologically relevant perceptual objects in animal behaviour.

Selective entrainment of brain oscillations drives auditory perceptual organization

Perceptual sound organization supports our ability to make sense of the complex acoustic environment, to understand speech and to enjoy music. However, the neuronal mechanisms underlying the subjective experience of perceiving univocal auditory patterns that can be listened to, despite hearing all sounds in a scene, are poorly understood. We hereby investigated the manner in which competing sound organizations are simultaneously represented by specific brain activity patterns and the way attention and task demands prime the internal model generating the current percept. Using a selective attention task on ambiguous auditory stimulation coupled with EEG recordings, we found that the phase of low-frequency oscillatory activity dynamically tracks multiple sound organizations concurrently. However, whereas the representation of ignored sound patterns is circumscribed to auditory regions, large-scale oscillatory entrainment in auditory, sensory-motor and executive-control network areas reflects the active perceptual organization, thereby giving rise to the subjective experience of a unitary percept.

Auditory multistability and neurotransmitter concentrations in the human brain.

Multistability in perception is a powerful tool for investigating sensory-perceptual transformations, because it produces dissociations between sensory inputs and subjective experience. Spontaneous switching between different perceptual objects occurs during prolonged listening to a sound sequence of tone triplets or repeated words (termed auditory streaming and verbal transformations, respectively). We used these examples of auditory multistability to examine to what extent neurochemical and cognitive factors influence the observed idiosyncratic patterns of switching between perceptual objects. The concentrations of glutamate-glutamine (Glx) and γ-aminobutyric acid (GABA) in brain regions were measured by magnetic resonance spectroscopy, while personality traits and executive functions were assessed using questionnaires and response inhibition tasks. Idiosyncratic patterns of perceptual switching in the two multistable stimulus configurations were identified using a multidimensional scaling (MDS) analysis. Intriguingly, although switching patterns within each individual differed between auditory streaming and verbal transformations, similar MDS dimensions were extracted separately from the two datasets. Individual switching patterns were significantly correlated with Glx and GABA concentrations in auditory cortex and inferior frontal cortex but not with the personality traits and executive functions. Our results suggest that auditory perceptual organization depends on the balance between neural excitation and inhibition in different brain regions.This article is part of the themed issue 'Auditory and visual scene analysis'.

音楽と聴覚の知覚的体制化(音楽の人間科学,2010年度 第2回フォーラム)

音楽と聴覚の知覚的体制化(音楽の人間科学,2010年度 第2回フォーラム)

Effects of source- and head-motion on auditory perceptual organization

The perceptual organization of auditory scenes needs to parse the incoming flow of acoustic inputs into perceptual streams. It is likely that cues from several modalities are pooled for auditory scene analysis, including sensory-motor cues related to the active exploration of the scene. We show effects of source- and head-motion on auditory streaming. In a streaming paradigm, listeners hear a sequence of repeating tone triplets and indicate their perception of one or two subjective sources called streams. We used a robotic telepresence system, Telehead, to disentangle the effects of head motion: changes in acoustic cues at the ear, subjective location cues, and motor cues. We found that head motion induced perceptual reorganization even when the auditory scene had not changed. We further analyzed the data to probe the time course of sensory-motor integration. Motor cues impacted perceptual organization earlier and for a shorter time than acoustic or subjective location cues, with successive positive and negative contributions to streaming. An additional experiment showed that arm or leg movements did not have an impact on scene analysis. Our results suggest a loose temporal coupling between the different mechanisms involved.

Subcortical correlates of auditory perceptual organization in humans

To make sense of complex auditory scenes, the auditory system sequentially organizes auditory components into perceptual objects or streams. In the conventional view of this process, the cortex plays a major role in perceptual organization, and subcortical mechanisms merely provide the cortex with acoustical features. Here, we show that the neural activities of the brainstem are linked to perceptual organization, which alternates spontaneously for human listeners without any stimulus change. The stimulus used in the experiment was an unchanging sequence of repeated triplet tones, which can be interpreted as either one or two streams. Listeners were instructed to report the perceptual states whenever they experienced perceptual switching between one and two streams throughout the stimulus presentation. Simultaneously, we recorded event related potentials with scalp electrodes. We measured the frequency-following response (FFR), which is considered to originate from the brainstem. We also assessed thalamo-cortical activity through the middle-latency response (MLR). The results demonstrate that the FFR and MLR varied with the state of auditory stream perception. In addition, we found that the MLR change precedes the FFR change with perceptual switching from a one-stream to a two-stream percept. This suggests that there are top-down influences on brainstem activity from the thalamo-cortical pathway. These findings are consistent with the idea of a distributed, hierarchical neural network for perceptual organization and suggest that the network extends to the brainstem level.

Language and short-term memory: The role of perceptual-motor affordance.

The advantage for real words over nonwords in serial recall—the lexicality effect—is typically attributed to support for item-level phonology, either via redintegration, whereby partially degraded short-term traces are “cleaned up” via support from long-term representations of the phonological material or via the more robust temporary activation of long-term lexical phonological knowledge that derives from its combination with established lexical and semantic levels of representation. The much smaller effect of lexicality in serial recognition, where the items are re-presented in the recognition cue, is attributed either to the minimal role for redintegration from long-term memory or to the minimal role for item memory itself in such retrieval conditions. We show that the reduced lexicality effect in serial recognition is not a function of the retrieval conditions, but rather because previous demonstrations have used auditory presentation, and we demonstrate a robust lexicality effect for visual serial recognition in a setting where auditory presentation produces no such effect. Furthermore, this effect is abolished under conditions of articulatory suppression. We argue that linguistic knowledge affects the readiness with which verbal material is segmentally recoded via speech motor processes that support rehearsal and therefore affects tasks that involve recoding. On the other hand, auditory perceptual organization affords sequence matching in the absence of such a requirement for segmental recoding and therefore does not show such effects of linguistic knowledge.

Performance-based and subjective measures of perceptual organization in humans

Measures of auditory perceptual organization in humans have traditionally relied on subjective reports from subjects. For instance, in alternating tone sequences, subjects might be asked to report whether they hear one or two streams. In order to investigate perceptual organization in non-human species, it can be useful to devise tests that are more objective, in that they have a right and a wrong answer and do not rely on introspection. Recent studies from our laboratory have employed various objective and subjective tasks to provide converging evidence in the search for the underlying principles of auditory perceptual organization. It is suggested that different tasks can bias potentially multi-stable percepts in one way or another, which in turn may be useful in uncovering neural correlates of perceptual organization that can vary even when the acoustic stimuli remain the same. [Work supported by NIH grant R01DC007657.]

Auditory distraction and perceptual organization: Streams of unconscious processing

Perceptual organization is key to understanding auditory distraction. In order to achieve a fundamental understanding of distraction it is necessary to understand how auditory stimuli are perceived; specifically, how they are organized into entities that do not map directly onto simple single stimuli as defined by the experimenter. It is important not to mistake some arbitrary unit of analysis, such as the word, as the correct unit for understanding auditory processing; rather, the unit of the auditory object and its relative position to other auditory objects is the key to understanding distraction (as well as the whole of auditory cognition more generally). Here I provide two illustrative examples of auditory perceptual organization showing the superlative power of organizational principles: streaming by similarity and stimulus capture. I go on to show how these have been used to refine our understanding of distraction, and of the effects of distraction from sequences of sound, from single sounds, or single changes within a sequence. A common feature of work described here is that it compares the effects of different forms of organization: The nominal stimuli themselves are largely unchanged but the way they relate to each other can change distraction appreciably. That is, it is not the mere presence of sound that causes distraction but its organization and the way that relates to the currently prevailing activity.

The role of spatiotemporal and spectral cues in segregating short sound events: evidence from auditory Ternus display

Previous studies using auditory sequences with rapid repetition of tones revealed that spatiotemporal cues and spectral cues are important cues used to fuse or segregate sound streams. However, the perceptual grouping was partially driven by the cognitive processing of the periodicity cues of the long sequence. Here, we investigate whether perceptual groupings (spatiotemporal grouping vs. frequency grouping) could also be applicable to short auditory sequences, where auditory perceptual organization is mainly subserved by lower levels of perceptual processing. To find the answer to that question, we conducted two experiments using an auditory Ternus display. The display was composed of three speakers (A, B and C), with each speaker consecutively emitting one sound consisting of two frames (AB and BC). Experiment 1 manipulated both spatial and temporal factors. We implemented three 'within-frame intervals' (WFIs, or intervals between A and B, and between B and C), seven 'inter-frame intervals' (IFIs, or intervals between AB and BC) and two different speaker layouts (inter-distance of speakers: near or far). Experiment 2 manipulated the differentiations of frequencies between two auditory frames, in addition to the spatiotemporal cues as in Experiment 1. Listeners were required to make two alternative forced choices (2AFC) to report the perception of a given Ternus display: element motion (auditory apparent motion from sound A to B to C) or group motion (auditory apparent motion from sound 'AB' to 'BC'). The results indicate that the perceptual grouping of short auditory sequences (materialized by the perceptual decisions of the auditory Ternus display) was modulated by temporal and spectral cues, with the latter contributing more to segregating auditory events. Spatial layout plays a less role in perceptual organization. These results could be accounted for by the 'peripheral channeling' theory.

Segregation of complex acoustic scenes based on temporal coherence

In contrast to the complex acoustic environments we encounter everyday, most studies of auditory segregation have used relatively simple signals. Here, we synthesized a new stimulus to examine the detection of coherent patterns ('figures') from overlapping 'background' signals. In a series of experiments, we demonstrate that human listeners are remarkably sensitive to the emergence of such figures and can tolerate a variety of spectral and temporal perturbations. This robust behavior is consistent with the existence of automatic auditory segregation mechanisms that are highly sensitive to correlations across frequency and time. The observed behavior cannot be explained purely on the basis of adaptation-based models used to explain the segregation of deterministic narrowband signals. We show that the present results are consistent with the predictions of a model of auditory perceptual organization based on temporal coherence. Our data thus support a role for temporal coherence as an organizational principle underlying auditory segregation. DOI:http://dx.doi.org/10.7554/eLife.00699.001.

Role of serial order in the impact of talker variability on short-term memory: testing a perceptual organization-based account

In two experiments, we examined the impact of the degree of match between sequential auditory perceptual organization processes and the demands of a short-term memory task (memory for order vs. item information). When a spoken sequence of digits was presented so as to promote its perceptual partitioning into two distinct streams by conveying it in alternating female (F) and male (M) voices (FMFMFMFM)--thereby disturbing the perception of true temporal order--recall of item order was greatly impaired (as compared to recall of item identity). Moreover, an order error type consistent with the formation of voice-based streams was committed more quickly in the alternating-voice condition (Exp. 1). In contrast, when the perceptual organization of the sequence mapped well onto an optimal two-group serial rehearsal strategy--by presenting the two voices in discrete clusters (FFFFMMMM)--order, but not item, recall was enhanced (Exp. 2). The results are consistent with the view that the degree of compatibility between perceptual and deliberate sequencing processes is a key determinant of serial short-term memory performance. Alternative accounts of talker variability effects in short-term memory, based on the concept of a dedicated phonological short-term store and a capacity-limited focus of attention, are also reviewed.

The Effect of Auditory Stream Segregation on Synchronized Tapping

We can tap in synchrony with a simple sequence of auditory tones, but our taps generally precede it by several tens of milliseconds(“negative mean asynchrony”). A simple sequence of sounds sometimes causes auditory perceptual organization different from the physical sequence. However little is known about interactions between the auditory perceptual organization and sensorimotor synchronization. The present study aimed at investigating effects of the auditory stream segregation (one stream or two) on the synchronized tapping. Participants listened to a sequence of two different auditory tones presented alternately, and were asked to tap in synchrony with one group of tones (ie, high tones or low) with index fingers. After that, they were also asked whether the tones were perceived one or two streams. The results showed that negative mean asynchrony was caused both in perceived one stream and two. However, the amplitude of the asynchrony was smaller when they perceived the tones as one stream than as two. This indicates that auditory perceptual organization affects sensorimotor synchronization even if auditory tones were presented with the same timing. When we synchronize bodily with auditory tones as in the case of a motor rehabilitation, we have to take account of auditory perceptual organization.

Auditory distraction and serial memory: The avoidable and the ineluctable

One mental activity that is very vulnerable to auditory distraction is serial recall. This review of the contemporary findings relating to serial recall charts the key determinants of distraction. It is evident that there is one form of distraction that is a joint product of the cognitive characteristics of the task and of the obligatory cognitive processing of the sound. For sequences of sound, distraction appears to be an ineluctable product of similarity-of-process, specifically, the serial order processing of the visually presented items and the serial order coding that is the by-product of the streaming of the sound. However, recently emerging work shows that the distraction from a single sound (one deviating from a prevailing sequence) results in attentional capture and is qualitatively distinct from that of a sequence in being restricted in its action to encoding, not to rehearsal of list members. Capture is also sensitive to the sensory task load, suggesting that it is subject to top-down control and therefore avoidable. These two forms of distraction-conflict of process and attentional capture-may be two consequences of auditory perceptual organization processes that serve to strike the optimal balance between attentional selectivity and distractability.

Neurophysiological mechanisms involved in auditory perceptual organization

In our complex acoustic environment, we are confronted with a mixture of sounds produced by several simultaneous sources. However, we rarely perceive these sounds as incomprehensible noise. Our brain uses perceptual organization processes to independently follow the emission of each sound source over time. If the acoustic properties exploited in these processes are well-established, the neurophysiological mechanisms involved in auditory scene analysis remain unclear and have recently raised more interest. Here, we review the studies investigating these mechanisms using electrophysiological recordings from the cochlear nucleus to the auditory cortex, in animals and humans. Their findings reveal that basic mechanisms such as frequency selectivity, forward suppression and multi-second habituation shape the automatic brain responses to sounds in a way that can account for several important characteristics of perceptual organization of both simultaneous and successive sounds. One challenging question remains unresolved: how are the resulting activity patterns integrated to yield the corresponding conscious percepts?

Distal prosodic context affects word segmentation and lexical processing

Three experiments investigated the role of distal (i.e., nonlocal) prosody in word segmentation and lexical processing. In Experiment 1, prosodic characteristics of the initial five syllables of eight-syllable sequences were manipulated; the final portions of these sequences were lexically ambiguous (e.g., note bookworm, notebook worm). Distal prosodic context affected the rate with which participants heard disyllabic final words, although identical acoustic material was judged. In Experiment 2, removing four syllables of initial context reduced the magnitude of the distal prosodic effect. Experiment 3 used a study-test recognition design; better recognition was demonstrated for visually-presented disyllabic words when these items were comprised of adjacent syllables previously heard in distal prosodic contexts predicted to facilitate perceptual grouping of these two syllables. Overall, this research identifies distal prosody as a new factor in word segmentation and lexical processing and provides support for a perceptual grouping hypothesis derived from principles of auditory perceptual organization.

Asynchrony tolerance in the perceptual organization of speech

Researchers have claimed that listeners tolerate large temporal distortion when integrating the spectral components of speech. In some estimates, perceivers resolve linguistic attributes at spectral desynchronies as great as the duration of a syllable. We obtained new measures of perceptual tolerance of auditory asynchrony, using sine-wave synthesis in order to require perceivers to resolve the speech stream dynamically. Listeners transcribed sentences in which the tone analogue of a second formant was desynchronized relative to the remaining tones of a sentence, with desynchrony ranging from a 250-msec lead to a 250-msec lag. Intelligibility declined symmetrically from 72% at synchrony to 7% at +/-100 msec. This finding of narrow asynchrony tolerance indicates a time-critical feature of the auditory perceptual organization of speech.

A mismatch negativity study of local–global auditory processing

We used mismatch negativity (MMN) to examine structural encoding of local and global auditory patterns in perceptual memory. Unlike previous MMN studies of local–global auditory perceptual organization that used interval-contour stimuli, here we presented hierarchical stimuli in which local pattern organization formed global patterns. Importantly, our stimuli allowed independent manipulation of the two structural levels. In separate blocks, participants were exposed to frequent local standard patterns and rare local deviant patterns, or to frequent global standard patterns and rare global deviant patterns. Within each deviant pattern, the variation from the standard pattern could occur at onset ( early), towards the end of the pattern ( late) or over both time windows ( both). To isolate pattern indexing at one level, the other level continuously changed (e.g., in a global standard block, local elements varied trial-by-trial). MMN was found only for global deviant patterns, and only when deviation occurred late in the pattern. In a separate behavioral experiment, global deviants were detected more often than local ones, although initial similarity followed by a late deviation from the standard pattern was not required for explicit deviant detection (as with the MMN). This report demonstrates neural structural encoding for global information, when independently manipulated from local information. Furthermore, it extends previous MMN findings that have revealed indexing of complex abstract auditory information to the realm of hierarchical perceptual organization.

Temporal Dynamics of Auditory and Visual Bistability Reveal Common Principles of Perceptual Organization

When dealing with natural scenes, sensory systems have to process an often messy and ambiguous flow of information. A stable perceptual organization nevertheless has to be achieved in order to guide behavior. The neural mechanisms involved can be highlighted by intrinsically ambiguous situations. In such cases, bistable perception occurs: distinct interpretations of the unchanging stimulus alternate spontaneously in the mind of the observer. Bistable stimuli have been used extensively for more than two centuries to study visual perception. Here we demonstrate that bistable perception also occurs in the auditory modality. We compared the temporal dynamics of percept alternations observed during auditory streaming with those observed for visual plaids and the susceptibilities of both modalities to volitional control. Strong similarities indicate that auditory and visual alternations share common principles of perceptual bistability. The absence of correlation across modalities for subject-specific biases, however, suggests that these common principles are implemented at least partly independently across sensory modalities. We propose that visual and auditory perceptual organization could rely on distributed but functionally similar neural competition mechanisms aimed at resolving sensory ambiguities.

The role of predictive models in the formation of auditory streams

Sounds provide us with useful information about our environment which complements that provided by other senses, but also poses specific processing problems. How does the auditory system distentangle sounds from different sound sources? And what is it that allows intermittent sound events from the same source to be associated with each other? Here we review findings from a wide range of studies using the auditory streaming paradigm in order to formulate a unified account of the processes underlying auditory perceptual organization. We present new computational modelling results which replicate responses in primary auditory cortex [Fishman, Y.I., Arezzo, J.C., Steinschneider, M., 2004. Auditory stream segregation in monkey auditory cortex: effects of frequency separation, presentation rate, and tone duration. J. Acoust. Soc. Am. 116, 1656–1670; Fishman, Y. I., Reser, D. H., Arezzo, J.C., Steinschneider, M., 2001. Neural correlates of auditory stream segregation in primary auditory cortex of the awake monkey. Hear. Res. 151, 167–187] to tone sequences. We also present the results of a perceptual experiment which confirm the bi-stable nature of auditory streaming, and the proposal that the gradual build-up of streaming may be an artefact of averaging across many subjects [Pressnitzer, D., Hupé, J. M., 2006. Temporal dynamics of auditory and visual bi-stability reveal common principles of perceptual organization. Curr. Biol. 16(13), 1351–1357.]. Finally we argue that in order to account for all of the experimental findings, computational models of auditory stream segregation require four basic processing elements; segregation, predictive modelling, competition and adaptation, and that it is the formation of effective predictive models which allows the system to keep track of different sound sources in a complex auditory environment.