Ventral Auditory Stream Research Articles

Neural correlates of musical timbre: an ALE meta-analysis of neuroimaging data.

Timbre is a central aspect of music that allows listeners to identify musical sounds and conveys musical emotion, but also allows for the recognition of actions and is an important structuring property of music. The former functions are known to be implemented in a ventral auditory stream in processing musical timbre. While the latter functions are commonly attributed to areas in a dorsal auditory processing stream in other musical domains, its involvement in musical timbre processing is so far unknown. To investigate if musical timbre processing involves both dorsal and ventral auditory pathways, we carried out an activation likelihood estimation (ALE) meta-analysis of 18 experiments from 17 published neuroimaging studies on musical timbre perception. We identified consistent activations in Brodmann areas (BA) 41, 42, and 22 in the bilateral transverse temporal gyri, the posterior superior temporal gyri and planum temporale, in BA 40 of the bilateral inferior parietal lobe, in BA 13 in the bilateral posterior Insula, and in BA 13 and 22 in the right anterior insula and superior temporal gyrus. The vast majority of the identified regions are associated with the dorsal and ventral auditory processing streams. We therefore propose to frame the processing of musical timbre in a dual-stream model. Moreover, the regions activated in processing timbre show similarities to the brain regions involved in processing several other fundamental aspects of music, indicating possible shared neural bases of musical timbre and other musical domains.

Preliminary Evidence for Global Properties in Human Listeners During Natural Auditory Scene Perception.

Theories of auditory and visual scene analysis suggest the perception of scenes relies on the identification and segregation of objects within it, resembling a detail-oriented processing style. However, a more global process may occur while analyzing scenes, which has been evidenced in the visual domain. It is our understanding that a similar line of research has not been explored in the auditory domain; therefore, we evaluated the contributions of high-level global and low-level acoustic information to auditory scene perception. An additional aim was to increase the field's ecological validity by using and making available a new collection of high-quality auditory scenes. Participants rated scenes on 8 global properties (e.g., open vs. enclosed) and an acoustic analysis evaluated which low-level features predicted the ratings. We submitted the acoustic measures and average ratings of the global properties to separate exploratory factor analyses (EFAs). The EFA of the acoustic measures revealed a seven-factor structure explaining 57% of the variance in the data, while the EFA of the global property measures revealed a two-factor structure explaining 64% of the variance in the data. Regression analyses revealed each global property was predicted by at least one acoustic variable (R2 = 0.33-0.87). These findings were extended using deep neural network models where we examined correlations between human ratings of global properties and deep embeddings of two computational models: an object-based model and a scene-based model. The results support that participants' ratings are more strongly explained by a global analysis of the scene setting, though the relationship between scene perception and auditory perception is multifaceted, with differing correlation patterns evident between the two models. Taken together, our results provide evidence for the ability to perceive auditory scenes from a global perspective. Some of the acoustic measures predicted ratings of global scene perception, suggesting representations of auditory objects may be transformed through many stages of processing in the ventral auditory stream, similar to what has been proposed in the ventral visual stream. These findings and the open availability of our scene collection will make future studies on perception, attention, and memory for natural auditory scenes possible.

Auditory cortical connectivity in humans.

To understand auditory cortical processing, the effective connectivity between 15 auditory cortical regions and 360 cortical regions was measured in 171 Human Connectome Project participants, and complemented with functional connectivity and diffusion tractography. 1. A hierarchy of auditory cortical processing was identified from Core regions (including A1) to Belt regions LBelt, MBelt, and 52; then to PBelt; and then to HCP A4. 2. A4 has connectivity to anterior temporal lobe TA2, and to HCP A5, which connects to dorsal-bank superior temporal sulcus (STS) regions STGa, STSda, and STSdp. These STS regions also receive visual inputs about moving faces and objects, which are combined with auditory information to help implement multimodal object identification, such as who is speaking, and what is being said. Consistent with this being a "what" ventral auditory stream, these STS regions then have effective connectivity to TPOJ1, STV, PSL, TGv, TGd, and PGi, which are language-related semantic regions connecting to Broca's area, especially BA45. 3. A4 and A5 also have effective connectivity to MT and MST, which connect to superior parietal regions forming a dorsal auditory "where" stream involved in actions in space. Connections of PBelt, A4, and A5 with BA44 may form a language-related dorsal stream.

Psychopathic and autistic traits differentially influence the neural mechanisms of social cognition from communication signals

Psychopathy is associated with severe deviations in social behavior and cognition. While previous research described such cognitive and neural alterations in the processing of rather specific social information from human expressions, some open questions remain concerning central and differential neurocognitive deficits underlying psychopathic behavior. Here we investigated three rather unexplored factors to explain these deficits, first, by assessing psychopathy subtypes in social cognition, second, by investigating the discrimination of social communication sounds (speech, non-speech) from other non-social sounds, and third, by determining the neural overlap in social cognition impairments with autistic traits, given potential common deficits in the processing of communicative voice signals. The study was exploratory with a focus on how psychopathic and autistic traits differentially influence the function of social cognitive and affective brain networks in response to social voice stimuli. We used a parametric data analysis approach from a sample of 113 participants (47 male, 66 female) with ages ranging between 18 and 40 years (mean 25.59, SD 4.79). Our data revealed four important findings. First, we found a phenotypical overlap between secondary but not primary psychopathy with autistic traits. Second, primary psychopathy showed various neural deficits in neural voice processing nodes (speech, non-speech voices) and in brain systems for social cognition (mirroring, mentalizing, empathy, emotional contagion). Primary psychopathy also showed deficits in the basal ganglia (BG) system that seems specific to the social decoding of communicative voice signals. Third, neural deviations in secondary psychopathy were restricted to social mirroring and mentalizing impairments, but with additional and so far undescribed deficits at the level of auditory sensory processing, potentially concerning deficits in ventral auditory stream mechanisms (auditory object identification). Fourth, high autistic traits also revealed neural deviations in sensory cortices, but rather in the dorsal auditory processing streams (communicative context encoding). Taken together, social cognition of voice signals shows considerable deviations in psychopathy, with differential and newly described deficits in the BG system in primary psychopathy and at the neural level of sensory processing in secondary psychopathy. These deficits seem especially triggered during the social cognition from vocal communication signals.

Language and music side by side: post-stroke aprosodia and amusia are subserved by the same brain networks.

When I was a medical student—in the ’70s of the last century—I was taught that language is processed in the left, music in the right hemisphere. At the time this was state of knowledge, based on a history of more than 100 years of clinical case reports, such as early observations by the French neurologist Jean-Baptiste Bouillaud who documented the capacity to sing in two patients with aphasia. Remarkably, he reported on a 50-year-old composer with aphasia and verbal agraphia who was able to compose a new tune and write down the notes. The patient was also able to sing or ‘modulate his voice’, thus maintaining emotional prosody (for an excellent review on these historical cases, see [1]. Within 50 years, a much more complex picture has emerged and is still further developing. Neuropsychology is a dynamically evolving field and benefits from the enormous progress in standardized clinical testing methods as well as in neuroimaging methods. In a beautiful study in this volume of the European Journal of Neurology, Sihvonen et al. [2] have carefully assessed lesion patterns and structural connectivity in stroke patients at the 3-week subacute and 3-month post-stroke stages to investigate the neural correlates of amusia and aprosodia. Thirty-nine patients with right or left hemisphere stroke were enrolled and tested for linguistic and affective prosody and musical pitch and rhythm perception. Structural magnetic resonance imaging and machine-learning-based lesion-symptom mapping was performed in line with deterministic tractography. The results showed that both aprosodia and amusia are strongly correlated and associated with comparable lesion patterns in right fronto-insular and striatal structures. Multiple regression models revealed fractional anisotropy changes and lower tract volume in the right inferior fronto-occipital fasciculus as the strongest predictors for both neuropsychological deficits over time. This is a convincing paper, demonstrating the core neural commonality of amusia and aprosodia. Both arise from damage and disconnection of the right ventral auditory stream, which accordingly seems to be crucial for integrating rhythmic and melodic acoustic information in prosody and music perception. The fact that deficits in emotional prosody recognition and melody processing are highly correlated is per se not surprising. The term ‘speech melody’ is used colloquially for affective prosody. Patel [3] has formulated the OPERA thesis, which essentially emphasizes the anatomical overlap of brain networks that process acoustic features used in both music and speech, for example waveform periodicity, amplitude envelope or fundamental frequency modulations. However, the detailed identification of anatomical structures underlying these common perceptual faculties is new. Another result is worth mentioning: out of the 24 patients with aprosodia or amusia in the subacute phase, 16 did not show these deficits any more in the 3-month follow-up. Obviously, there is an enormous re-organization occurring in the first 3 months after the lesion, behaviourally as well as functionally. For amusia, this has been demonstrated in a study by the same group [4]. Similarly, in an earlier behavioural study no more auditory deficit was found after 1 year in two out of eight patients suffering from severe amusia, demonstrating compensating functional neuroplasticity [5]. Although music induced emotions were not directly tested in this study, it is highly probable that deficits in melody and rhythm processing will result in reduced appreciation of music. This has consequences for the rehabilitation of stroke patients. If patients are unable to respond emotionally to music, they will most probably not benefit from the enormous effect preferred music has in the rehabilitation of cognitive and emotional deficits following stroke. Work from the same group by Särkämö et al. [6], has convincingly demonstrated that listening to favourite musical excerpts improves short-term memory, executive functions, language impairment or depression in stroke patients. Consequently, nowadays preferred music listening is offered to stroke victims to facilitate neuropsychological rehabilitation in many stroke units. Similarly neurological music therapy in stroke patients with motor deficits, applying musical sonification of arm movements to promote motor rehabilitation, is driven by the motivational and emotional power of music [7]. Given such a therapeutic potential of music, in the case of amusia, music therapists and neuropsychologists should establish therapies, exercises and other means to re-enable emotional appreciation of Apollo’s gift. This is a task that has not yet been tackled. Open access funding enabled and organized by ProjektDEAL. The authors have no conflict of interest. Eckart Altenmüller: Conceptualization (equal); writing—original draft (equal); writing—review and editing (equal).

Right ventral stream damage underlies both poststroke aprosodia and amusia.

This study was undertaken to determine and compare lesion patterns and structural dysconnectivity underlying poststroke aprosodia and amusia, using a data-driven multimodal neuroimaging approach. Thirty-nine patients with right or left hemisphere stroke were enrolled in a cohort study and tested for linguistic and affective prosody perception and musical pitch and rhythm perception at subacute and 3-month poststroke stages. Participants listened to words spoken with different prosodic stress that changed their meaning, and to words spoken with six different emotions, and chose which meaning or emotion was expressed. In the music tasks, participants judged pairs of short melodies as the same or different in terms of pitch or rhythm. Structural magnetic resonance imaging data were acquired at both stages, and machine learning-based lesion-symptom mapping and deterministic tractography were used to identify lesion patterns and damaged white matter pathways giving rise to aprosodia and amusia. Both aprosodia and amusia were behaviorally strongly correlated and associated with similar lesion patterns in right frontoinsular and striatal areas. In multiple regression models, reduced fractional anisotropy and lower tract volume of the right inferior fronto-occipital fasciculus were the strongest predictors for both disorders, over time. These results highlight a common origin of aprosodia and amusia, both arising from damage and disconnection of the right ventral auditory stream integrating rhythmic-melodic acoustic information in prosody and music. Comorbidity of these disabilities may worsen the prognosis and affect rehabilitation success.

A novel beamformer-based imaging of phase–amplitude coupling (BIPAC) unveiling the inter-regional connectivity of emotional prosody processing in women with primary dysmenorrhea

Objective. Neural communication or the interactions of brain regions play a key role in the formation of functional neural networks. A type of neural communication can be measured in the form of phase–amplitude coupling (PAC), which is the coupling between the phase of low-frequency oscillations and the amplitude of high-frequency oscillations. This paper presents a beamformer-based imaging method, beamformer-based imaging of PAC (BIPAC), to quantify the strength of PAC between a seed region and other brain regions. Approach. A dipole is used to model the ensemble of neural activity within a group of nearby neurons and represents a mixture of multiple source components of cortical activity. From ensemble activity at each brain location, the source component with the strongest coupling to the seed activity is extracted, while unrelated components are suppressed to enhance the sensitivity of coupled-source estimation. Main results. In evaluations using simulation data sets, BIPAC proved advantageous with regard to estimation accuracy in source localization, orientation, and coupling strength. BIPAC was also applied to the analysis of magnetoencephalographic signals recorded from women with primary dysmenorrhea in an implicit emotional prosody experiment. In response to negative emotional prosody, auditory areas revealed strong PAC with the ventral auditory stream and occipitoparietal areas in the theta–gamma and alpha–gamma bands, which may respectively indicate the recruitment of auditory sensory memory and attention reorientation. Moreover, patients with more severe pain experience appeared to have stronger coupling between auditory areas and temporoparietal regions. Significance. Our findings indicate that the implicit processing of emotional prosody is altered by menstrual pain experience. The proposed BIPAC is feasible and applicable to imaging inter-regional connectivity based on cross-frequency coupling estimates. The experimental results also demonstrate that BIPAC is capable of revealing autonomous brain processing and neurodynamics, which are more subtle than active and attended task-driven processing.

Neural bases of social communicative intentions in speech.

Our ability to understand others’ communicative intentions in speech is key to successful social interaction. Indeed, misunderstanding an ‘excuse me’ as apology, while meant as criticism, may have important consequences. Recent behavioural studies have provided evidence that prosody, that is, vocal tone, is an important indicator for speakers’ intentions. Using a novel audio-morphing paradigm, the present functional magnetic resonance imaging study examined the neurocognitive mechanisms that allow listeners to ‘read’ speakers’ intents from vocal prosodic patterns. Participants categorized prosodic expressions that gradually varied in their acoustics between criticism, doubt, and suggestion. Categorizing typical exemplars of the three intentions induced activations along the ventral auditory stream, complemented by amygdala and mentalizing system. These findings likely depict the stepwise conversion of external perceptual information into abstract prosodic categories and internal social semantic concepts, including the speaker’s mental state. Ambiguous tokens, in turn, involved cingulo-opercular areas known to assist decision-making in case of conflicting cues. Auditory and decision-making processes were flexibly coupled with the amygdala, depending on prosodic typicality, indicating enhanced categorization efficiency of overtly relevant, meaningful prosodic signals. Altogether, the results point to a model in which auditory prosodic categorization and socio-inferential conceptualization cooperate to translate perceived vocal tone into a coherent representation of the speaker’s intent.

Information Processing in the Auditory Ventral Stream

The auditory cortex in humans comprises multiple auditory fields organized hierarchically, similar to that in non-human primates. The ventral auditory stream of the macaque consists of several subdivisions on the supratemporal plane (STP) and the superior temporal gyrus (STG). There are two main axes (caudorostral and mediolateral) for processing auditory information in the STP and STG. Here, we review the neural basis of the integration of spectral and temporal auditory information along the two axes of the ventral auditory stream in the macaque.

P 39. Functional causality of the dorsal stream in sensorimotor integration of speech repetition

Acquisition of language skills is necessary for social interactions in humans, and speech repetition plays a fundamental role in its acquisition by mapping auditory speech input onto matching speech output. At the level of neuronal networks in the brain, speech repetition occurs within the auditory dorsal stream, which is constituted by a temporo-parieto-frontal network formed by the posterior part of superior temporal sulcus (pSTS), the posterior inferior frontal gyrus (pIFG), and the temporo-parietal junction (Tpj). However, it is less known which component of the dorsal stream plays an essential role in the modulation of speech repetition. To investigate the issue of causality of the dorsal stream in sensorimotor integration of speech repetition, we employed continuous theta-burst transcranial magnetic stimulation (cTBS) in order to disrupt neuronal activity in the stimulated areas. Nineteen right-handed German volunteers performed four behavioral speech tasks. (1) Word-picture matching test: subjects listened to a German noun and were asked to choose a picture which fit to the word heard from four different pictures; the target picture, pictures implying a phonemic or semantic error, and an unrelated picture. (2) Syllable repetition test: subjects listened to one of six different syllables and repeated the perceived syllable immediately. (3) Pseudo-word repetition test: subjects listened to meaningless pseudo-words and immediately repeated it. (4) Sentence repetition test: subjects listened to German sentences and immediately repeated it. In all the tests, error rates (ER) and reaction time (RT) were calculated. Inhibitory cTBS was applied over the individual activated regions of the left pIFG, Tpj, pSTS by using an fMRI-guided TMS neuronavigation system. Left middle occipital gyrus (MOG) was also stimulated as a control region to clarify the topographical specificity of cTBS effects because it is assumed that the MOG is concerned with visual processing. The behavioral speech tests were performed after cTBS over four sites mentioned above, and they were compared by repeated measures ANOVAs. (1) In word-picture matching test, cTBS of Tpj, pSTS and pIFG increased phonemic errors compared to cTBS of MOG. No other types of errors were affected by any sites of cTBS. (2) The syllable repetition ERs were significantly larger after cTBS of Tpj, pSTS and pIFG than after cTBS over MOG. (3) Pseudo-word repetition test demonstrated that ERs significantly increased after cTBS of Tpj and pSTS than after cTBS of MOG and pIFG. (4) In the sentence repetition test, no differences were observed in ERs between sites of cTBS. Throughout all the behavioral tests, RTs were not significantly different between sites of cTBS. Disruption of the dorsal stream led to increases of phonemic errors in word-picture matching test and to increases of errors of syllable and pseudo-word repetitions, demonstrating that the dorsal stream plays a crucial role in sensorimotor integration of phonological processing. The lack of cTBS effects on native sentences may be explained by the idea that the lexical/semantic material processing is done by the ventral auditory stream. The sensory phonological system performs critical modulation of subsequent articulation in the motor system via a sensorimotor interface, supporting evidence that the sensory system is situated hierarchically at higher level to modulate the motor articulation.

Decoding stimulus identity from multi-unit activity and local field potentials along the ventral auditory stream in the awake primate: implications for cortical neural prostheses

Objective. Hierarchical processing of auditory sensory information is believed to occur in two streams: a ventral stream responsible for stimulus identity and a dorsal stream responsible for processing spatial elements of a stimulus. The objective of the current study is to examine neural coding in this processing stream in the context of understanding the possibility for an auditory cortical neural prosthesis. Approach. We examined the selectivity for species-specific primate vocalizations in the ventral auditory processing stream by applying a statistical classifier to neural data recorded from microelectrode arrays. Multi-unit activity (MUA) and local field potential (LFP) data recorded simultaneously from primary auditory complex (AI) and rostral parabelt (PBr) were decoded on a trial-by-trial basis. Main results. While decode performance in AI was well above chance, mean performance in PBr did not deviate >15% from chance level. Mean performance levels were similar for MUA and LFP decodes. Increasing the spectral and temporal resolution improved decode performance; while inter-electrode spacing could be as large as 1.14 mm without degrading decode performance. Significance. These results serve as preliminary guidance for a human auditory cortical neural prosthesis; instructing interface implementation, microstimulation patterns and anatomical placement.

Differential representation of auditory categories between cell classes in primate auditory cortex

A comprehensive understanding of the neural mechanisms of cognitive function requires an understanding of how neural representations are transformed across different scales of neural organization: from within local microcircuits to across different brain areas. However, the neural transformations within the local microcircuits are poorly understood. Particularly, the role that two main cell classes of neurons in cortical microcircuits (i.e. pyramidal neurons and interneurons) have in auditory behaviour and cognition remains unknown. In this study, we tested the hypothesis that pyramidal cells and interneurons in the auditory cortex play a differential role in auditory categorization. To test this hypothesis, we recorded single-unit activity from the auditory cortex of rhesus monkeys while they categorized speech sounds. Based on the spike-waveform shape, a neuron was classified as either a narrow-spiking putative interneuron or a broad-spiking putative pyramidal neuron. We found that putative interneurons and pyramidal neurons in the auditory cortex differentially coded category information: interneurons were more selective for auditory categories than pyramidal neurons. These differences between cell classes may be an essential property of the neural computations underlying auditory categorization within the microcircuitry of the auditory cortex.

Neural Processing of What and Who Information in Speech

Human speech is composed of two types of information, related to content (lexical information, i.e., "what" is being said [e.g., words]) and to the speaker (indexical information, i.e., "who" is talking [e.g., voices]). The extent to which lexical versus indexical information is represented separately or integrally in the brain is unresolved. In the current experiment, we use short-term fMRI adaptation to address this issue. Participants performed a loudness judgment task during which single or multiple sets of words/pseudowords were repeated with single (repeat) or multiple talkers (speaker-change) conditions while BOLD responses were collected. As reflected by adaptation fMRI, the left posterior middle temporal gyrus, a crucial component of the ventral auditory stream performing sound-to-meaning computations ("what" pathway), showed sensitivity to lexical as well as indexical information. Previous studies have suggested that speaker information is abstracted during this stage of auditory word processing. Here, we demonstrate that indexical information is strongly coupled with word information. These findings are consistent with a plethora of behavioral results that have demonstrated that changes to speaker-related information can influence lexical processing.

Merging of Phonological and Gestural Circuits in Early Language Evolution

In the monkey, cortical auditory projections subdivide into a dorsal stream mostly involved in spatiotemporal processing, that projects mainly to dorsal frontal areas; and a ventral stream involved in stimulus identification, connected to the ventrolateral prefrontal cortex (VLPFC). We propose that in the human lineage, part of the dorsal auditory pathway has specialized in vocalization processing, enhancing vocal repetition and short-term memory capacities that are crucial for linguistic development. In the human, the vocalization-related dorsal auditory component tends to converge in the VLPFC with the ventral auditory stream and with projections involved in gestural control; and consists of a direct connection between the auditory cortex and the VLPFC via the arcuate fasciculus, and an indirect pathway via the supramarginal gyrus. Additionally, intraparietal and inferior parietal afferents to the VLPFC are associated with communicative hand gestures, with manipulation skills and with early tool-making. Although in general terms compatible with the mirror-neuron gestural hypothesis for language origins, this proposal underlines the participation of the dorsal auditory pathway in voice processing as a key event that marked the beginning of human phonology and the subsequent evolution of language. Instead, the mirror neuron system for gestures and the primitive vocalization network (ventral pathway) contributed to provide a communicative scaffolding that facilitated the emergence of human-like phonology. Furthermore, we emphasize the phylogenetic continuity (homology) between non-human and human vocalization and their neural substrates, something that is not usually stressed in the mirror neuron perspective.

Human Dorsal and Ventral Auditory Streams Subserve Rehearsal-Based and Echoic Processes during Verbal Working Memory

To hear a sequence of words and repeat them requires sensory-motor processing and something more-temporary storage. We investigated neural mechanisms of verbal memory by using fMRI and a task designed to tease apart perceptually based ("echoic") memory from phonological-articulatory memory. Sets of two- or three-word pairs were presented bimodally, followed by a cue indicating from which modality (auditory or visual) items were to be retrieved and rehearsed over a delay. Although delay-period activation in the planum temporale (PT) was insensible to the source modality and showed sustained delay-period activity, the superior temporal gyrus (STG) activated more vigorously when the retrieved items had arrived to the auditory modality and showed transient delay-period activity. Functional connectivity analysis revealed two topographically distinct fronto-temporal circuits, with STG co-activating more strongly with ventrolateral prefrontal cortex and PT co-activating more strongly with dorsolateral prefrontal cortex. These argue for separate contributions of ventral and dorsal auditory streams in verbal working memory.