Middle Superior Temporal Sulcus Research Articles

Neural decoding of emotional prosody in voice-sensitive auditory cortex predicts social communication abilities in children.

During social interactions, speakers signal information about their emotional state through their voice, which is known as emotional prosody. Little is known regarding the precise brain systems underlying emotional prosody decoding in children and whether accurate neural decoding of these vocal cues is linked to social skills. Here, we address critical gaps in the developmental literature by investigating neural representations of prosody and their links to behavior in children. Multivariate pattern analysis revealed that representations in the bilateral middle and posterior superior temporal sulcus (STS) divisions of voice-sensitive auditory cortex decode emotional prosody information in children. Crucially, emotional prosody decoding in middle STS was correlated with standardized measures of social communication abilities; more accurate decoding of prosody stimuli in the STS was predictive of greater social communication abilities in children. Moreover, social communication abilities were specifically related to decoding sadness, highlighting the importance of tuning in to negative emotional vocal cues for strengthening social responsiveness and functioning. Findings bridge an important theoretical gap by showing that the ability of the voice-sensitive cortex to detect emotional cues in speech is predictive of a child's social skills, including the ability to relate and interact with others.

The impact of stimulus size and orientation on individual face coding in monkey face-selective cortex

Face-selective neurons in the monkey temporal cortex discharge at different rates in response to pictures of different individual faces. Here we tested whether this pattern of response across single neurons in the face-selective area ML (located in the middle Superior Temporal Sulcus) tolerates two affine transformations; picture-plane inversion, known to decrease the average response of face-selective neurons and the other, stimulus size. We recorded the response of 57 ML neurons in two awake and fixating monkeys. Face stimuli were presented at two sizes (10 and 5 degrees of visual angle) and two orientations (upright and inverted). Different faces elicited distinct patterns of activity across ML neurons that were reliable (i.e., predictable with a classifier) within a specific size and orientation condition. Despite observing a reduction in the average response magnitude of face-selective neurons to inverted faces, compared to upright faces, classifier performance was above chance for both upright and inverted faces. While decoding was largely preserved across changes in stimulus size, a classifier trained with one orientation condition and tested on the other did not lead to performance above chance level. We conclude that different individual faces can be decoded from patterns of responses in the monkey area ML regardless of orientation or size, but with qualitatively different patterns of responses for upright and inverted faces.

Transformation of Visual Representations Across Ventral Stream Body-selective Patches.

Although the neural processing of visual images of bodies is critical for survival, it is much less well understood than face processing. Functional imaging studies demonstrated body selective regions in primate inferior temporal cortex. To advance our understanding of how the visual brain represents bodies, we compared the representation of animate and inanimate objects in two such body patches with fMRI-guided single unit recordings in rhesus monkeys. We found that the middle Superior Temporal Sulcus body patch (MSB) distinguishes to a greater extent bodies from non-bodies than the anterior Superior Temporal Sulcus body patch (ASB). Importantly, ASB carried more viewpoint-tolerant information about body posture and body identity than MSB, while MSB showed greater orientation selectivity. Combined with previous work on faces, this suggests that an increase in view-tolerant representations, coupled with a refined individuation, along the visual hierarchy is a general property of information processing within the inferior temporal cortex.

Functional organization of face processing in the human superior temporal sulcus: a 7T high-resolution fMRI study.

The superior temporal sulcus (STS) is a major component of the human face perception network, implicated in processing dynamic changeable aspects of faces. However, it remains unknown whether STS holds functionally segregated subdivisions for different categories of facial movements. We used high-resolution functional magnetic resonance imaging (fMRI) at 7T in 16 volunteers to compare STS activation with faces displaying angry or happy expressions, eye-gaze shifts and lip-speech movements. Combining univariate and multivariate analyses, we show a systematic topological organization within STS, with gaze-related activity predominating in the most posterior and superior sector, speech-related activity in the anterior sector and emotional expressions represented in the intermediate middle STS. Right STS appeared to hold a finer functional segregation between all four types of facial movements, and best discriminative abilities within the face-selective posterior STS (pSTS). Conversely, left STS showed greater overlap between conditions, with a lack of distinction between mouth movements associated to speech or happy expression and better discriminative abilities (for gaze and speech vs emotion conditions) outside pSTS. Differential sensitivity to upper (eye) or lower (mouth) facial features may contribute to, but does not appear to fully account for, these response patterns.

Shape Selectivity of Middle Superior Temporal Sulcus Body Patch Neurons.

Functional MRI studies in primates have demonstrated cortical regions that are strongly activated by visual images of bodies. The presence of such body patches in macaques allows characterization of the stimulus selectivity of their single neurons. Middle superior temporal sulcus body (MSB) patch neurons showed similar stimulus selectivity for natural, shaded, and textured images compared with their silhouettes, suggesting that shape is an important determinant of MSB responses. Here, we examined and modeled the shape selectivity of single MSB neurons. We measured the responses of single MSB neurons to a variety of shapes producing a wide range of responses. We used an adaptive stimulus sampling procedure, selecting and modifying shapes based on the responses of the neuron. Forty percent of shapes that produced the maximal response were rated by humans as animal-like, but the top shape of many MSB neurons was not judged as resembling a body. We fitted the shape selectivity of MSB neurons with a model that parameterizes shapes in terms of curvature and orientation of contour segments, with a pixel-based model, and with layers of units of convolutional neural networks (CNNs). The deep convolutional layers of CNNs provided the best goodness-of-fit, with a median explained explainable variance of the neurons’ responses of 77%. The goodness-of-fit increased along the convolutional layers’ hierarchy but was lower for the fully connected layers. Together with demonstrating the successful modeling of single unit shape selectivity with deep CNNs, the data suggest that semantic or category knowledge determines only slightly the single MSB neuron’s shape selectivity.

Auditory, Visual and Audiovisual Speech Processing Streams in Superior Temporal Sulcus.

The human superior temporal sulcus (STS) is responsive to visual and auditory information, including sounds and facial cues during speech recognition. We investigated the functional organization of STS with respect to modality-specific and multimodal speech representations. Twenty younger adult participants were instructed to perform an oddball detection task and were presented with auditory, visual, and audiovisual speech stimuli, as well as auditory and visual nonspeech control stimuli in a block fMRI design. Consistent with a hypothesized anterior-posterior processing gradient in STS, auditory, visual and audiovisual stimuli produced the largest BOLD effects in anterior, posterior and middle STS (mSTS), respectively, based on whole-brain, linear mixed effects and principal component analyses. Notably, the mSTS exhibited preferential responses to multisensory stimulation, as well as speech compared to nonspeech. Within the mid-posterior and mSTS regions, response preferences changed gradually from visual, to multisensory, to auditory moving posterior to anterior. Post hoc analysis of visual regions in the posterior STS revealed that a single subregion bordering the mSTS was insensitive to differences in low-level motion kinematics yet distinguished between visual speech and nonspeech based on multi-voxel activation patterns. These results suggest that auditory and visual speech representations are elaborated gradually within anterior and posterior processing streams, respectively, and may be integrated within the mSTS, which is sensitive to more abstract speech information within and across presentation modalities. The spatial organization of STS is consistent with processing streams that are hypothesized to synthesize perceptual speech representations from sensory signals that provide convergent information from visual and auditory modalities.

Stimulus features coded by single neurons of a macaque body category selective patch

Body category-selective regions of the primate temporal cortex respond to images of bodies, but it is unclear which fragments of such images drive single neurons' responses in these regions. Here we applied the Bubbles technique to the responses of single macaque middle superior temporal sulcus (midSTS) body patch neurons to reveal the image fragments the neurons respond to. We found that local image fragments such as extremities (limbs), curved boundaries, and parts of the torso drove the large majority of neurons. Bubbles revealed the whole body in only a few neurons. Neurons coded the features in a manner that was tolerant to translation and scale changes. Most image fragments were excitatory but for a few neurons both inhibitory and excitatory fragments (opponent coding) were present in the same image. The fragments we reveal here in the body patch with Bubbles differ from those suggested in previous studies of face-selective neurons in face patches. Together, our data indicate that the majority of body patch neurons respond to local image fragments that occur frequently, but not exclusively, in bodies, with a coding that is tolerant to translation and scale. Overall, the data suggest that the body category selectivity of the midSTS body patch depends more on the feature statistics of bodies (e.g., extensions occur more frequently in bodies) than on semantics (bodies as an abstract category).

Tolerance of macaque middle superior temporal sulcus body patch neurons to shape-preserving stimulus transformations.

Tolerance of macaque middle superior temporal sulcus body patch neurons to shape-preserving stimulus transformations.

Heterogeneous Single-Unit Selectivity in an fMRI-Defined Body-Selective Patch

Although the visual representation of bodies is essential for reproduction, survival, and social communication, little is known about the mechanisms of body recognition at the single neuron level. Imaging studies showed body-category selective regions in the primate occipitotemporal cortex, but it is difficult to infer the stimulus selectivities of the neurons from the population activity measured in these fMRI studies. To overcome this, we recorded single unit activity and local field potentials (LFPs) in the middle superior temporal sulcus body patch, defined by fMRI in the same rhesus monkeys. Both the spiking activity, averaged across single neurons, and LFP gamma power in this body patch was greater for bodies (including monkey bodies, human bodies, mammals, and birds) compared with other objects, which fits the fMRI activation. Single neurons responded to a small proportion of body images. Thus, the category selectivity at the population level resulted from averaging responses of a heterogeneous population of single units. Despite such strong within-category selectivity at the single unit level, two distinct clusters, bodies and nonbodies, were present when analyzing the responses at the population level, and a classifier that was trained using the responses to a subset of images was able to classify novel images of bodies with high accuracy. The body-patch neurons showed strong selectivity for individual body parts at different orientations. Overall, these data suggest that single units in the fMRI-defined body patch are biased to prefer bodies over nonbody objects, including faces, with a strong selectivity for individual body images.

Neural correlates of the encoding of multimodal contextual features

Functional magnetic resonance imaging (fMRI) was employed to identify neural regions engaged during the encoding of contextual features belonging to different modalities. Subjects studied objects that were presented to the left or right of fixation. Each object was paired with its name, spoken in either a male or a female voice. The test requirement was to discriminate studied from unstudied pictures and, for each picture judged old, to retrieve its study location and the gender of the voice that spoke its name. Study trials associated with accurate rather than inaccurate location memory demonstrated enhanced activity in the fusiform and parahippocampal cortex and the hippocampus and reduced activity (a negative subsequent memory effect) in the medial occipital cortex. Successful encoding of voice information was associated with enhanced study activity in the right middle superior temporal sulcus and activity reduction in the right superior frontal cortex. These findings support the proposal that encoding of a contextual feature is associated with enhanced activity in regions engaged during its online processing. In addition, they indicate that negative subsequent memory effects can also demonstrate feature-selectivity. Relative to other classes of study trials, trials for which both contextual features were later retrieved demonstrated enhanced activity in the lateral occipital complex and reduced activity in the temporo-parietal junction. These findings suggest that multifeatural encoding was facilitated when the study item was processed efficiently and study processing was not interrupted by redirection of attention toward extraneous events.

Dissimilar processing of emotional facial expressions in human and monkey temporal cortex.

Emotional facial expressions play an important role in social communication across primates. Despite major progress made in our understanding of categorical information processing such as for objects and faces, little is known, however, about how the primate brain evolved to process emotional cues. In this study, we used functional magnetic resonance imaging (fMRI) to compare the processing of emotional facial expressions between monkeys and humans. We used a 2×2×2 factorial design with species (human and monkey), expression (fear and chewing) and configuration (intact versus scrambled) as factors. At the whole brain level, neural responses to conspecific emotional expressions were anatomically confined to the superior temporal sulcus (STS) in humans. Within the human STS, we found functional subdivisions with a face-selective right posterior STS area that also responded to emotional expressions of other species and a more anterior area in the right middle STS that responded specifically to human emotions. Hence, we argue that the latter region does not show a mere emotion-dependent modulation of activity but is primarily driven by human emotional facial expressions. Conversely, in monkeys, emotional responses appeared in earlier visual cortex and outside face-selective regions in inferior temporal cortex that responded also to multiple visual categories. Within monkey IT, we also found areas that were more responsive to conspecific than to non-conspecific emotional expressions but these responses were not as specific as in human middle STS. Overall, our results indicate that human STS may have developed unique properties to deal with social cues such as emotional expressions.

Stimulus representations in body-selective regions of the macaque cortex assessed with event-related fMRI

Functional imaging studies in humans and monkeys have shown category-selective regions in the temporal cortex, in particular for faces and bodies. Although the body-selective regions have been well studied in humans, little is understood about the functional properties of such regions in macaques. To address this, we first mapped body-selective activations in the visual cortex of four rhesus monkeys in a block design fMRI study and identified two regions in the middle and anterior Superior Temporal Sulcus (STS) that were more strongly activated by monkey bodies compared to well-controlled manmade objects. These two regions partially overlapped with regions that were more activated by faces than manmade objects. Secondly, using an event-related, single image fMRI design we measured the activations to 200 images of 10 stimulus classes (monkey bodies, human bodies, mammals, birds, monkey faces, human faces, body-like sculptures, fruits/vegetables, and two sets of control objects). Multivoxel-pattern analyses showed that both body-selective regions primarily distinguished faces from other inanimate and animate objects, including bodies. Another distinction was present between inanimate objects and bodies in the middle STS body region. The category-based clustering was less pronounced in the anterior compared to the middle STS body-selective regions. In addition, both body-selective regions showed further selectivity for different "subclasses" of the broad body category such as monkeys, human, mammals and birds. Overall, these data indicate strong spatial clustering of animate categories in the macaque STS with a surprisingly marked distinction between faces and bodies within body-selective regions which was stronger than between manmade objects and bodies.

Altered Intra- and Inter-Regional Synchronization of Superior Temporal Cortex in Deaf People

Functional organization of the brain can be fundamentally altered by auditory deprivation. Previous studies found that the superior temporal cortex in deaf people is reorganized to process non-auditory stimuli, as revealed by the extrinsic task-induced brain activities. However, it is unknown how the intrinsic activities of this region are impacted by deafness. This study explored this issue using resting-state functional magnetic resonance imaging. We examined 60 congenitally deaf (CD) individuals, 39 acquired deaf (AD) individuals, and 38 hearing controls (HC), and focused on the effect of deafness on the intra- and inter-regional synchronization of different parts of superior temporal sulcus (STS). We found that intra-regional synchronization or regional homogeneity (ReHo) of the middle STS (mSTS) was decreased in AD compared with HC or CD, while the CD had preserved ReHo in mSTS. Greater connectivity was observed between mSTS and posterior STS in CD and HC than in AD, while both CD and AD had weaker connectivity of mSTS with the anterior STS (aSTS) compared with HC. Moreover, the connectivity of mSTS-aSTS in CD and AD was associated with their language skills. These findings confirmed our hypothesis that the intrinsic function of different parts of STS is distinctly impacted by deafness.

Single cell stimulus selectivities in the macaque middle superior temporal sulcus body patch, defined by fMRI

Event Abstract Back to Event Single cell stimulus selectivities in the macaque middle superior temporal sulcus body patch, defined by fMRI Ivo D. Popivanov1*, Jan Jastorff1, Wim Vanduffel1, 2, 3 and Rufin Vogels1 1 KU Leuven, Belgium 2 MGH Martinos Center, United States 3 Harvard Medical School, United States Single-cell and fMRI studies in non-human primates have shown responses to faces and body parts in the inferotemporal cortex. In the present study we recorded single-cell spiking and local field potential (LFP) activity in and surrounding an fMRI-defined body patch to determine the selectivity for animate and inanimate objects. The stimuli were 100 monochrome images, grouped in 10 classes (10 images each): headless monkey and human bodies, 2 sets of control objects (matched for each of the body classes), monkey and human faces, 4-legged mammals, birds, fruits and sculptures (by H. Moore). We defined two main body responsive patches in the superior temporal sulcus (STS) by fMRI. Recordings focused on the posterior one, located at 2-3 mm anterior to the auditory mediatus. Stimuli were shown for 200 ms during a fixation task. Stimulus position was adapted to the receptive field of the neuron. Single neurons in the body patch responded on average significantly stronger to images of bodies (monkey, human, mammal and bird) than to images of other stimulus classes (including faces). Nevertheless, all neurons showed a large difference in responses to different exemplars of the body classes (high within-class selectivity). Cluster analysis of the neural distances between the images based on net spiking responses and high gamma LFP power revealed two main clusters -all bodies (monkeys, humans, mammals, and birds) vs. all other stimuli (mainly faces, objects, fruits). The sculptures were found equally often in both clusters. Lateral to the body patch at the same A/P level, the neurons responded equally strong to bodies and faces, but less to the images of other classes. Cluster analysis based on the responses of those neurons showed 3 main clusters: bodies vs. Faces vs. other inanimate stimuli. In addition, most neurons showed strong selectivity for the orientation of the image. Keywords: Vision, temporal cortex, bodies, Categories, Rhesus Monkey Conference: Belgian Brain Council, Liège, Belgium, 27 Oct - 27 Oct, 2012. Presentation Type: Poster Presentation Topic: Higher Brain Functions in health and disease: cognition and memory Citation: Popivanov ID, Jastorff J, Vanduffel W and Vogels R (2012). Single cell stimulus selectivities in the macaque middle superior temporal sulcus body patch, defined by fMRI. Conference Abstract: Belgian Brain Council. doi: 10.3389/conf.fnhum.2012.210.00095 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 22 Aug 2012; Published Online: 12 Sep 2012. * Correspondence: Mr. Ivo D Popivanov, KU Leuven, Leuven, 3000, Belgium, ivo.popivanov@med.kuleuven.be Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Ivo D Popivanov Jan Jastorff Wim Vanduffel Rufin Vogels Google Ivo D Popivanov Jan Jastorff Wim Vanduffel Rufin Vogels Google Scholar Ivo D Popivanov Jan Jastorff Wim Vanduffel Rufin Vogels PubMed Ivo D Popivanov Jan Jastorff Wim Vanduffel Rufin Vogels Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

A Head View-Invariant Representation of Gaze Direction in Anterior Superior Temporal Sulcus

SummaryHumans show a remarkable ability to discriminate others' gaze direction, even though a given direction can be conveyed by many physically dissimilar configurations of different eye positions and head views. For example, eye contact can be signaled by a rightward glance in a left-turned head or by direct gaze in a front-facing head. Such acute gaze discrimination implies considerable perceptual invariance. Previous human research found that superior temporal sulcus (STS) responds preferentially to gaze shifts [1], but the underlying representation that supports such general responsiveness remains poorly understood. Using multivariate pattern analysis (MVPA) of human functional magnetic resonance imaging (fMRI) data, we tested whether STS contains a higher-order, head view-invariant code for gaze direction. The results revealed a finely graded gaze direction code in right anterior STS that was invariant to head view and physical image features. Further analyses revealed similar gaze effects in left anterior STS and precuneus. Our results suggest that anterior STS codes the direction of another's attention regardless of how this information is conveyed and demonstrate how high-level face areas carry out fine-grained, perceptually relevant discrimination through invariance to other face features.

Cortical processing of head- and eye-gaze cues guiding joint social attention

Previous fMRI experiments showed an involvement of the STS in the processing of eye-gaze direction in joint attention. Since head-gaze direction can also be used for the assessment of another person's attentional focus, we compared the mechanisms underlying the processing of head- and eye-gaze direction using a combined psychophysical and fMRI approach. Subjects actively followed the head- or eye-gaze direction of a person in a photograph towards one of seven possible targets by moving their eyes. We showed that the right posterior superior temporal sulcus (STS) as well as the right fusiform gyrus (FSG) were involved in both processing of head- as well as eye-gaze direction. Another finding was a bilateral deactivation of a distinct area in the middle STS (mSTS) as well as the left anterior STS (aSTS), that was stronger when subjects followed eye-gaze direction than when they followed head-gaze direction. We assume that this deactivation is based on an active suppression of information arising from the distracting other directional cue, i.e. head-gaze direction in the eye-gaze direction task and eye-gaze direction in the head-gaze direction task.These results further support the hypothesis that the human equivalent of the gaze sensitive area in monkeys lies in more anterior parts of the STS than previously thought.

Synchrony of audio–visual speech stimuli modulates left superior temporal sulcus

The superior temporal sulcus has been suggested to play a significant role in the integration of auditory and visual sensory information. Here, we presented vowels and short video clips of the corresponding articulatory gestures to healthy adult humans with two auditory-visual stimulus intervals during sparse sampling 3-T functional magnetic resonance imaging to detect which brain areas are sensitive to synchrony of speech sounds and associated articulatory gestures. The upper bank of the left middle superior temporal sulcus showed stronger activation during naturally asynchronous stimulation than during simultaneous stimulus presentation. It is possible that this reflects sensitivity of the left middle superior temporal sulcus to temporal synchrony of audio-visual speech stimuli.

Specialization along the Left Superior Temporal Sulcus for Auditory Categorization

The affinity and temporal course of functional fields in middle and posterior superior temporal cortex for the categorization of complex sounds was examined using functional magnetic resonance imaging (fMRI) and event-related potentials (ERPs) recorded simultaneously. Data were compared before and after subjects were trained to categorize a continuum of unfamiliar nonphonemic auditory patterns with speech-like properties (NP) and a continuum of familiar phonemic patterns (P). fMRI activation for NP increased after training in left posterior superior temporal sulcus (pSTS). The ERP P2 response to NP also increased with training, and its scalp topography was consistent with left posterior superior temporal generators. In contrast, the left middle superior temporal sulcus (mSTS) showed fMRI activation only for P, and this response was not affected by training. The P2 response to P was also independent of training, and its estimated source was more anterior in left superior temporal cortex. Results are consistent with a role for left pSTS in short-term representation of relevant sound features that provide the basis for identifying newly acquired sound categories. Categorization of highly familiar phonemic patterns is mediated by long-term representations in left mSTS. Results provide new insight regarding the function of ventral and dorsal auditory streams.

Facial expressions of pain modulate observer's long‐latency responses in superior temporal sulcus

The strength of brain responses to others' pain has been shown to depend on the intensity of the observed pain. To investigate the temporal profile of such modulation, we recorded neuromagnetic brain responses of healthy subjects to facial expressions of pain. The subjects observed grayscale photos of the faces of genuine chronic pain patients when the patients were suffering from their ordinary pain (Chronic) and when the patients' pain was transiently intensified (Provoked). The cortical activation sequence during observation of the facial expressions of pain advanced from occipital to temporo-occipital areas, and it differed between Provoked and Chronic pain expressions in the right middle superior temporal sulcus (STS) at 300-500 ms: the responses were about a third stronger for Provoked than Chronic pain faces. Furthermore, the responses to Provoked pain faces were about 40% stronger in the right than the left STS, and they decreased from the first to the second measurement session by one-fourth, whereas no similar decrease in responses was found for Chronic pain faces. Thus, the STS responses to the pain expressions were modulated by the intensity of the observed pain and by stimulus repetition; the location and latency of the responses suggest close similarities between processing of pain and other affective facial expressions.

Phonological repetition-suppression in bilateral superior temporal sulci

Evidence has accumulated that posterior superior temporal sulcus (STS) is critically involved in phonological processing during speech perception, although there are conflicting accounts regarding the degree of lateralization. The current fMRI experiment aimed to identify phonological processing during speech perception through repetition-suppression effects. Repetition-suppression occurs when brain activity decreases from repetitive presentation of stimulus characteristics, in regions of cortex that process those characteristics. We manipulated the degree of phonological repetition among words in short lists to obtain systematic decreases in brain response, indicative of phonological processing. The fMRI experiment presented seventeen participants with recorded wordlists, of low, medium, or high phonological repetition, defined by how many phonemes were shared among words. Bilaterally, middle STS demonstrated activity differences consistent with our prediction of repetition-suppression, as responses decreased systematically with each increase in phonological repetition. Phonological repetition-suppression in bilateral STS converges with neuroimaging evidence for phonological processing, and word deafness resulting from bilateral superior temporal lesions.