Representational Similarity Analysis Research Articles

Lingering Neural Representations of Past Task Features Adversely Affect Future Behavior.

During goal-directed behavior, humans purportedly form and retrieve so-called event files, conjunctive representations that link context-specific information about stimuli, their associated actions, and the expected action outcomes. The automatic formation, and later retrieval, of such conjunctive representations can substantially facilitate efficient action selection. However, recent behavioral work suggests that these event files may also adversely affect future behavior, especially when action requirements have changed between successive instances of the same task context (e.g., during task switching). Here, we directly tested this hypothesis with a recently developed method for measuring the strength of the neural representations of context-specific stimulus-action conjunctions (i.e., event files). Thirty-five male and female adult humans performed a task switching paradigm while undergoing EEG recordings. Replicating previous behavioral work, we found that changes in action requirements between two spaced repetitions of the same task incurred a significant reaction time cost. By combining multivariate pattern analysis and representational similarity analysis of the EEG recordings with linear mixed-effects modeling of trial-to-trial behavior, we then found that the magnitude of this behavioral cost was directly proportional to the strength of the conjunctive representation formed during the most recent previous exposure to the same task, that is, the most recent event file. This confirms that the formation of conjunctive representations of specific task contexts, stimuli, and actions in the brain can indeed adversely affect future behavior. Moreover, these findings demonstrate the potential of neural decoding of complex task set representations toward the prediction of behavior beyond the current trial.SIGNIFICANCE STATEMENT Understanding how the human brain organizes individual components of complex tasks is paramount for understanding higher-order cognition. During complex tasks, the brain forms conjunctive representations that link individual task features (contexts, stimuli, actions), which aids future performance of the same task. However, this can have adverse effects when the required sequence of actions within a task changes. We decoded conjunctive representations from electroencephalographic recordings during a task that included frequent changes to the rules determining the response. Indeed, stronger initial conjunctive representations predicted significant future response-time costs when task contexts repeated with changed response requirements. Showing that the formation of conjunctive task representations can have negative future effects generates novel insights into complex behavior and cognition, including task switching, planning, and problem solving.

Long-term changes in cortical representation through perceptual learning of spectrally degraded speech.

Listeners can adapt to acoustically degraded speech with perceptual training. The learning processes for long periods underlies the rehabilitation of patients with hearing aids or cochlear implants. Perceptual learning of acoustically degraded speech has been associated with the frontotemporal cortices. However, neural processes during and after long-term perceptual learning remain unclear. Here we conducted perceptual training of noise-vocoded speech sounds (NVSS), which is spectrally degraded signals, and measured the cortical activity for seven days and the follow up testing (approximately 1year later) to investigate changes in neural activation patterns using functional magnetic resonance imaging. We demonstrated that young adult participants (n = 5) improved their performance across seven experimental days, and the gains were maintained after 10months or more. Representational similarity analysis showed that the neural activation patterns of NVSS relative to clear speech in the left posterior superior temporal sulcus (pSTS) were significantly different across seven training days, accompanying neural changes in frontal cortices. In addition, the distinct activation patterns to NVSS in the frontotemporal cortices were also observed 10-13months after the training. We, therefore, propose that perceptual training can induce plastic changes and long-term effects on neural representations of the trained degraded speech in the frontotemporal cortices. These behavioral improvements and neural changes induced by the perceptual learning of degraded speech will provide insights into cortical mechanisms underlying adaptive processes in difficult listening situations and long-term rehabilitation of auditory disorders.

Post-encoding Reactivation Is Related to Learning of Episodes in Humans.

Prior animal and human studies have shown that post-encoding reinstatement plays an important role in organizing the temporal sequence of unfolding episodes in memory. Here, we investigated whether post-encoding reinstatement serves to promote the encoding of "one-shot" episodic learning beyond the temporal structure in humans. In Experiment 1, participants encoded sequences of pictures depicting unique and meaningful episodic-like events. We used representational similarity analysis on scalp EEG recordings during encoding and found evidence of rapid picture-elicited EEG pattern reinstatement at episodic offset (around 500 msec post-episode). Memory reinstatement was not observed between successive elements within an episode, and the degree of memory reinstatement at episodic offset predicted later recall for that episode. In Experiment 2, participants encoded a shuffled version of the picture sequences from Experiment 1, rendering each episode meaningless to the participant but temporally structured as in Experiment 1, and we found no evidence of memory reinstatement at episodic offset. These results suggest that post-encoding memory reinstatement is akin to the rapid formation of unique and meaningful episodes that unfold over time.

Context effects on object recognition in real-world environments: A study protocol.

Background: The environments that we live in impact on our ability to recognise objects, with recognition being facilitated when objects appear in expected locations (congruent) compared to unexpected locations (incongruent). However, these findings are based on experiments where the object is isolated from its environment. Moreover, it is not clear which components of the recognition process are impacted by the environment. In this experiment, we seek to examine the impact real world environments have on object recognition. Specifically, we will use mobile electroencephalography (mEEG) and augmented reality (AR) to investigate how the visual and semantic processing aspects of object recognition are changed by the environment. Methods: We will use AR to place congruent and incongruent virtual objects around indoor and outdoor environments. During the experiment a total of 34 participants will walk around the environments and find these objects while we record their eye movements and neural signals. We will perform two primary analyses. First, we will analyse the event-related potential (ERP) data using paired samples t-tests in the N300/400 time windows in an attempt to replicate congruency effects on the N300/400. Second, we will use representational similarity analysis (RSA) and computational models of vision and semantics to determine how visual and semantic processes are changed by congruency. Conclusions: Based on previous literature, we hypothesise that scene-object congruence would facilitate object recognition. For ERPs, we predict a congruency effect in the N300/N400, and for RSA we predict that higher level visual and semantic information will be represented earlier for congruent scenes than incongruent scenes. By collecting mEEG data while participants are exploring a real-world environment, we will be able to determine the impact of a natural context on object recognition, and the different processing stages of object recognition.

Neural representations of self-generated thought during think-aloud fMRI

Is the brain at rest during the so-called resting state? Ongoing experiences in the resting state vary in unobserved and uncontrolled ways across time, individuals, and populations. However, the role of self-generated thoughts in resting-state fMRI remains largely unexplored. In this study, we collected real-time self-generated thoughts during “resting-state” fMRI scans via the think-aloud method (i.e., think-aloud fMRI), which required participants to report whatever they were currently thinking. We first investigated brain activation patterns during a think-aloud condition and found that significantly activated brain areas included all brain regions required for speech. We then calculated the relationship between divergence in thought content and brain activation during think-aloud and found that divergence in thought content was associated with many brain regions. Finally, we explored the neural representation of self-generated thoughts by performing representational similarity analysis (RSA) at three neural scales: a voxel-wise whole-brain searchlight level, a region-level whole-brain analysis using the Schaefer 400-parcels, and at the systems level using the Yeo seven-networks. We found that “resting-state” self-generated thoughts were distributed across a wide range of brain regions involving all seven Yeo networks. This study highlights the value of considering ongoing experiences during resting-state fMRI and providing preliminary methodological support for think-aloud fMRI.

Perceptions of space and time of public transport travel associated with human brain activities: A case study of bus travel in Beijing

Understanding human perceptions of public transport (PT) travel is essential for improving PT provision/operation and the travel experiences of PT users, as well as for encouraging more people to use green and sustainable travel to reduce the congestion, air pollution, and energy costs that many urban systems are facing. Based on behavioral experiments and surveys, existing research has revealed that the spatial and temporal characteristics of PT travel greatly influence human perceptions; however, neuroimaging evidence confirming these findings remains lacking. In this study, we conducted a functional magnetic resonance imaging- (fMRI-) based PT travel and recall experiment with 9 participants and collected 651 unique bus travel trajectories in Beijing City over 21 days. We extracted 22 spatial and temporal characteristics and derived brain activity patterns for each PT travel route. A representational similarity analysis (RSA) was performed to quantify the relationship between these spatial and temporal characteristics and brain activity patterns. Verbal descriptions were collected to cross-check the brain imaging results. We show that the participants' brain activity patterns were significantly correlated with seven spatial (i.e., circuity, turn, angular deviation, slope, point of interest (POI), and land use) and two temporal (i.e., travel time and waiting time) features, although individuals' perceptions of PT travel differed substantially. These findings are in line with the participants' verbal reports. Our findings offer new neuroimaging evidence from an interdisciplinary perspective on the perception of the spatial and temporal characteristics of PT travel, which not only lays an empirical basis but also provides a “neurourbanism” approach to improve travelers' subjective well-being during PT travel.

Representations and decodability of diverse cognitive functions are preserved across the human cortex, cerebellum, and subcortex

Which part of the brain contributes to our complex cognitive processes? Studies have revealed contributions of the cerebellum and subcortex to higher-order cognitive functions; however, it has been unclear whether such functional representations are preserved across the cortex, cerebellum, and subcortex. In this study, we use functional magnetic resonance imaging data with 103 cognitive tasks and construct three voxel-wise encoding and decoding models independently using cortical, cerebellar, and subcortical voxels. Representational similarity analysis reveals that the structure of task representations is preserved across the three brain parts. Principal component analysis visualizes distinct organizations of abstract cognitive functions in each part of the cerebellum and subcortex. More than 90% of the cognitive tasks are decodable from the cerebellum and subcortical activities, even for the novel tasks not included in model training. Furthermore, we show that the cerebellum and subcortex have sufficient information to reconstruct activity in the cerebral cortex.

Integrative interaction of emotional speech in audio-visual modality.

Emotional clues are always expressed in many ways in our daily life, and the emotional information we receive is often represented by multiple modalities. Successful social interactions require a combination of multisensory cues to accurately determine the emotion of others. The integration mechanism of multimodal emotional information has been widely investigated. Different brain activity measurement methods were used to determine the location of brain regions involved in the audio-visual integration of emotional information, mainly in the bilateral superior temporal regions. However, the methods adopted in these studies are relatively simple, and the materials of the study rarely contain speech information. The integration mechanism of emotional speech in the human brain still needs further examinations. In this paper, a functional magnetic resonance imaging (fMRI) study was conducted using event-related design to explore the audio-visual integration mechanism of emotional speech in the human brain by using dynamic facial expressions and emotional speech to express emotions of different valences. Representational similarity analysis (RSA) based on regions of interest (ROIs), whole brain searchlight analysis, modality conjunction analysis and supra-additive analysis were used to analyze and verify the role of relevant brain regions. Meanwhile, a weighted RSA method was used to evaluate the contributions of each candidate model in the best fitted model of ROIs. The results showed that only the left insula was detected by all methods, suggesting that the left insula played an important role in the audio-visual integration of emotional speech. Whole brain searchlight analysis, modality conjunction analysis and supra-additive analysis together revealed that the bilateral middle temporal gyrus (MTG), right inferior parietal lobule and bilateral precuneus might be involved in the audio-visual integration of emotional speech from other aspects.

Correspondence between cognitive and neural representations for phonology, orthography, and semantics in supramarginal compared to angular gyrus.

The angular and supramarginal gyri (AG and SMG) together constitute the inferior parietal lobule (IPL) and have been associated with cognitive functions that support reading. How those functions are distributed across the AG and SMG is a matter of debate, the resolution of which is hampered by inconsistencies across stereotactic atlases provided by the major brain image analysis software packages. Schematic results from automated meta-analyses suggest primarily semantic (word meaning) processing in the left AG, with more spatial overlap among phonological (auditory word form), orthographic (visual word form), and semantic processing in the left SMG. To systematically test for correspondence between patterns of neural activation and phonological, orthographic, and semantic representations, we re-analyze a functional magnetic resonance imaging data set of participants reading aloud 465 words. Using representational similarity analysis, we test the hypothesis that within cytoarchitecture-defined subregions of the IPL, phonological representations are primarily associated with the SMG, while semantic representations are primarily associated with the AG. To the extent that orthographic representations can be de-correlated from phonological representations, they will be associated with cortex peripheral to the IPL, such as the intraparietal sulcus. Results largely confirmed these hypotheses, with some nuanced exceptions, which we discuss in terms of neurally inspired computational cognitive models of reading that learn mappings among distributed representations for orthography, phonology, and semantics. De-correlating constituent representations making up complex cognitive processes, such as reading, by careful selection of stimuli, representational formats, and analysis techniques, are promising approaches for bringing additional clarity to brain structure-function relationships.

Spatio-temporal theta pattern dissimilarity in the right centro-parietal area during memory generalization

Generalization across past events may guide our action in novel situations. Although generalization is a fundamental memory process, its neural underpinnings are not fully understood yet. In the present experiment, we combinedElectroencephalography(EEG) with multivariate representational similarity analysis (RSA) to examine in particular the role of spatio-temporal patterns of theta oscillations known to be important for associative memory processes, in memory generalization. We recorded EEG while healthy participants (n = 56) performed an acquired equivalence task. In this task, participants first acquired multiple associations among antecedent and consequent stimuli before they were required to transfer the acquired knowledge to novel stimulus pairs, thus probing memory generalization. Our behavioural data indicated that participants learned the initial associations well and transferred these associations successfully to novel test stimuli, demonstrating successful memory generalization. Our neural data revealed that, compared to mere memory retrieval, generalization was associated with significantly increased pattern dissimilarity of theta activity in the right centro-parietal area (electrodes P4 and P6). This pattern was specific to theta oscillations and not observed in other frequency bands. Our findings suggest an important role of theta oscillations in memory generalization, potentially serving the reactivation and integration of distinct events that enable the generalization across experiences.

Neural defensive circuits underlie helping under threat in humans.

Empathy for others' distress has long been considered the driving force of helping. However, when deciding to help others in danger, one must consider not only their distress, but also the risk to oneself. Whereas the role of self-defense in helping has been overlooked in human research, studies in other animals indicate defensive responses are necessary for the protection of conspecifics. In this pre-registered study (N=49), we demonstrate that human defensive neural circuits are implicated in helping others under threat. Participants underwent fMRI scanning while deciding whether to help another participant avoid aversive electrical shocks, at the risk of also being shocked. We found that higher engagement of neural circuits that coordinate fast escape from self-directed danger (including the insula, PAG, and ACC) facilitated decisions to help others. Importantly, using representational similarity analysis, we found that the strength with which the amygdala and insula uniquely represented the threat to oneself (and not the other's distress) predicted helping. Our findings indicate that in humans, as other mammals, defensive mechanisms play a greater role in helping behavior than previously understood.

Neural encoding of novel social networks: evidence that perceivers prioritize others' centrality.

Knowledge of someone's friendships can powerfully impact how one interacts with them. Previous research suggests that information about others' real-world social network positions-e.g. how well-connected they are (centrality), 'degrees of separation' (relative social distance)-is spontaneously encoded when encountering familiar individuals. However, many types of information covary with where someone sits in a social network. For instance, strangers' face-based trait impressions are associated with their social network centrality, and social distance and centrality are inherently intertwined with familiarity, interpersonal similarity and memories. To disentangle the encoding of the social network position from other social information, participants learned a novel social network in which the social network position was decoupled from other factors and then saw each person's image during functional magnetic resonance imaging scanning. Using representational similarity analysis, we found that social network centrality was robustly encoded in regions associated with visual attention and mentalizing. Thus, even when considering a social network in which one is not included and where centrality is unlinked from perceptual and experience-based features to which it is inextricably tied in naturalistic contexts, the brain encodes information about others' importance in that network, likely shaping future perceptions of and interactions with those individuals.

Imitation encourages empathic capacity toward other individuals with physical disabilities

Many people have difficulty empathizing with others who have dissimilar characteristics, such as physical disabilities. We hypothesized that people with no disabilities imitating the movements of individuals with disabilities could improve the empathic capacity toward their difficulties. To evaluate this hypothesis, we used functional magnetic resonance imaging to measure the neural activity patterns of 26 healthy participants while they felt the difficulties of individuals with hemiplegia by adopting their perspective. The participants initially either imitated or observed hemiplegic hand movements shown in video clips. Subsequently, the videos were rewatched and their difficulties were rated. Analysis of the subjective rating scores indicated that after imitating the hemiplegic movements, the participants felt into the difficulties of hemiplegia better than if they simply observed them. The cross-validation approach of multivoxel pattern analyses demonstrated that the information regarding the effect of imitation on empathizing with the difficulties was represented in specific activation patterns of brain regions involved in the mirror neuron system and cognitive empathy by comparing to other conditions that did not contain the information. The cross-classification approach detected distinct activation patterns in the brain regions involved in affective and cognitive empathy, commonly while imitating the hemiplegic movements and subsequently feeling them. This indicated that the common representation related to these two types of empathy existed between imitating and feeling the hemiplegic movements. Furthermore, representational similarity analysis revealed that activity patterns in the anterior cingulate cortex linked to affective empathy tuned to the subjective assessment of hemiplegic movements. Our findings indicate that imitating the movements of individuals with hemiplegia triggered the affective empathic response and improved the cognitive empathic response toward them. The affective empathic response also linked the subjective assessment to the difficulties of hemiplegia, which was especially modulated by the experience of imitation. Imitating the movements of individuals with disabilities likely encourages empathic capacity from both affective and cognitive aspects, resulting in people with no disabilities precisely feeling what they are feeling.

Morphological similarity of amygdala-ventral prefrontal pathways represents trait anxiety in younger and older adults

Stronger amygdala-ventral prefrontal white matter connectivity has been associated with lower trait anxiety, possibly reflecting an increased capacity for efficient communication between the two regions. However, there are also reports arguing against this brain-anxiety association. To address these inconsistencies in the literature, we tested the possibility that idiosyncratic tract morphology may account for meaningful individual differences in trait anxiety, even among those with comparable microstructural integrity. Here, we adopted intersubject representational similarity analysis, an analytic framework that captures multivariate patterns of similarity, to analyze the morphological similarity of amygdala-ventral prefrontal pathways. Data drawn from the Leipzig Study for Mind-Body-Emotion Interactions dataset showed that younger adults (20 to 35 y of age) with low trait anxiety, in contrast to trait-anxious individuals, had consistently similar morphological configurations in their left amygdala-ventral prefrontal pathways. Additional tests on an independent sample of older adults (60 to 75 y of age) validated this finding. Our study reveals a generalizable pattern of brain-anxiety association that is embedded within the shared geometries between fiber tract morphology and trait anxiety data.

Intact finger representation within primary sensorimotor cortex of musician's dystonia.

Musician's dystonia presents with a persistent deterioration of motor control during musical performance. A predominant hypothesis has been that this is underpinned by maladaptive neural changes to the somatotopic organization of finger representations within primary somatosensory cortex. Here, we tested this hypothesis by investigating the finger-specific activity patterns in the primary somatosensory and motor cortex using functional MRI and multivariate pattern analysis in nine musicians with dystonia and nine healthy musicians. A purpose-built keyboard device allowed characterization of activity patterns elicited during passive extension and active finger presses of individual fingers. We analysed the data using both traditional spatial analysis and state-of-the art multivariate analyses. Our analysis reveals that digit representations in musicians were poorly captured by spatial analyses. An optimized spatial metric found clear somatotopy but no difference in the spatial geometry between fingers with dystonia. Representational similarity analysis was confirmed as a more reliable technique than all spatial metrics evaluated. Significantly, the dissimilarity architecture was equivalent for musicians with and without dystonia. No expansion or spatial shift of digit representation maps were found in the symptomatic group. Our results therefore indicate that the neural representation of generic finger maps in primary sensorimotor cortex is intact in musician's dystonia. These results speak against the idea that task-specific dystonia is associated with a distorted hand somatotopy and lend weight to an alternative hypothesis that task-specific dystonia is due to a higher-order disruption of skill encoding. Such a formulation can better explain the task-specific deficit and offers alternative inroads for therapeutic interventions.

Early life stress is associated with the default mode and fronto-limbic network connectivity among young adults.

Exposure to early life stress (ELS) is associated with a variety of detrimental psychological and neurodevelopmental effects. Importantly, ELS has been associated with regional alterations and aberrant connectivity in the structure and functioning of brain regions involved in emotion processing and self-regulation, creating vulnerability to mental health problems. However, longitudinal research regarding the impact of ELS on functional connectivity between brain regions in the default mode network (DMN) and fronto-limbic network (FLN), both implicated in emotion-related processes, is relatively scarce. Neuroimaging research on ELS has mostly focused on single nodes or bi-nodal connectivity instead of functional networks. We examined how ELS is associated with connectivity patterns within the DMN and FLN during rest in early adulthood. The participants (n = 86; 47 females) in the current functional magnetic resonance imaging (fMRI) study were young adults (18–21 years old) whose families had participated in a longitudinal study since pregnancy. ELS was assessed both prospectively (parental reports of family relationship problems and mental health problems during pregnancy and infancy) and retrospectively (self-reported adverse childhood experiences). Inter-subject representational similarity analysis (IS-RSA) and multivariate distance matrix regression (MDMR) were used to analyze the association between ELS and the chosen networks. The IS-RSA results suggested that prospective ELS was associated with complex alterations within the DMN, and that retrospective ELS was associated with alterations in the FLN. MDMR results, in turn, suggested that that retrospective ELS was associated with DMN connectivity. Mean connectivity of the DMN was also associated with retrospective ELS. Analyses further showed that ELS-related alterations in the FLN were associated with increased connectivity between the prefrontal and limbic regions, and between different prefrontal regions. These results suggest that exposure to ELS in infancy might have long-lasting influences on functional brain connectivity that persist until early adulthood. Our results also speak for the importance of differentiating prospective and retrospective assessment methods to understand the specific neurodevelopmental effects of ELS.

MouseNet: A biologically constrained convolutional neural network model for the mouse visual cortex.

Convolutional neural networks trained on object recognition derive inspiration from the neural architecture of the visual system in mammals, and have been used as models of the feedforward computation performed in the primate ventral stream. In contrast to the deep hierarchical organization of primates, the visual system of the mouse has a shallower arrangement. Since mice and primates are both capable of visually guided behavior, this raises questions about the role of architecture in neural computation. In this work, we introduce a novel framework for building a biologically constrained convolutional neural network model of the mouse visual cortex. The architecture and structural parameters of the network are derived from experimental measurements, specifically the 100-micrometer resolution interareal connectome, the estimates of numbers of neurons in each area and cortical layer, and the statistics of connections between cortical layers. This network is constructed to support detailed task-optimized models of mouse visual cortex, with neural populations that can be compared to specific corresponding populations in the mouse brain. Using a well-studied image classification task as our working example, we demonstrate the computational capability of this mouse-sized network. Given its relatively small size, MouseNet achieves roughly 2/3rds the performance level on ImageNet as VGG16. In combination with the large scale Allen Brain Observatory Visual Coding dataset, we use representational similarity analysis to quantify the extent to which MouseNet recapitulates the neural representation in mouse visual cortex. Importantly, we provide evidence that optimizing for task performance does not improve similarity to the corresponding biological system beyond a certain point. We demonstrate that the distributions of some physiological quantities are closer to the observed distributions in the mouse brain after task training. We encourage the use of the MouseNet architecture by making the code freely available.

Decoding spatiotemporal features of emotional body language in social interactions

How are emotions perceived through human body language in social interactions? This study used point-light displays of human interactions portraying emotional scenes (1) to examine quantitative intrapersonal kinematic and postural body configurations, (2) to calculate interaction-specific parameters of these interactions, and (3) to analyze how far both contribute to the perception of an emotion category (i.e. anger, sadness, happiness or affection) as well as to the perception of emotional valence. By using ANOVA and classification trees, we investigated emotion-specific differences in the calculated parameters. We further applied representational similarity analyses to determine how perceptual ratings relate to intra- and interpersonal features of the observed scene. Results showed that within an interaction, intrapersonal kinematic cues corresponded to emotion category ratings, whereas postural cues reflected valence ratings. Perception of emotion category was also driven by interpersonal orientation, proxemics, the time spent in the personal space of the counterpart, and the motion–energy balance between interacting people. Furthermore, motion–energy balance and orientation relate to valence ratings. Thus, features of emotional body language are connected with the emotional content of an observed scene and people make use of the observed emotionally expressive body language and interpersonal coordination to infer emotional content of interactions.

Does the TPJ fit it all? Representational similarity analysis of different forms of mentalizing

ABSTRACT Mentalizing is the key socio-cognitive ability. Its heterogeneous structure may result from a variety of forms of mental state inference, which may be based on lower-level processing of cues encoded in the observable behavior of others, or rather involve higher-level computations aimed at understanding another person’s perspective. Here we aimed to investigate the representational content of the brain regions engaged in mentalizing. To this end, 61 healthy adults took part in an fMRI study. We explored ROI activity patterns associated with five well-recognized ToM tasks that induce either decoding of mental states from motion kinematics or belief-reasoning. By using multivariate representational similarity analysis, we examined whether these examples of lower- and higher-level forms of social inference induced common or distinct patterns of brain activity. Distinct patterns of brain activity related to decoding of mental states from motion kinematics and belief-reasoning were found in lTPJp and the left IFG, but not the rTPJp. This may indicate that rTPJp supports a general mechanism for the representation of mental states. The divergent patterns of activation in lTPJp and frontal areas likely reflect differences in the degree of involvement of cognitive functions which support the basic mentalizing processes engaged by the two task groups.

Emotion is perceived accurately from isolated body parts, especially hands

Body posture and configuration provide important visual cues about the emotion states of other people. We know that bodily form is processed holistically, however, emotion recognition may depend on different mechanisms; certain body parts, such as the hands, may be especially important for perceiving emotion. This study therefore compared participants' emotion recognition performance when shown images of full bodies, or of isolated hands, arms, heads and torsos. Across three experiments, emotion recognition accuracy was above chance for all body parts. While emotions were recognized most accurately from full bodies, recognition performance from the hands was more accurate than for other body parts. Representational similarity analysis further showed that the pattern of errors for the hands was related to that for full bodies. Performance was reduced when stimuli were inverted, showing a clear body inversion effect. The high performance for hands was not due only to the fact that there are two hands, as performance remained well above chance even when just one hand was shown. These results demonstrate that emotions can be decoded from body parts. Furthermore, certain features, such as the hands, are more important to emotion perception than others. Statement of relevanceSuccessful social interaction relies on accurately perceiving emotional information from others. Bodies provide an abundance of emotion cues; however, the way in which emotional bodies and body parts are perceived is unclear. We investigated this perceptual process by comparing emotion recognition for body parts with that for full bodies. Crucially, we found that while emotions were most accurately recognized from full bodies, emotions were also classified accurately when images of isolated hands, arms, heads and torsos were seen. Of the body parts shown, emotion recognition from the hands was most accurate. Furthermore, shared patterns of emotion classification for hands and full bodies suggested that emotion recognition mechanisms are shared for full bodies and body parts. That the hands are key to emotion perception is important evidence in its own right. It could also be applied to interventions for individuals who find it difficult to read emotions from faces and bodies.