The Context–Object–Manipulation Triad: Cross Talk during Action Perception Revealed by fMRI

Abstract

To recognize an action, an observer exploits information about the applied manipulation, the involved objects, and the context where the action occurs. Context, object, and manipulation information are hence expected to be tightly coupled in a triadic relationship (the COM triad hereafter). The current fMRI study investigated the hemodynamic signatures of reciprocal modulation in the COM triad. Participants watched short video clips of pantomime actions, that is, actions performed with inappropriate objects, taking place at compatible or incompatible contexts. The usage of pantomime actions enabled the disentanglement of the neural substrates of context-manipulation (CM) and context-object (CO) associations. There were trials in which (1) both manipulation and objects, (2) only manipulation, (3) only objects, or (4) neither manipulation nor objects were compatible with the context. CM compatibility effects were found in an action-related network comprising ventral premotor cortex, SMA, left anterior intraparietal sulcus, and bilateral occipito-temporal cortex. Conversely, CO compatibility effects were found bilaterally in lateral occipital complex. These effects interacted in subregions of the lateral occipital complex. An overlap of CM and CO effects was observed in the occipito-temporal cortex and the dorsal attention network, that is, superior frontal sulcus/dorsal premotor cortex and superior parietal lobe. Results indicate that contextual information is integrated into the analysis of actions. Manipulation and object information is linked by contextual associations as a function of co-occurrence in specific contexts. Activation of either CM or CO associations shifts attention to either action- or object-related relevant information.