A single mechanism of temporal integration unites neural adaptation and norm-based coding

Abstract

What information is encoded in a cortical visual representation? That visual representations are distributed across the ventral temporal cortex is well established. fMRI adaptation studies demonstrate these neural codes are modulated by the perceptual similarity of sequential stimuli. Studies investigating prototype-based coding effects propose that neural responses are proportional to distinctiveness from a central reference, or prototype. In existing fMRI work, these two effects are considered independently. Here, we propose that these two effects arise as a consequence of a single mechanism of coding based upon temporal integration over recent stimulus history. Using a carry-over fMRI design, we show significant neural adaptation and prototype-based coding effects in a face-responsive region of interest in the right fusiform gyrus when effects are modeled discretely. By considering these effects as extremes of a single drifting norm model, we find that visual representations tend to encode identity in terms of intermediate stimulus history. Looking beyond the region of interest, we demonstrate that the effect of temporal context varies smoothly across the cortex, with the modulatory effect of recent visual history extending further back in time in a posterior to anterior fashion along the right ventral temporal cortex. These findings reframe two branches of the visual representation literature in terms of a unified encoding model. Importantly, this finding offers a perspective on the cortical topology of visual identity representations. We discuss these advances in terms of prior work related to the temporal qualities of visual stimuli. Meeting abstract presented at VSS 2014