Invariant Representations of Objects in Natural Scenes in the Temporal Cortex Visual Areas

Abstract

Neurophysiological evidence is described showing that some neurons in the macaque inferior temporal visual cortex have responses that are invariant with respect to the position, size and view of faces and objects, and that these neurons show rapid processing and rapid learning. Which face or object is present is encoded using a distributed representation in which each neuron conveys independent information in its firing rate, with little information evident in the relative time of firing of different neurons. The operation of the inferior temporal cortex when objects are selected in natural scenes, and the encoding of multiple objects in a scene, are described. A theory is described of how such invariant representations may be produced in a hierarchically organized set of visual cortical areas with convergent connectivity. The theory proposes that neurons in these visual areas use a modified Hebb synaptic modification rule with a short term memory trace to capture whatever can be captured at each stage that is invariant about objects as the objects change in retinal view, position, size, and rotation. Another population of neurons in the cortex in the superior temporal sulcus encodes other aspects of faces such as face expression, eye gaze, face view, and whether the head is moving. Outputs of these systems reach the amygdala, in which face-selective neurons are found, and also the orbitofrontal cortex, in which some neurons are tuned to face identity and others to face expression. In humans, activation of the orbitofrontal cortex is found when a change of face expression acts as a social signal that behavior should change; and damage to the orbitofrontal cortex can impair face and voice expression identification, and also the reversal of emotional behavior that normally occurs when reinforcers are reversed (see Rolls, E.T. 2008, Memory, Attention and Decision-Making. Oxford University Press).