An ecological approach to neural computation

Abstract

In traditional theories of perception, primitives are extracted from sensory signals, and ecologically relevant stimuli are described in terms of complex combinations of these primitives. Recognizing these combinations requires learning. James Gibson criticized this view, for the concept of primitive is a mathematical abstraction that may have no ecological meaning. What is for a sensory system may not be simple in terms of mathematical primitives, and conversely. Instead, Gibson argued that objects in an animal's environment produce sensory signals that are highly structured, and the invariant is what is meaningful about the environment. He proposed that sensory systems directly extract this invariant structure. I will address two questions, focusing on the auditory system: 1) what invariant can be considered for a sensory system? 2) how can the nervous system extract this structure? I define elementary structure as the set of identities between the time-varying sensory signals produced by different receptors, up to a delay. That is, Si(t) = Sj(t+T), where i and j are receptor indexes. In the cochlea, this corresponds to a regularity in the spatio-temporal pattern of vibration of the basilar membrane. In a monaural context, sounds that produce such an are periodic sounds, that is, sounds that elicit a pitch percept. I postulate that the pitch of a sound is precisely the that it produces. When taking into account the physiological constraints in the identification of this (essentially, that conduction delays are limited), it explains many phenomena that were previously unexplained by any single model: lower and upper limits of pitch, the distinction between resolved and unresolved harmonic complexes, the topology of pitch (e.g. the octave similarity), and slight level dependences of pitch. In a binaural context, stimuli that produce an across the two cochleas are binaural sounds produced by a single source in the environment. The depends on source location but not on the signal emitted by the source. This is much more complex, mathematically, than often described. In particular, the interaural time difference (ITD) depends on frequency. I will show evidence from single unit recordings in the inferior colliculus of cats that binaural neurons do indeed encode this structure, and not the mathematically simpler ITD. How can this be extracted by the nervous system? Since neurons are highly sensitive to coincident inputs, a natural question to ask is for what signals the presynaptic neurons produce synchronous spikes. This yields to the concept of the synchrony receptive field of a group of neurons, defined as the set of stimuli that elicit synchronous responses in these neurons. When applied to sensory neurons, the synchrony receptive field is precisely the of sensory signals, and it can be decoded by postsynaptic neurons by coincidence detection. I will demonstrate that simple spiking models based on these ideas can indeed decode the of auditory signals corresponding to pitch and sound location. This is from the environmental viewpoint, but not from the mathematical viewpoint. Therefore, I suggest that this approach provides simple practical solutions to non-trivial sensory problems.