Sparse Coding and Natural Statistics

Abstract

The brain is exposed to continuous sensory input from the environment (and the body). How does the brain encode such continuous sensory input and translate it into neural activity, e.g., stimulus-induced activity? Results from cellular recordings show that single neurons and a population of neurons represent the stimulus in a rather sparse way so that many stimuli are represented in one neuron’s (or one population of neurons’) activity. This amounts to a many-to-one relationship between stimuli and neurons entailing sparse coding. As such, sparse coding must be distinguished from other coding strategies like dense and local coding that propose a one-to-many and one-to-one relationship between stimuli and neurons. How is such sparse coding possible? The neurons’ (and population of neurons’) activity seems to encode the statistical frequency distribution of stimuli across their different discrete points in physical time and space; that is, their natural statistics. However, this is possible only when presupposing that differences between the stimuli’s different discrete points in physical space and time are encoded into neural activity. In other words, spatial and temporal differences (between the different discrete points in physical time and space) must be encoded into neural activity in order for sparse coding as a many-to-one relationship between stimuli and neurons to be possible.