Neural coding.

Abstract

In the years following the invention of the computer, an idea that caused great excitement was that computers might be made to mimic some of the brain's most basic cognitive capacities. Pattern recognition—the recognition of repeated occurrences of the same or similar stimuli—is, perhaps, the most basic of these, but a long history of pattern recognition research in computer science has shown it to be much more difficult than early investigators would have predicted. It was soon appreciated, for example, that pattern recognition performance of the kind we take for granted in ourselves is virtually impossible when based on isomorphic (e.g., photographic) representations: simple differences that we ignore (in size, location, orientation, etc.) create a mismatch between a new view and a stored view of an object as great as if the new view were of a completely different object. A powerful, elegant solution, which originated in neuroscience (13Pitts W McCulloch W.S Bull. Math. Biophys. 1947; 9: 127-147Crossref PubMed Scopus (372) Google Scholar), is to transform the initial isomorphic representation of an object into a different form in which invariant properties (form, structure, composition, etc.) are separated from properties that may vary from one exposure to the next. The solution is to transform the representation of each new object as it is encountered, store it in memory, and compare it with the transformed representations of previously stored objects. Whether the brain solves the pattern recognition problem in just this way is not known, but it is evident that the initial, isomorphic neural representations in each of the sensory systems are transformed to completely different forms of representation. The search for a form of representation that makes pattern recognition possible continues, and the human brain provides proof of its existence (19Tarr M.J Bülthoff H.H Object Recognition in Man, Monkey, and Machine. MIT Press, Cambridge, MA1998Google Scholar). In order to understand how information is represented and transformed as it flows through the sensory systems, we need to understand two things at each stage. First, we need to understand the neural activity and its relationship to the stimuli that evoke it. The second and often much more difficult problem is to understand how the information is coded—what is the relationship between that neural activity and perception? A complete understanding of the relationship between external stimuli and the neural activity evoked by them is no guarantee that an understanding of the significance of the neural activity will follow. The neural activity may have nothing to do with perception. The more difficult and usual case arises when the neural activity is rich in coding possibilities or is sufficiently complex that no obvious relationship to perception presents itself. Although the specific neural coding questions differ between the major sensory systems, the conceptual issues are the same. This review begins with the general experimental design used to address the neural coding problem and progresses to the discussion of an old coding problem that is being addressed with new methods. The general approach to the neural coding problem, illustrated in Figure 1, was laid out by Mountcastle in the 1960s. Psychophysical studies of a particular aspect of perceptual behavior are followed by neurophysiological studies using exactly the same stimuli and stimulus conditions. The object of the neurophysiological studies is to obtain a statistically accurate description of the neural activity underlying the psychophysical behavior. The final step is “to inquire which quantitative aspects of the neural response tally with psychophysical measurements.” (8Mountcastle V.B Poggio G.F Werner G J. Neurophysiol. 1963; 26: 807-834PubMed Google Scholar)—i.e., to advance all the plausible hypotheses and eliminate them one by one until a single hypothesis remains or another experiment is required to further reduce the possibilities. Any such hypothesis has two parts. The first is the hypothetical neural code—what aspect of the neural activity signals the information on which the behavior is based? The second part is a linking hypothesis, a model of the mechanism that acts upon the information provided by the putative neural code to produce the observed behavior. Statistical decision theory provides an accurate trial-by-trial link between the information provided by a successful putative neural code and behavior when the behavior is detection or discrimination (reviewed by 12Parker A.J Newsome W.T Annu. Rev. Neurosci. 1998; 21: 227-277Crossref PubMed Scopus (558) Google Scholar). When the behavior is subjective scaling, the individual subjective reports bear a linear relationship to the neural responses on which they are based (not a power law or a logarithmic law as older psychophysical theories have suggested; 4Johnson, K.O., Hsiao, S.S., and Blake, D.T. (1996). In Somesthesis and the Neurobiology of the Somatosensory Cortex, O. Franzén, R.S. Johansson, and L. Terenius, eds. (Basel: Birkhäuser Verlag), pp. 213–228.Google Scholar). Studies using the design illustrated in Figure 1 have shown that it is reasonable to expect close agreement between the predictions of a successful neural coding hypothesis and psychophysical behavior even when the stimuli are very complex (16Salzman C.D Murasugi C.M Britten K.H Newsome W.T J. Neurosci. 1992; 12: 2331-2355PubMed Google Scholar, 5May B.J Huang A.Y Le Prell G.S Hienz R.D Audit. Neurosci. 1996; 3: 135-162Google Scholar, 1Blake D.T Hsiao S.S Johnson K.O J. Neurosci. 1997; 17: 7480-7489PubMed Google Scholar). The first combined psychophysical and neurophysiological experiments designed explicitly to apply the scientific method to the question of neural coding, as illustrated in Figure 1, were conducted by Mountcastle and his colleagues (18Talbot W.H Darian-Smith I Kornhuber H.H Mountcastle V.B J. Neurophysiol. 1968; 31: 301-334PubMed Google Scholar, 9Mountcastle V.B Talbot W.H Sakata H Hyvärinen J J. Neurophysiol. 1969; 32: 452-484Crossref PubMed Scopus (498) Google Scholar). The aim was to identify the neural information used for the detection, discrimination, and scaling of vibratory stimuli applied to the hand. There are four cutaneous mechanoreceptor types, and all are sensitive to dynamic stimuli. Just determining which receptor type (or types) is responsible for a particular aspect of tactile perception is difficult, but much less so than determining how information about it is coded. Psychophysical experiments first characterized the human's and the monkey's ability to detect, discriminate, and scale vibratory stimuli across a wide range of frequencies and amplitudes. Recordings from hundreds of primary afferents showed that vibratory detection is accounted for by a dual mechanism—the occurrence of a threshold level of neural activity in Meissner afferents at low frequencies and in Pacinian afferents at higher frequencies. Frequency discrimination was found to depend on the periodicity of firing in these afferents (7Mountcastle V.B Johns Hopkins Med. J. 1975; 136: 109-131PubMed Google Scholar). Perceived vibratory intensity was found to depend on the integrated population response (3Johnson K.O J. Neurophysiol. 1974; 37: 48-72PubMed Google Scholar). Mountcastle carried this study to the cortex (9Mountcastle V.B Talbot W.H Sakata H Hyvärinen J J. Neurophysiol. 1969; 32: 452-484Crossref PubMed Scopus (498) Google Scholar, 10Mountcastle V.B Steinmetz M.A Romo R J. Neurosci. 1990; 10: 3032-3044PubMed Google Scholar) with the aim of determining whether vibratory frequency discrimination continues to depend on the periodicity of firing within cortical neurons. He pointed out that there is widespread interest in the temporal patterning of cortical neuronal firing (e.g., see 17Singer W Curr. Opin. Neurobiol. 1999; 9: 189-194Crossref PubMed Scopus (122) Google Scholar) but little concrete evidence that it is used as a neural coding mechanism. His studies in primary somatosensory cortex of monkeys (performing a vibratory discrimination task) showed that many rapidly adapting cortical neurons (driven by Meissner afferents) respond to a vibratory stimulus with a periodic discharge. He further showed that the mean impulse rate in these neurons is unaffected by stimulus frequency. On the basis of these and many other data, he inferred that the basis for frequency discrimination is discharge periodicity, not mean impulse rates. The task of a neuroscientist in a neural coding study is exactly like that of a perceptual psychologist, except that instead of trying to determine how perception depends on stimuli in the external world the investigator tries to determine how perception depends on the neural activity under investigation. But the perceptual psychologist can control the stimulus components directly to determine which aspects of the stimulus are relevant and which are not; the neural coding investigator can manipulate the neural activity only indirectly by manipulating the stimuli on which it depends. This can make it difficult to control the candidate codes to determine which are relevant and which are not. A relatively new development in neural coding studies is the use of electrical stimuli to affect the neural activity and even to control it in the absence of an external stimulus (6McIntyre C.C Grill W.M Ann. Biomed. Eng. 2000; 28: 219-233Crossref PubMed Scopus (163) Google Scholar). Electrical microstimulation through a recording electrode was first used in neural coding studies by 11Ochoa J.L Torebjörk H.E J. Physiol. 1983; 342: 633-654PubMed Google Scholar and 20Vallbo Å.B Olsson K.A Westberg K.G Clark F.J Brain. 1984; 107: 727-749Crossref PubMed Scopus (153) Google Scholar, who recorded from and stimulated single or very few peripheral nerve fibers in awake humans. The subjective reports confirmed Mountcastle's hypotheses concerning the functional roles of Meissner and Pacinian afferents and thereby added an element of proof that had been missing. Subjective report is, of course, not useful in animal experiments, which bear the major burden of discovery in this field. Newsome and his colleagues (16Salzman C.D Murasugi C.M Britten K.H Newsome W.T J. Neurosci. 1992; 12: 2331-2355PubMed Google Scholar) adapted the microstimulation method to animal experimentation by stimulating cortical neurons in animals performing a complex discrimination task. Monkeys were trained to discriminate the direction of group motion in random motion displays with weak group motion. They then recorded action potentials from single neurons in an area (MT) thought to be critical for motion perception while the animal performed the task. If the firing rate of a neuron signals motion in a particular direction, then increasing its firing rate should bias discrimination performance in that direction. Newsome and his colleagues used electrical microstimulation and decision theory to show that this is exactly what happens; they thereby established beyond reasonable doubt that area MT plays an important role in the discrimination of motion direction. Ranulfo Romo and his colleagues (14Romo R Hernández A Zainos A Salinas E Nature. 1998; 392: 387-390Crossref PubMed Scopus (341) Google Scholar, 15Romo R Hernández A Zainos A Brody C.D Lemus L Neuron. 2000; 26: 273-278Abstract Full Text Full Text PDF PubMed Scopus (221) Google Scholar, 2Hernández A Zainos A Romo R Proc. Natl. Acad. Sci. USA. 2000; 97: 6191-6196Crossref PubMed Scopus (155) Google Scholar) have carried this method further by using the neural activity evoked by microstimulation as a substitute for the neural activity evoked by an external stimulus. Theirs is also the first application of microstimulation that challenges a widely accepted coding hypothesis. Romo's studies are an extension of Mountcastle's neural coding studies, of which Romo was a part (10Mountcastle V.B Steinmetz M.A Romo R J. Neurosci. 1990; 10: 3032-3044PubMed Google Scholar). A long-standing problem in neural coding is how to challenge a temporal coding hypothesis—how to show that a high-fidelity temporal representation of the stimulus, although present in the neural response, is not used (if it is, in fact, not used). Romo and his colleagues addressed this problem by training monkeys to perform the tactile frequency discrimination task and by recording from neurons in somatosensory cortex as they performed the task (14Romo R Hernández A Zainos A Salinas E Nature. 1998; 392: 387-390Crossref PubMed Scopus (341) Google Scholar). In this task, two vibratory stimuli are presented in sequence (see Figure 2), and the monkey signals whether the frequency of the second stimulus is higher or lower than the first. When the recording electrode was positioned within a group of rapidly adapting neurons, sequences of electrical pulses were substituted for the second vibratory stimulus on some trials (Figure 2A). The result was that the discrimination behavior was indistinguishable from trials involving mechanical stimuli alone, which showed that these neural responses are involved in vibratory frequency discrimination. That does not mean that the sensations evoked by the electrical stimuli were the same as those evoked by the mechanical stimuli, but it does appear to mean that the neural activity on which the discrimination behavior was based, the neural code, was replicated by the electrical stimulation. They next challenged the hypothesis that frequency discrimination depends on periodicity in the neuronal discharge by randomizing the timing of the mechanical sinusoids (and the timing of the electrical pulses that replaced them on half the trials; Figure 2B). They found that the monkey's discrimination behavior was unaffected, which shows that periodicity was not necessary for the task that the monkey was trained to perform. The possibility that the perceived stimulus intensity might have covaried with the stimulus frequency and thereby served as a cue for behavior was eliminated by varying the intensity of the second stimulus randomly. A second study by 15Romo R Hernández A Zainos A Brody C.D Lemus L Neuron. 2000; 26: 273-278Abstract Full Text Full Text PDF PubMed Scopus (221) Google Scholar elaborates these findings in several ways. First, they showed that the neural responses evoked by electrical stimuli can be stored in memory and later compared with the responses evoked by either electrical or mechanical stimuli to make a frequency judgment (Figure 2C). In fact, any mix of electrical and mechanical stimuli in the first and second position produced indistinguishable behavior (Figure 2a–2d). This suggests that the sensations evoked by microstimulation and mechanical stimulation are similar; the slopes and positions of psychometric functions like those shown in Figure 2 are generally very sensitive to any change in experimental conditions. Second, they showed that this behavior is based on the stimulation of rapidly adapting neurons. When the electrodes were located within groups of slowly adapting neurons (which are driven by a class of primary afferent fibers that has nothing to do with vibratory sensation), the monkeys detected the electrical stimuli, but frequency discrimination performance was random (50% correct). Third, by positioning three electrodes within a column of rapidly adapting neurons, they showed, on several occasions, that the behavior illustrated in Figure 2 can be elicited anywhere within a column of rapidly adapting neurons. A third study by Romo and his colleagues (2Hernández A Zainos A Romo R Proc. Natl. Acad. Sci. USA. 2000; 97: 6191-6196Crossref PubMed Scopus (155) Google Scholar) has reexamined the periodicity and the mean impulse rates of over 200 rapidly adapting neurons in monkey somatosensory cortex as the monkey performed the vibratory discrimination task with periodic stimuli (Figure 2A). The new finding is that about one-third of these neurons signal vibratory frequency reliably by an increase in mean impulse rate with increasing vibratory frequency; about two-thirds signal vibratory frequency by the temporal structure of their responses. A careful analysis by Hernández et al. shows that mean impulse rate and temporal structure in single neuronal responses both transmit as much or more information than is required to account for behavior. What does all this mean? Romo's demonstration that monkeys easily compare periodic and aperiodic stimuli (Figure 2B) for the purposes of frequency discrimination rules out periodicity in the neuronal responses as the basis for the observed discrimination behavior, but it does not rule out temporal codes more generally. An important observation by 14Romo R Hernández A Zainos A Salinas E Nature. 1998; 392: 387-390Crossref PubMed Scopus (341) Google Scholar is that humans easily distinguish the periodic and aperiodic stimuli even when they have the same mean frequency; this suggests that the percept evoked by these stimuli is based on the temporal structure of the evoked neural activity—i.e., a temporal code. The monkeys' performance indicates that they were able to extract accurate measures of the mean stimulus rate in both the aperiodic and periodic conditions. One possibility is that frequency discrimination is based on a temporal code that is more general than periodicity—a coding mechanism that allows a judgment of mean frequency even though the neural activity is not periodic and that allows the recipient to perceive the difference between a periodic and an aperiodic stimulus at the same mean frequency. A different, promising possibility raised by the findings of Hernández et al. is that a measure of mean frequency is extracted within somatosensory cortex and that this is the basis for the monkeys' performance—the temporal structure of the cortical neuronal responses accounts for the human ability to distinguish aperiodic from periodic stimuli with the same mean frequency; the rate signal reported by Hernández et al. accounts for the monkeys' ability to discriminate mean frequency in the absence of periodicity. This is the interpretation favored by Hernández et al., but proof is still missing. The temporal structure of the responses of rapidly adapting neurons to aperiodic and periodic stimuli conveys more information than is required to account for the behavior illustrated in Figure 2. The first step in understanding the representations and transformations that underlie perception—understanding the neural activity evoked by sensory stimuli—is progressing rapidly. The second step—understanding how perception depends on that neural activity—can be difficult, as the studies reviewed here demonstrate. The principal difficulty lies in controlling the putative neural codes so that their relative contributions to the perceptual behavior in question can be distinguished. The basic approach will continue to be to manipulate the neural activity through its dependence on the stimulus (and, at higher levels, the animal's behavior) or, as the studies reviewed here show, to use electrical microstimulation to modify or even control the relevant neural activity directly. Efforts to develop effective sensory prostheses based on electrical microstimulation (6McIntyre C.C Grill W.M Ann. Biomed. Eng. 2000; 28: 219-233Crossref PubMed Scopus (163) Google Scholar) may make these methods more powerful and hence more useful for neural coding studies. Regardless of the difficult nature of the experiments, neural coding studies will grow in importance, because understanding how information is coded at the highest levels of information processing in the brain is a major, necessary step toward understanding the neural mechanisms of perception and cognition.*E-mail: [email protected]