Simple Feedforward Model Research Articles

Biased competition through variations in amplitude of γ-oscillations

Experiments in visual cortex have shown that the firing rate of a neuron in response to the simultaneous presentation of a preferred and non-preferred stimulus within the receptive field is intermediate between that for the two stimuli alone (stimulus competition). Attention directed to one of the stimuli drives the response towards the response induced by the attended stimulus alone (selective attention). This study shows that a simple feedforward model with fixed synaptic conductance values can reproduce these two phenomena using synchronization in the gamma-frequency range to increase the effective synaptic gain for the responses to the attended stimulus. The performance of the model is robust to changes in the parameter values. The model predicts that the phase locking between presynaptic input and output spikes increases with attention.

Learning in a geometric model of place cell firing

Following Hartley et al. (Hartley et al. (2000) Hippocampus 10:369-379), we present a simple feed-forward model of place cell (PC) firing predicated on neocortical information regarding the environmental geometry surrounding the animal. Incorporating the idea of boundaries with distinct sensory qualities, we show that synaptic plasticity mediated by a BCM-like rule (Bienenstock et al. (1982) J Neurosci 2:32-48) produces PCs that encode position relative to specific extended landmarks. In an unchanging environment the model is shown to undergo an initial phase of learning, resulting in the formation of stable place fields. In familiar environments, perturbation of environmental cues produces graded changes in the firing rate and position of place fields. Model simulations are compared favorably with three sets of experimental data: (1) Results published by Barry et al. (Barry et al. (2006) Rev Neurosci 17:71-97) showing the slow disappearance of duplicate place fields produced when a barrier is placed into a familiar environment. (2) Rivard et al.'s (Rivard et al. (2004) J Gen Physiol 124:9-25) study showing a graded response in PC firing such that fields near to a centrally placed object encode space relative to the object, whereas more distant fields respond to the surrounding environment. (3) Fenton et al.'s (Fenton et al. (2000a) J Gen Physiol 116:191-209) observation that inconsistent rotation of cue cards produces parametric changes in place field positions. The merits of the model are discussed in terms of its extensibility and biological plausibility.

Optimal Coding Predicts Attentional Modulation of Activity in Neural Systems

Neuronal activity in response to a fixed stimulus has been shown to change as a function of attentional state, implying that the neural code also changes with attention. We propose an information-theoretic account of such modulation: that the nervous system adapts to optimally encode sensory stimuli while taking into account the changing relevance of different features. We show using computer simulation that such modulation emerges in a coding system informed about the uneven relevance of the input features. We present a simple feedforward model that learns a covert attention mechanism, given input patterns and coding fidelity requirements. After optimization, the system gains the ability to reorganize its computational resources (and coding strategy) depending on the incoming attentional signal, without the need of multiplicative interaction or explicit gating mechanisms between units. The modulation of activity for different attentional states matches that observed in a variety of selective attention experiments. This model predicts that the shape of the attentional modulation function can be strongly stimulus dependent. The general principle presented here accounts for attentional modulation of neural activity without relying on special-purpose architectural mechanisms dedicated to attention. This principle applies to different attentional goals, and its implications are relevant for all modalities in which attentional phenomena are observed.

Visual cortex neurons of monkeys and cats: temporal dynamics of the spatial frequency response function.

We measured the responses of striate cortex neurons as a function of spatial frequency on a fine time scale, over the course of an interval that is comparable to the duration of a single fixation (200 ms). Stationary gratings were flashed on for 200 ms and then off for 300 ms; the responses were analyzed at sequential 1-ms intervals. We found that 1) the preferred spatial frequency shifts through time from low frequencies to high frequencies, 2) the latency of the response increases as a function of spatial frequency, and 3) the poststimulus time histograms (PSTHs) are relatively shape-invariant across spatial frequency. The dynamic shifts in preferred spatial frequency appear to be a simple consequence of the latency shifts and the transient nature of the PSTH. The effects of these dynamic shifts on the coding of spatial frequency information are examined within the context of several different temporal integration strategies, and pattern-detection performance is determined as a function of the interval of integration, following response onset. The findings are considered within the context of related investigations as well as a number of functional issues: motion selectivity in depth, "coarse-to-fine" processing, direction selectivity, latency as a code for stimulus attributes, and behavioral response latency. Finally, we demonstrate that the results are qualitatively consistent with a simple feedforward model, similar to the one originally proposed in 1962 by Hubel and Wiesel, that incorporates measured differences in the response latencies and the receptive field sizes of different lateral geniculate nucleus inputs.

A spherical model for orientation and spatial–frequency tuning in a cortical hypercolumn

A theory is presented of the way in which the hypercolumns in primary visual cortex (V1) are organized to detect important features of visual images, namely local orientation and spatial-frequency. Given the existence in V1 of dual maps for these features, both organized around orientation pinwheels, we constructed a model of a hypercolumn in which orientation and spatial-frequency preferences are represented by the two angular coordinates of a sphere. The two poles of this sphere are taken to correspond, respectively, to high and low spatial-frequency preferences. In Part I of the paper, we use mean-field methods to derive exact solutions for localized activity states on the sphere. We show how cortical amplification through recurrent interactions generates a sharply tuned, contrast-invariant population response to both local orientation and local spatial frequency, even in the case of a weakly biased input from the lateral geniculate nucleus (LGN). A major prediction of our model is that this response is non-separable with respect to the local orientation and spatial frequency of a stimulus. That is, orientation tuning is weaker around the pinwheels, and there is a shift in spatial-frequency tuning towards that of the closest pinwheel at non-optimal orientations. In Part II of the paper, we demonstrate that a simple feed-forward model of spatial-frequency preference, unlike that for orientation preference, does not generate a faithful representation when amplified by recurrent interactions in V1. We then introduce the idea that cortico-geniculate feedback modulates LGN activity to generate a faithful representation, thus providing a new functional interpretation of the role of this feedback pathway. Using linear filter theory, we show that if the feedback from a cortical cell is taken to be approximately equal to the reciprocal of the corresponding feed-forward receptive field (in the two-dimensional Fourier domain), then the mismatch between the feed-forward and cortical frequency representations is eliminated. We therefore predict that cortico-geniculate feedback connections innervate the LGN in a pattern determined by the orientation and spatial-frequency biases of feed-forward receptive fields. Finally, we show how recurrent cortical interactions can generate cross-orientation suppression.

Analytical model for the effects of learning on spike count distributions.

The spike count distribution observed when recording from a variety of neurons in many different conditions has a fairly stereotypical shape, with a single mode at zero or close to a low average count, and a long, quasi-exponential tail to high counts. Such a distribution has been suggested to be the direct result of three simple facts: the firing frequency of a typical cortical neuron is close to linear in the summed input current entering the soma, above a threshold; the input current varies on several timescales, both faster and slower than the window used to count spikes; and the input distribution at any timescale can be taken to be approximately normal. The third assumption is violated by associative learning, which generates correlations between the synaptic weight vector on the dendritic tree of a neuron, and the input activity vectors it is repeatedly subject to. We show analytically that for a simple feed-forward model, the normal distribution of the slow components of the input current becomes the sum of two quasi-normal terms. The term important below threshold shifts with learning, while the term important above threshold does not shift but grows in width. These deviations from the standard distribution may be observable in appropriate recording experiments.

Dynamics of orientation tuning in macaque primary visual cortex

Orientation tuning of neurons is one of the chief emergent characteristics of the primary visual cortex, V1. Neurons of the lateral geniculate nucleus, which comprise the thalamic input to V1, are not orientation-tuned, but the majority of V1 neurons are quite selective. How orientation tuning arises within V1 is still controversial. To study this problem, we measured how the orientation tuning of neurons evolves with time using a new method: reverse correlation in the orientation domain. Orientation tuning develops after a delay of 30-45 milliseconds and persists for 40-85 ms. Neurons in layers 4C alpha or 4C beta, which receive direct input from the thalamus, show a single orientation preference which remains unchanged throughout the response period. In contrast, the preferred orientations of output layer neurons (in layers 2, 3, 4B, 5 or 6) usually change with time, and in many cases the orientation tuning may have more than one peak. This difference in dynamics is accompanied by a change in the sharpness of orientation tuning; cells in the input layers are more broadly tuned than cells in the output layers. Many of these observed properties of output layer neurons cannot be explained by simple feedforward models, whereas they arise naturally in feedback networks. Our results indicate that V1 is more than a bank of static oriented filters; the dynamics of output layer cells appear to be shaped by intracortical feedback.

29] Artificial neural networks

This chapter focuses on two classes of widely used artificial neural networks (ANNs), the perceptron-back-propagation and the Hopfield–Boltzmann machine models. It explores in detail the characteristics of a simple feedforward ANN model. The perceptron-back-propagation model is presented in the chapter in an intuitive applications-oriented style. The chapter describes the Hopfield–Boltzmann machine ANN models. ANNs represent a fundamentally different approach to computation. They are explicitly designed to mimic the basic organizational features of biological nervous systems—parallel, distributed processing. ANNs have also been called “parallel distributed processing,” “connectionist,” and “neuromorphic systems.” ANNs are novel, robust computational tools for pattern recognition, data mapping, and other applications. The demonstrated success of ANN techniques has lead to an explosion of interest among scientific and engineering circles. Both commercial and public domain ANN software exists for a variety of computer systems ranging from small personal computers to large, massively parallel supercomputers. ANNs consist of a large number of simple interconnected processing elements, where the processing elements are simplified models of neurons and the interconnections between the processing elements are simplified models of the synapses between neurons.

Large signal performance of feedback automatic gain control systems

Feedback automatic gain control (FBAGC) is used to maintain the mean level of one or more signals throughout a signal processing system at a fixed value. The paper is particularly concerned with the application of FBAGC to receiver design and discusses the problem of attempting to suppress large unwanted envelope variations of an incoming signal without distorting any wanted modulation. A simple feedforward model of FBAGC systems predicts their effect on deep sinusoidal variations of the input signal's envelope. Experimental results confirming the predictions are presented, and a method of determining the parameters of an undefined AGC system given. Finally, the fundamental problems of FBAGC dynamics are discussed, and a technique is suggested for greatly improving them.