Local Recurrent Connections Research Articles

Strong Recurrent Networks Compute the Orientation Tuning of Surround Modulation in the Primate Primary Visual Cortex

In macaque primary visual cortex (V1), neuronal responses to stimuli inside the receptive field (RF) are modulated by stimuli in the RF surround. This modulation is orientation specific. Previous studies suggested that, for some cells, this specificity may not be fixed but changes with the stimulus orientation presented to the RF. We demonstrate, in recording studies, that this tuning behavior is instead highly prevalent in V1 and, in theoretical work, that it arises only if V1 operates in a regime of strong local recurrence. Strongest surround suppression occurs when the stimuli in the RF and the surround are iso-oriented, and strongest facilitation when the stimuli are cross-oriented. This is the case even when the RF is suboptimally activated by a stimulus of nonpreferred orientation but only if this stimulus can activate the cell when presented alone. This tuning behavior emerges from the interaction of lateral inhibition (via the surround pathways), which is tuned to the preferred orientation of the RF, with weakly tuned, but strong, local recurrent connections, causing maximal withdrawal of recurrent excitation at the feedforward input orientation. Thus, horizontal and feedback modulation of strong recurrent circuits allows the tuning of contextual effects to change with changing feedforward inputs.

Optophysiological analysis of associational circuits in the olfactory cortex.

Primary olfactory cortical areas receive direct input from the olfactory bulb, but also have extensive associational connections that have been mainly studied with classical anatomical methods. Here, we shed light on the functional properties of associational connections in the anterior and posterior piriform cortices (aPC and pPC) using optophysiological methods. We found that the aPC receives dense functional connections from the anterior olfactory nucleus (AON), a major hub in olfactory cortical circuits. The local recurrent connectivity within the aPC, long invoked in cortical autoassociative models, is sparse and weak. By contrast, the pPC receives negligible input from the AON, but has dense connections from the aPC as well as more local recurrent connections than the aPC. Finally, there are negligible functional connections from the pPC to aPC. Our study provides a circuit basis for a more sensory role for the aPC in odor processing and an associative role for the pPC.

Performance evaluation of artificial neural network approaches in forecasting reservoir inflow

This study investigates the potential of Time Lag Recurrent Neural Networks (TLRN) for modeling the daily inflow into Eleviyan reservoir, Iran. TLRN are extended with short term memory structures that have local recurrent connections, thus making them an appropriate model for processing temporal (time-varying) information. For this study, the daily inflow into Eleviyan reservoir between years 2004–2007 was considered. To compare the performance of TLRN, a back propagation neural network was used. The TLRN model with gamma memory structure, eight input layer nodes, two hidden layer and one output layer (8-2-1) was found performing best out of three different models used in forecasting daily inflow. A comparison of results with back propagation neural network suggest that neither TLRN nor back propagation approaches were good in forecasting high inflow but, both approaches perform well when used to forecast low inflow values. However, statistical test suggests that both TLRN and back propagation neural network models were able to reproduce similar basic statistics as that of the actual data.

The Timing of Vision – How Neural Processing Links to Different Temporal Dynamics

In this review, we describe our recent attempts to model the neural correlates of visual perception with biologically inspired networks of spiking neurons, emphasizing the dynamical aspects. Experimental evidence suggests distinct processing modes depending on the type of task the visual system is engaged in. A first mode, crucial for object recognition, deals with rapidly extracting the glimpse of a visual scene in the first 100 ms after its presentation. The promptness of this process points to mainly feedforward processing, which relies on latency coding, and may be shaped by spike timing-dependent plasticity (STDP). Our simulations confirm the plausibility and efficiency of such a scheme. A second mode can be engaged whenever one needs to perform finer perceptual discrimination through evidence accumulation on the order of 400 ms and above. Here, our simulations, together with theoretical considerations, show how predominantly local recurrent connections and long neural time-constants enable the integration and build-up of firing rates on this timescale. In particular, we review how a non-linear model with attractor states induced by strong recurrent connectivity provides straightforward explanations for several recent experimental observations. A third mode, involving additional top-down attentional signals, is relevant for more complex visual scene processing. In the model, as in the brain, these top-down attentional signals shape visual processing by biasing the competition between different pools of neurons. The winning pools may not only have a higher firing rate, but also more synchronous oscillatory activity. This fourth mode, oscillatory activity, leads to faster reaction times and enhanced information transfers in the model. This has indeed been observed experimentally. Moreover, oscillatory activity can format spike times and encode information in the spike phases with respect to the oscillatory cycle. This phenomenon is referred to as “phase-of-firing coding,” and experimental evidence for it is accumulating in the visual system. Simulations show that this code can again be efficiently decoded by STDP. Future work should focus on continuous natural vision, bio-inspired hardware vision systems, and novel experimental paradigms to further distinguish current modeling approaches.

Regionally Localized Recurrent Excitation in the Dentate Gyrus of a Cortical Contusion Model of Posttraumatic Epilepsy

Posttraumatic epilepsy is a frequent consequence of brain trauma, but relatively little is known about how neuronal circuits are chronically altered after closed head injury. We examined whether local recurrent excitatory synaptic connections form between dentate granule cells in mice 8-12 wk after cortical contusion injury. Mice were monitored for behavioral seizures shortly after brain injury and < or = 10 wk postinjury. Injury-induced seizures were observed in 15% of mice, and spontaneous seizures were observed weeks later in 40% of mice. Timm's staining revealed mossy fiber sprouting into the inner molecular layer of the dorsal dentate gyrus ipsilateral to the injury in 95% of mice but not contralateral to the injury or in uninjured controls. Whole cell patch-clamp recordings were made from granule cells in isolated hippocampal brain slices. Cells in slices with posttraumatic mossy fiber sprouting had an increased excitatory postsynaptic current (EPSC) frequency compared with cells in slices without sprouting from injured and control animals (P < 0.001). When perfused with Mg(2+)-free artificial cerebrospinal fluid containing 100 microM picrotoxin, these cells had spontaneous bursts of EPSCs and action potentials. Focal glutamate photostimulation of the granule cell layer evoked a burst of EPSCs and action potentials indicative of recurrent excitatory connections in granule cells of slices with mossy fiber sprouting. In granule cells of slices without sprouting from injured animals and controls, spontaneous or photostimulation-evoked epileptiform activity was never observed. These results suggest that a new regionally localized excitatory network forms between dentate granule cells near the injury site within weeks after cortical contusion head injury.

Hyperconnectivity of Local Neocortical Microcircuitry Induced by Prenatal Exposure to Valproic Acid

Exposure to valproic acid (VPA) during embryogenesis can cause several teratogenic effects, including developmental delays and in particular autism in humans if exposure occurs during the third week of gestation. We examined the postnatal effects of embryonic exposure to VPA on microcircuit properties of juvenile rat neocortex using in vitro electrophysiology. We found that a single prenatal injection of VPA on embryonic day 11.5 causes a significant enhancement of the local recurrent connectivity formed by neocortical pyramidal neurons. The study of the biophysical properties of these connections revealed weaker excitatory synaptic responses. A marked decrease of the intrinsic excitability of pyramidal neurons was also observed. Furthermore, we demonstrate a diminished number of putative synaptic contacts in connection between layer 5 pyramidal neurons. Local hyperconnectivity may render cortical modules more sensitive to stimulation and once activated, more autonomous, isolated, and more difficult to command. This could underlie some of the core symptoms observed in humans prenatally exposed to valproic acid.

Modulation of synchrony without changes in firing rates.

It was often reported and suggested that the synchronization of spikes can occur without changes in the firing rate. However, few theoretical studies have tested its mechanistic validity. In the present study, we investigate whether changes in synaptic weights can induce an independent modulation of synchrony while the firing rate remains constant. We study this question at the level of both single neurons and neuronal populations using network simulations of conductance based integrate-and-fire neurons. The network consists of a single layer that includes local excitatory and inhibitory recurrent connections, as well as long-range excitatory projections targeting both classes of neurons. Each neuron in the network receives external input consisting of uncorrelated Poisson spike trains. We find that increasing this external input leads to a linear increase of activity in the network, as well as an increase in the peak frequency of oscillation. In contrast, balanced changes of the synaptic weight of excitatory long-range projections for both classes of postsynaptic neurons modulate the degree of synchronization without altering the firing rate. These results demonstrate that, in a simple network, synchronization and firing rate can be modulated independently, and thus, may be used as independent coding dimensions.

Mean-Driven and Fluctuation-Driven Persistent Activity in Recurrent Networks

Spike trains from cortical neurons show a high degree of irregularity, with coefficients of variation (CV) of their interspike interval (ISI) distribution close to or higher than one. It has been suggested that this irregularity might be a reflection of a particular dynamical state of the local cortical circuit in which excitation and inhibition balance each other. In this "balanced" state, the mean current to the neurons is below threshold, and firing is driven by current fluctuations, resulting in irregular Poisson-like spike trains. Recent data show that the degree of irregularity in neuronal spike trains recorded during the delay period of working memory experiments is the same for both low-activity states of a few Hz and for elevated, persistent activity states of a few tens of Hz. Since the difference between these persistent activity states cannot be due to external factors coming from sensory inputs, this suggests that the underlying network dynamics might support coexisting balanced states at different firing rates. We use mean field techniques to study the possible existence of multiple balanced steady states in recurrent networks of current-based leaky integrate-and-fire (LIF) neurons. To assess the degree of balance of a steady state, we extend existing mean-field theories so that not only the firing rate, but also the coefficient of variation of the interspike interval distribution of the neurons, are determined self-consistently. Depending on the connectivity parameters of the network, we find bistable solutions of different types. If the local recurrent connectivity is mainly excitatory, the two stable steady states differ mainly in the mean current to the neurons. In this case, the mean drive in the elevated persistent activity state is suprathreshold and typically characterized by low spiking irregularity. If the local recurrent excitatory and inhibitory drives are both large and nearly balanced, or even dominated by inhibition, two stable states coexist, both with subthreshold current drive. In this case, the spiking variability in both the resting state and the mnemonic persistent state is large, but the balance condition implies parameter fine-tuning. Since the degree of required fine-tuning increases with network size and, on the other hand, the size of the fluctuations in the afferent current to the cells increases for small networks, overall we find that fluctuation-driven persistent activity in the very simplified type of models we analyze is not a robust phenomenon. Possible implications of considering more realistic models are discussed.

Recruitment of local excitatory circuits in the superior colliculus following deafferentation and the regeneration of retinocollicular inputs

The local synaptic connectivity in the superficial gray layer of the superior colliculus (SC) was assessed following retinal ganglion cell axonal regeneration through a peripheral nerve graft into the SC of Lister Hooded rats, using in vitro brain slice techniques. Repair was effected between the ipsilateral eye and SC, following bilateral lesion of optic nerves and ablation of ipsilateral occipital cortex. Deafferentation surgery alone resulted in a complete loss of synaptic potentials of extrinsic origin, once both retinal and cortical inputs were removed. Stimulation of graft insertion sites elicited synaptic responses comprising monosynaptic and network-mediated depolarising events. This activity, together with similar spontaneous bursts of depolarising events and action potential firing, was generated by the activation of non-N-methyl-D-aspartate glutamate receptors. This behaviour may reflect the development of a local recurrent synaptic connectivity following the repair surgery, as both evoked and spontaneous responses developed into large long-lasting bursts of excitatory activity when inhibition mediated by GABA receptors was blocked. These results suggest that the ultrastructural changes in the superficial layers of the SC resulting from deafferentation are reflected functionally at the synaptic level in the target structure even after repair. Such changes are likely to compromise the ability of the target structure to function normally during information processing. Therefore, although axons regenerating along peripheral nerve grafts can make functional synaptic connections, their efficacy in activating the target structure will probably be compromised by local changes in synaptic connectivity.

Face localization and tracking in the neural abstraction pyramid

One of the major tasks in some human–computer interface applications, such as face recognition and video telephony, is to localize a human face in an image. In this paper, we propose to use hierarchical neural networks with local recurrent connectivity to solve this task not only in unambiguous situations, but also in the presence of complex backgrounds, difficult lighting, and noise. The networks are trained using a database of gray-scale still images and manually determined eye coordinates. They are able to produce reliable and accurate eye coordinates for unknown images by iteratively refining initial solutions. Because the networks process entire images, there is no need for any time-consuming scanning across positions and scales. Furthermore, the fast network updates allow for real-time face tracking. In this case, the networks are trained using still images that move in random directions. The trained networks are able to accurately track the eye positions in the test image sequences.

Turning on and off recurrent balanced cortical activity.

The vast majority of synaptic connections onto neurons in the cerebral cortex arise from other cortical neurons, both excitatory and inhibitory, forming local and distant 'recurrent' networks. Although this is a basic theme of cortical organization, its study has been limited largely to theoretical investigations, which predict that local recurrent networks show a proportionality or balance between recurrent excitation and inhibition, allowing the generation of stable periods of activity. This recurrent activity might underlie such diverse operations as short-term memory, the modulation of neuronal excitability with attention, and the generation of spontaneous activity during sleep. Here we show that local cortical circuits do indeed operate through a proportional balance of excitation and inhibition generated through local recurrent connections, and that the operation of such circuits can generate self-sustaining activity that can be turned on and off by synaptic inputs. These results confirm the long-hypothesized role of recurrent activity as a basic operation of the cerebral cortex.

Genetic Algorithm Training of Elman Neural Network in Motor Fault Detection

Fault detection methods are crucial in acquiring safe and reliable operation in motor drive systems. Remarkable maintenance costs can also be saved by applying advanced detection techniques to find potential failures. However, conventional motor fault detection approaches often have to work with explicit mathematic models. In addition, most of them are deterministic or non-adaptive, and therefore cannot be used in time-varying cases. In this paper, we propose an Elman neural network-based motor fault detection scheme to address these difficulties. The Elman neural network has the advantageous time series prediction capability because of its memory nodes, as well as local recurrent connections. Motor faults are detected from the variants in the expectation of feature signal prediction error. A Genetic Algorithm (GA) aided training strategy for the Elman neural network is further introduced to improve the approximation accuracy, and achieve better detection performance. Experiments with a practical automobile transmission gearbox with an artificial fault are carried out to verify the effectiveness of our method. Encouraging fault detection results have been obtained without any prior information on the gearbox model.

Intracortical origin of visual maps.

Previous experiments indicate that the shape of maps of preferred orientation in the primary visual cortex does not depend on visual experience. We propose a network model that demonstrates that the orientation and direction selectivity of individual units and the structure of the corresponding angle maps could emerge from local recurrent connections. Our model reproduces the structure of preferred orientation and direction maps, and explains the origin of their interrelationship. The model also provides an explanation for the correlation between position shifts of receptive fields and changes of preferred orientations of single neurons across the surface of the cortex.

Complex cells as cortically amplified simple cells.

The majority of synapses in primary visual cortex mediate excitation between nearby neurons, yet the role of local recurrent connections in visual processing remains unclear. We propose that these connections are responsible for the spatial-phase invariance of complex-cell responses. In a network model with selective cortical amplification, neurons exhibit simple-cell responses when recurrent connections are weak and complex-cell responses when they are strong, suggesting that simple and complex cells are the low- and high-gain limits of the same basic cortical circuit. Given the ubiquity of invariant responses in cognitive processing, the recurrent mechanism we propose for complex cells may be widely applicable.

Horizontal propagation of visual activity in the synaptic integration field of area 17 neurons.

The receptive field of a visual neuron is classically defined as the region of space (or retina) where a visual stimulus evokes a change in its firing activity. At the cortical level, a challenging issue concerns the roles of feedforward, local recurrent, intracortical, and cortico-cortical feedback connectivity in receptive field properties. Intracellular recordings in cat area 17 showed that the visually evoked synaptic integration field extends over a much larger area than that established on the basis of spike activity. Synaptic depolarizing responses to stimuli flashed at increasing distances from the center of the receptive field decreased in strength, whereas their onset latency increased. These findings suggest that subthreshold responses in the unresponsive region surrounding the classical discharge field result from the integration of visual activation waves spread by slowly conducting horizontal axons within primary visual cortex.

Nested reentrant and recurrent computation in early vision: a Bayesian neuromorphic model applied to hyperacuity

Hyperacuity is demonstrated in a neuromorphic model of the early visual system. The model incorporates Bayesian principles which are embodied in the dynamics of reentrant and recurrent feedback processes. Each retinotopically mapped area in the model represents a transformation of data from the visual field. Sensory information propagates in a bottom-up direction from one area to the next, while information based on Bayesian priors propagates in a top-down direction through reentrant connections. The 'bottom-up' and 'top-down' information maintain a separate existence in distinct layers of the model, but they interact through local connections within each area. Transformations between one area and the next are defined by the reentrant synaptic connections between areas, while local prior probability maps are defined by local recurrent connections within layers. The representation of hyperacuity is accomplished using a model of functional multiplicity: the large ratio of neurons in striate cortex compared with the number of afferent fibers projecting from the lateral geniculate nucleus. High functional multiplicity, in conjunction with hierarchical reentrant processing, allows the model to represent a fine-grained restoration of the line structure of visual input.

Connections of the hypothalamic paraventricular necleus with the neurohypophysis, median eminence, amygdala, lateral septum and midbrain periaqueductal gray: An electrophysiological study in the rat

Extracellular recordings were obtained from 555 paraventricular (PVN) nucleus neurons in pentobarbital-anesthetized male rats. Cells were examined for their spontaneous activity patterns and response to single 1-Hz electrical stimulation of the neurohypophysis, median eminence, amygdala, lateral septum (LS) and midbrain periaqueductal gray (PAG). Neurohypophyseal stimulation evoked antidromic activation from 109 neurons. Among spontaneouly active neurohypophyseal neurons, evidence of a recurrent inhibitory pathway usually required pituitary stimulus intensities twice threshold for antidromic activation. Orthodromic excitatory or inhibitory responses followed amygdala and LS stimulation, but not PAG stimulation. The amygdala influence was predominantly inhibitory to ‘phasic’ (putative vasopressin-secreting) PVN neurohypophyseal neurons. Neurohypophyseal stimulation evoked orthodromic responses from 124 PVN cells; some of these neurons were also responsive to stimulation in other sites. Median eminence stimulation evoked antidromic responses from 37 PVN neurons; some of these cells also displayed phasic activity but not evidence for recurrent inhibition. Twelve cells in this group were also activated antidromically from both the median eminence and the neurohypophysis; collision tests suggest that the median eminence innervation may be an axon collateral of a neurohypophyseal pathway. Amygdala stimulation was inhibitory to some cells in this category. Amygdala, LS and PAG stimulation evoked antidromic activation from a small number of PVN cells, but none of these cells appeared to innervate more than one area, including the neurohypophysis, and none displayed phasic activity. Orthodromic responses were recorded among other PVN neurons after stimulation in these sites; however, PAG stimulation was the least effective stimulation area. These observations provide additional electrophysiological data that confirm efferent PVN connections to all areas tested, afferent connections from amygdala and LS but not PAG, and the possibility for coordinated activity among PVN neurons through local recurrent or common afferent connections.