Neurons In Intermediate Layers Research Articles

Analysis of the Strengths of Various Intermediate Layer Neurons of Radial Basis Function and Vanilla Neural Networks in the Prediction of Signal Power Loss Using Measurement Data from Micro-Cell Environment

Analysis of the Strengths of Various Intermediate Layer Neurons of Radial Basis Function and Vanilla Neural Networks in the Prediction of Signal Power Loss Using Measurement Data from Micro-Cell Environment

Continual Learning in a Multi-Layer Network of an Electric Fish

Distributing learning across multiple layers has proven extremely powerful in artificial neural networks. However, little is known about how multi-layer learning is implemented in the brain. Here, we provide an account of learning across multiple processing layers in the electrosensory lobe (ELL) of mormyrid fish and report how it solves problems well known from machine learning. Because the ELL operates and learns continuously, it must reconcile learning and signaling functions without switching its mode of operation. We show that this is accomplished through a functional compartmentalization within intermediate layer neurons in which inputs driving learning differentially affect dendritic and axonal spikes. We also find that connectivity based on learning rather than sensory response selectivity assures that plasticity at synapses onto intermediate-layer neurons is matched to the requirements of output neurons. The mechanisms we uncover have relevance to learning in the cerebellum, hippocampus, and cerebral cortex, as well as in artificial systems.

Continual Learning in a Multi-Layer Network of an Electric Fish

Distributing learning across multiple layers has proven extremely powerful in artificial neural networks. However, little is known about how such multi-layer learning is implemented in the brain. Here we provide a detailed account of multi-layer learning in the electrosensory lobe (ELL) of mormyrid fish, a tractable biological model system, and report how two longstanding problems in multi-layer learning are solved. First, we find that the problem of separating learning signals from output signals suitable for driving behavior is solved by a functional compartmentalization, in which inputs driving learning differentially affect dendritic and axonal spikes in intermediate layer neurons. Second, we find that error 'credit assignment' is solved by organizing the connectivity between intermediate and output layer neurons on the basis of learning rather than sensory response selectivity. These mechanisms have broad relevance to learning in the cerebellum, hippocampus and cerebral cortex as well as in artificial systems.

Non-accidental properties, metric invariance, and encoding by neurons in a model of ventral stream visual object recognition, VisNet.

When objects transform into different views, some properties are maintained, such as whether the edges are convex or concave, and these non-accidental properties are likely to be important in view-invariant object recognition. The metric properties, such as the degree of curvature, may change with different views, and are less likely to be useful in object recognition. It is shown that in a model of invariant visual object recognition in the ventral visual stream, VisNet, non-accidental properties are encoded much more than metric properties by neurons. Moreover, it is shown how with the temporal trace rule training in VisNet, non-accidental properties of objects become encoded by neurons, and how metric properties are treated invariantly. We also show how VisNet can generalize between different objects if they have the same non-accidental property, because the metric properties are likely to overlap. VisNet is a 4-layer unsupervised model of visual object recognition trained by competitive learning that utilizes a temporal trace learning rule to implement the learning of invariance using views that occur close together in time. A second crucial property of this model of object recognition is, when neurons in the level corresponding to the inferior temporal visual cortex respond selectively to objects, whether neurons in the intermediate layers can respond to combinations of features that may be parts of two or more objects. In an investigation using the four sides of a square presented in every possible combination, it was shown that even though different layer 4 neurons are tuned to encode each feature or feature combination orthogonally, neurons in the intermediate layers can respond to features or feature combinations present is several objects. This property is an important part of the way in which high capacity can be achieved in the four-layer ventral visual cortical pathway. These findings concerning non-accidental properties and the use of neurons in intermediate layers of the hierarchy help to emphasise fundamental underlying principles of the computations that may be implemented in the ventral cortical visual stream used in object recognition.

A sequential learning algorithm for a spiking neural classifier

This paper presents a biologically inspired, sequential learning spiking neural classifier (SLSNC) for pattern classification problems. It consists of a two layered neural network and a separate decision block which estimates the predicted class label. Inspired by observations in the neuroscience literature, the input layer employs a new neuron model which converts real valued stimuli into spikes with varying amplitudes and firing times. The intermediate layer neurons are modeled as integrate-and-fire spiking neurons. The decision block identifies that intermediate neuron which fires first and returns the class label associated with that neuron as the predicted class label. The sequential learning algorithm for the spiking neural network automatically determines the network structure from the training samples and adapts its synaptic weights by long term potentiation and long term depression. Performance of SLSNC has been evaluated using a number of benchmark classification problems and the results have been compared with other well-known spiking neural network classifiers in the literature as well as with the standard support vector machine (SVM) with a Gaussian kernel and the fast learning Extreme Learning Machine (ELM) classifiers. The results clearly indicate that the described spiking neural network produces similar or better generalization performance with a smaller network.

Alternative Sites of Synaptic Plasticity in Two Homologous “Fan-out Fan-in” Learning and Memory Networks

To what extent are the properties of neuronal networks constrained by computational considerations? Comparative analysis of the vertical lobe (VL) system, a brain structure involved in learning and memory, in two phylogenetically close cephalopod mollusks, Octopus vulgaris and the cuttlefish Sepia officinalis, provides a surprising answer to this question. We show that in both the octopus and the cuttlefish the VL is characterized by the same simple fan-out fan-in connectivity architecture, composed of the same three neuron types. Yet, the sites of short- and long-term synaptic plasticity and neuromodulation are different. In the octopus, synaptic plasticity occurs at the fan-out glutamatergic synaptic layer, whereas in the cuttlefish plasticity is found at the fan-in cholinergic synaptic layer. Does this dramatic difference in physiology imply a difference in function? Not necessarily. We show that the physiological properties of the VL neurons, particularly the linear input-output relations of the intermediate layer neurons, allow the two different networks to perform the same computation. The convergence of different networks to the same computational capacity indicates that it is the computation, not the specific properties of the network, that is self-organized or selected for by evolutionary pressure.

Binary object extraction using bi-directional self-organizing neural network (BDSONN) architecture with fuzzy context sensitive thresholding

A novel neural network architecture suitable for image processing applications and comprising three interconnected fuzzy layers of neurons and devoid of any back-propagation algorithm for weight adjustment is proposed in this article. The fuzzy layers of neurons represent the fuzzy membership information of the image scene to be processed. One of the fuzzy layers of neurons acts as an input layer of the network. The two remaining layers viz. the intermediate layer and the output layer are counter-propagating fuzzy layers of neurons. These layers are meant for processing the input image information available from the input layer. The constituent neurons within each layer of the network architecture are fully connected to each other. The intermediate layer neurons are also connected to the corresponding neurons and to a set of neighbors in the input layer. The neurons at the intermediate layer and the output layer are also connected to each other and to the respective neighbors of the corresponding other layer following a neighborhood based connectivity. The proposed architecture uses fuzzy membership based weight assignment and subsequent updating procedure. Some fuzzy cardinality based image context sensitive information are used for deciding the thresholding capabilities of the network. The network self organizes the input image information by counter-propagation of the fuzzy network states between the intermediate and the output layers of the network. The attainment of stability of the fuzzy neighborhood hostility measures at the output layer of the network or the corresponding fuzzy entropy measures determine the convergence of the network operation. An application of the proposed architecture for the extraction of binary objects from various degrees of noisy backgrounds is demonstrated using a synthetic and a real life image.

Activity in deep intermediate layer collicular neurons during interrupted saccades.

The activity of neurons located in the deep intermediate and adjacent deep layers (hereafter called just deep intermediate layer neurons) of the superior colliculus (SC) in monkeys was recorded during saccades interrupted by electrical stimulation of the brainstem omnipause neuron (OPN) region. The goal of the experiment was to determine if these neurons maintained their discharge during the saccadic interruption, and thus, could potentially provide a memory trace for the intended movement which ends accurately on target in spite of the perturbation. The collicular neurons recorded in the present study were located in the rostral three-fifths of the colliculus. Most of these cells tended to show considerable presaccadic activity during a delayed saccade paradigm, and, therefore, probably overlap with the population of SC cells called buildup neurons or prelude bursters in previous studies. The effect of electrical stimulation in the OPN region (which interrupted ongoing saccades) on the discharge of these neurons was measured by computing the percentage reduction in a cell's activity compared to that present during non-interrupted saccades. During saccade interruption about 70% of deep intermediate layer neurons experienced a major reduction (30% or greater) in their activity, but discharge recovered quickly after the termination of the stimulation as the eyes resumed their movement to finish the saccade on the target. Therefore, the pattern of activity recorded in most of the deep intermediate layer neurons during interrupted saccades qualitatively resembled that previously reported for the saccade-related burst neurons which tend to be located more dorsally in the intermediate layer. In contrast, some of our cells (30%) showed little or no perturbation in their activity caused by the saccade interrupting stimulation. Because all the more dorsally located burst neurons and the majority of our deep intermediate layer neurons show a total or major suppression in their discharge during interrupted saccades, it seems unlikely that the colliculus by itself could maintain an accurate memory of the desired saccadic goal or the remaining dynamic motor error required to account for the accuracy of the resumed movement which occurs following the interruption. However, it remains possible that the smaller proportion of our neurons whose activity was not perturbed during interrupted movements could play a role in the mechanisms underlying saccade accuracy in the interrupted saccade paradigm. Interrupted saccades have longer durations than normal saccades to the same target. Therefore, we investigated whether the discharge of our deeper collicular cells was also necessarily prolonged during interrupted saccades, and, if so, how the prolongation compared to the prolongation of the saccade. Sixty percent of our sample neurons showed a prolongation in discharge that was approximately the same as the prolongation in saccade duration (difference < 15 ms in magnitude). The, observation that temporal discharge in our neurons was perturbed to roughly match saccadic temporal perturbation suggests that dynamic feedback about ongoing saccadic motion is provided to the colliculus, but does not necessarily imply that this structure is the site responsible for the computation of dynamic motor error.

Role of intrinsic synaptic circuitry in collicular sensorimotor integration.

The superficial gray layer of the superior colliculus contains a map that represents the visual field, whereas the underlying intermediate gray layer contains a vector map of the saccades that shift the direction of gaze. These two maps are aligned so that a particular region of the visual field is represented directly above the neurons that orient the highest acuity area of the retina toward that region. Although it has been proposed that the transmission of information from the visuosensory to the motor map plays an important role in the generation of visually guided saccades, experiments have failed to demonstrate any functional linkage between the two layers. We examined synaptic transmission between these layers in vitro by stimulating the superficial layer while using whole-cell patch-clamp methods to measure the responses of intermediate layer neurons. Stimulation of superficial layer neurons evoked excitatory postsynaptic currents in premotor cells. This synaptic input was columnar in organization, indicating that the connections between the layers link corresponding regions of the visuosensory and motor maps. Excitatory postsynaptic currents were large enough to evoke action potentials and often occurred in clusters similar in duration to the bursts of action potentials that premotor cells use to command saccades. Our results indicate the presence of functional connections between the superficial and intermediate layers and show that such connections could play a significant role in the generation of visually guided saccades.

Migration of peptide-immunoreactive local circuit neurons to rat cingulate cortex.

The times of origin of neurons immunoreactive for somatostatin (SRIF), cholecystokinin (CCK), and vasoactive intestinal polypeptide (VIP) were determined using a combined immunohistochemical-autoradiographic technique. 3H-thymidine (3H-dT) was injected into pregnant rats on gestational day 13 (G13), G15, G17, G19, or G21. Animals were killed on postnatal day 30, that is, after the completion of neuronal migration. Sections of the brain including posterior cingulate cortex (area 29), visual cortex (area 17), and somatosensory cortex (area 3) were processed serially for peptide immunoreactivity and autoradiographically for labeling with 3H-dT. SRIF- and CCK-immunoreactive neurons were cogenerated and comigrated according to an inside-to-outside sequence. Accordingly, neurons in layer VI were born on G13, neurons in intermediate layers were born on G15-G17, and neurons in layer II/III were born on G19-G21. In contrast, VIP-positive neurons did not follow such a sequence. Neurons in all layers of cortex were generated at relatively constant rates between G13 and G21. VIP-immunoreactive neurons were the only known subpopulation of neurons that did not migrate into cortex by an inside-to-outside sequence. Thus, the migration of local circuit neurons to cingulate cortex follows patterns that are similar to those discerned in more differentiated cortical areas.

Effects of kainic acid and other excitotoxins in the rat superior colliculus: Relations to glutametergic afferents

In this study we have performed surgical, chemical and combined surgical/chemical lesions in order to elucidate neurotransmitter mechanisms in the superior colliculus (SC) of albino rats. Visual cortex (VC) ablation reduced high affinity (HA) uptake of d-Asp by 32% in the deafferented SC. Local injection of kainic acid (KA) into SC reduced HA d-Asp uptake selectively in the lower dose range ( < 1nmol) by 50–60%. The GABAergic marker glutamate decar☐ylase (GAD) was decreased by maximally 60% only at doses exceeding 2 nmol. Choline acetyltransferase (ChAT), however, was not affected at any of the doses administered. VC ablation provided an almost complete protection against 1 nmol KA. When KA was injected 2 days prior to VC ablation an additive effect on HA d-Asp uptake of the two lesions was observed. From these observations we infer that the notion of a glutamatergic projection from VC to SC has been strengthened. Moreover, local neurons in intermediate layers account for about 60% of the HA d-Asp uptake in SC, and these are most likely impinged upon by the glutamatergic afferents. The neurotoxic effects of KA were compared with those of some suspected endogenous excitotoxins, i.e. N-methyl tetrahydrofolic acid (Me-THF), other folates and the tryptophan metabolite quinolinic acid (QA). N-methyl tetrahydrofolic acid, Me-THF (4 and 10 nmol) reduced HA d-Asp uptake by about 50%, only when coinjected with ascorbic acid. GAD and ChAT were not affected at either of the doses. QA was about 100-fold less potent than KA on a molar basis, and the maximal reduction of GAD was similar in QA and KA injected animals, whereas the maximal reduction of HA d-Asp was only 40% after QA injection in SC. We conclude that Me-THF, QA and KA exert their neurotoxic actions by different mechanisms as judged by the behavioral, histopathological and biochemical sequelae seen after local injections of the respective substances in intermediate layers of SC and corroborate data obtained from other brain areas.

Laminar distribution and the response of superior colliculus neurons to stimulation of the periodontal ligament in the rabbit

Unit discharges of superior colliculus neurons fired by electrical stimulation of the periodontal ligament and/or mechanical stimulation of the mandibular incisor tooth were extracellularly recorded, and their laminar distribution and response properties were examined. The neurons were widely distributed in the deep layer of the anterior two-thirds of the superior colliculus. Most of them were located in the lateral site of the profound layer, a few in the lateral site of the intermediate layer. Their locations correspond to the lower nasal quadrant of the visual field. The mean first-spike latency of the intermediate layer neurons (7.2 ms) elicited by electrical stimulation of the periodontal ligament was shorter than that of the profound-layer neurons (13.8 ms). About one-third of the neurons were suppressed by conditioning electrical stimulation of the optic chiasma. These results suggested that in addition to visual, auditory, and body surface tactile sensations, intraoral somesthetic sensations also participate in the functioning of the superior colliculus which appears to play a role in orienting the animal toward visual, somesthetic, and auditory stimuli.