Synaptic Operation Research Articles

A novel memristive Hopfield neural network with application in associative memory

Memristor is a nanoscale electronic device that exhibits the synaptic characteristics in artificial neural network. Some valuable memristor-based synaptic circuits have been presented. However, the circuitry implementations of some simple neural network are still rarely involved before. This paper contributes to construct a novel memristive Hopfield neural network circuit. On one hand, an improved memristor bridge circuit is employed to realize synaptic operation which better performs zero, positive and negative synaptic weights without requiring any switches and inverters, and Pspice implementation scheme is also considered. On the other hand, the proposed bridge circuit greatly simplifies the structure of neural network, and reduces the conversion process between current and voltage signal. Furthermore, the associative memory in binary and color images is demonstrated on the basis of the proposed memristive network. A series of numerical simulations are designed to verify associative memory capability, and experimental results demonstrate the effectiveness of the proposed neural network via the cases of single-associative memory and multi-associative memory.

Multistate Resistive Switching Memory for Synaptic Memory Applications

Reproducible low bias bipolar resistive switching memory in HfZnOx based memristors is reported. The modification of the concentration of oxygen vacancies in the ternary oxide film, which is facilitated by adding ZnO into HfO2, results in improved memory operation by the ternary oxide compared to the single binary oxides. A controlled multistate memory operation is achieved by controlling current compliance and RESET stop voltages. A high DC cyclic stability up to 400 cycles in the multistate memory performance is observed. Conventional synaptic operation in terms of potentiation, depression plasticity, and Ebbinghaus forgetting process are also studied. The memory mechanism is shown to originate from the migration of the oxygen vacancies and modulation of the interfacial layers.

Modular neuron comprises of memristor-based synapse

Design of analog modular neuron based on memristor is proposed here. Since neural networks are built by repetition of basic blocks that are called neurons, using modular neurons is essential for the neural network hardware. In this work modularity of the neuron is achieved through distributed neurons structure. Some major challenges in implementation of synaptic operation are weight programmability, weight multiplication by input signal and nonvolatile weight storage. Introduction of memristor bridge synapse addresses all of these challenges. The proposed neuron is a modular neuron based on distributed neuron structure which it uses the benefits of the memristor bridge synapse for synaptic operations. In order to test appropriate operation of the proposed neuron, it is used in a real-world application of neural network. Off-chip method is used to train the neural network. The results show 86.7 % correct classification and about 0.0695 mean square error for 4-5-3 neural network based on proposed modular neuron.

Pattern classification by memristive crossbar circuits using ex situ and in situ training

Memristors are memory resistors that promise the efficient implementation of synaptic weights in artificial neural networks. Whereas demonstrations of the synaptic operation of memristors already exist, the implementation of even simple networks is more challenging and has yet to be reported. Here we demonstrate pattern classification using a single-layer perceptron network implemented with a memrisitive crossbar circuit and trained using the perceptron learning rule by ex situ and in situ methods. In the first case, synaptic weights, which are realized as conductances of titanium dioxide memristors, are calculated on a precursor software-based network and then imported sequentially into the crossbar circuit. In the second case, training is implemented in situ, so the weights are adjusted in parallel. Both methods work satisfactorily despite significant variations in the switching behaviour of the memristors. These results give hope for the anticipated efficient implementation of artificial neuromorphic networks and pave the way for dense, high-performance information processing systems.

Energy-Efficient Neuron, Synapse and STDP Integrated Circuits

Ultra-low energy biologically-inspired neuron and synapse integrated circuits are presented. The synapse includes a spike timing dependent plasticity (STDP) learning rule circuit. These circuits have been designed, fabricated and tested using a 90 nm CMOS process. Experimental measurements demonstrate proper operation. The neuron and the synapse with STDP circuits have an energy consumption of around 0.4 pJ per spike and synaptic operation respectively.

Memristor Bridge Synapses

In this paper, we propose a memristor bridge circuit consisting of four identical memristors that is able to perform zero, negative, and positive synaptic weightings. Together with three additional transistors, the memristor bridge weighting circuit is able to perform synaptic operation for neural cells. It is compact as both weighting and weight programming are performed in a memristor bridge synapse. It is power efficient, since the operation is based on pulsed input signals. Its input terminals are utilized commonly for applying both weight programming and weight processing signals via time sharing. In this paper, features of the memristor bridge synapses are investigated using the TiO memristor model via simulations.

A voltage mode memristor bridge synaptic circuit with memristor emulators.

A memristor bridge neural circuit which is able to perform signed synaptic weighting was proposed in our previous study, where the synaptic operation was verified via software simulation of the mathematical model of the HP memristor. This study is an extension of the previous work advancing toward the circuit implementation where the architecture of the memristor bridge synapse is built with memristor emulator circuits. In addition, a simple neural network which performs both synaptic weighting and summation is built by combining memristor emulators-based synapses and differential amplifier circuits. The feasibility of the memristor bridge neural circuit is verified via SPICE simulations.

Maintenance of Synaptic Stability Requires Calcium-Independent Phospholipase A2Activity

Phospholipases A2 (PLA2s) represent one of the largest groups of lipid-modifying enzymes. Over the years, significant advances have been made in understanding their potential physiological and pathological functions. Depending on their calcium requirement for activation, PLA2s are classified into calcium dependent and independent. This paper mainly focuses on brain calcium-independent PLA2 (iPLA2) and on the mechanisms by which they influence neuronal function and regulate synaptic plasticity. Particular attention will be given to the iPLA2 γ isoform and its role in the regulation of synaptic glutamate receptors. In particular, the paper discusses the possibility that brain iPLA2 γ deficiencies could destabilise normal synaptic operation and might contribute to the aetiology of some brain disorders. In this line, the paper presents new data indicating that iPLA2 γ deficiencies accentuate AMPA receptor destabilization and tau phosphorylation, which suggests that this iPLA2 isoform should be considered as a potential target for the treatment of Tau-related disorders.

River-Flow Forecasting Using Higher-Order Neural Networks

In this paper, we propose a novel neural modeling methodology for forecasting daily river discharge that makes use of neural units with higher-order synaptic operations (NU-HSOs). For hydrologic forecasting, conventional rainfall-runoff models based on mechanistic approaches in the literature have shown limitations attributable to their overparameterization and complexity. With the use of neural units with quadratic synaptic operation (NU-QSO) and cubic synaptic operation (NU-CSO), as suggested in this paper, the refined neural modeling methodology can overcome the intricacy and inefficiency of conventional models. In this paper, neural network (NN) models with NU-HSO are compared with conventional NNs with neural units with linear synaptic operation (NU-LSO) for forecasting river discharge. This study was conducted using 1- to 5-day lead time forecasting in the Mahanadi River basin at the Naraj gauging site to evaluate the effectiveness of the higher-order neural networks (HO-NNs). Performance indices for the prediction of daily discharge forecasting indicated that NNs with NU-CSO and NNs with NU-QSO achieved better performance than NNs with NU-LSO even with a lower number of hidden neurons. Thus, this study shows that HO-NNs can be effective in hydrologic forecasting. DOI: 10.1061/(ASCE)HE.1943-5584.0000486. © 2012 American Society of Civil Engineers.

Foundations of Nonconventional Neural Units and their Classification

This article introduces basic types of nonconventional neural units and focuses on their notation and classification. Namely, the notation and classification of higher order nonlinear neural units, time-delay dynamic neural units, and time-delay higher order nonlinear neural units are introduced. Brief introduction into the simplified parallels of the higher order nonlinear aggregating function of higher order neural units with both the synaptic and somatic neural operation of biological neurons is made. Based on the mathematical notation of neural input intercorrelations of higher order neural units, it is shown that the higher order polynomial aggregating function of neural inputs can be understood as a single-equation representation of synaptic neural operation plus partial somatic neural operation. Thus, it unravels new simplified yet universal mathematical insight into understanding the higher computational power of neurons that also conforms to biological neuronal morphology. The classification of nonconventional neural units is founded first according to the nonlinearity of the aggregating function; second, according to the dynamic order; and third, according to time-delay implementation within neural units.

Functional Maturation of CA1 Synapses Involves Activity-Dependent Loss of Tonic Kainate Receptor-Mediated Inhibition of Glutamate Release

Early in development, excitatory synapses transmit with low efficacy, one mechanism for which is a low probability of transmitter release (Pr). However, little is known about the developmental mechanisms that control activity-dependent maturation of the presynaptic release. Here, we show that during early development, transmission at CA3-CA1 synapses is regulated by a high-affinity, G protein-dependent kainate receptor (KAR), which is endogenously activated by ambient glutamate. By tonically depressing glutamate release, this mechanism sets the dynamic properties of neonatal inputs to favor transmission during high frequency bursts of activity, typical for developing neuronal networks. In response to induction of LTP, the tonic activation of KAR is rapidly down regulated, causing an increase in Pr and profoundly changing the dynamic properties of transmission. Early development of the glutamatergic connectivity thus involves an activity-dependent loss of presynaptic KAR function producing maturation in the mode of excitatory transmission from CA3 to CA1.

Essential function of alpha-calcium/calmodulin-dependent protein kinase II in neurotransmitter release at a glutamatergic central synapse.

A significant fraction of the total calciumcalmodulin-dependent protein kinase II (CaMKII) activity in neurons is associated with synaptic connections and is present in nerve terminals, thus suggesting a role for CaMKII in neurotransmitter release. To determine whether CaMKII regulates neurotransmitter release, we generated and analyzed knockout mice in which the dominant alpha-isoform of CaMKII was specifically deleted from the presynaptic side of the CA3-CA1 hippocampal synapse. Conditional CA3 alpha-CaMKII knockout mice exhibited an unchanged basal probability of neurotransmitter release at CA3-CA1 synapses but showed a significant enhancement in the activity-dependent increase in probability of release during repetitive presynaptic stimulation, as was shown with the analysis of unitary synaptic currents. These data indicate that alpha-CaMKII serves as a negative activity-dependent regulator of neurotransmitter release at hippocampal synapses and maintains synapses in an optimal range of release probabilities necessary for normal synaptic operation.

Modification of glutamate receptors by phospholipase A2: its role in adaptive neural plasticity.

Long-term potentiation (LTP) and long-term depression (LTD) are two electrophysiological models that have been studied extensively in recent years as they may represent basic mechanisms in many neuronal networks to store certain types of information. In several brain regions, it has been shown that these two forms of synaptic plasticity require sufficient dendritic depolarization, with the amplitude of the calcium signal being crucial for the generation of either LTP or LTD. The rise in calcium concentration mediated by the N-methyl-D-aspartate (NMDA) subtype of glutamate receptors has been proposed to stimulate various calcium-dependent enzymatic processes that could convert the induction signal into long-lasting changes in synaptic structure; protein kinases and phosphatases have so far been considered predominantly with regard to LTP and LTD formation. According to several lines of experimental evidence, changes in synaptic function observed with LTP and LTD are thought to be the result of modifications of postsynaptic currents mediated by the alpha-amino-3-hydroxy-5-methyl-4-isoxazole propionate (AMPA) subtype of glutamate receptors. Moreover, it has become apparent recently that activation of the calcium-dependent enzyme phospholipase A2 (PLA2) could be part of the molecular mechanisms involved in alterations of AMPA receptor properties during long-term changes in synaptic operation. In the present review, we will first describe the results that indicate a critical role of the phospholipases in regulating synaptic function. Next, sections will be devoted to the effects of PLA2 and phospholipids on the binding properties of glutamate receptors, and a revised biochemical model will be presented as an attempt to integrate the PLA2 enzyme into the mechanisms ( in particular kinases and phosphatases) that participate in adaptive neural plasticity. Finally, we will review data relevant to the issue of selective changes in AMPA binding after environmental enrichment and LTP.

Nucleus reticularis neurons mediate diverse inhibitory effects in thalamus.

Detailed information regarding the contribution of individual gamma-aminobutyric acid (GABA)-containing inhibitory neurons to the overall synaptic activity of single postsynaptic cells is essential to our understanding of fundamental elements of synaptic integration and operation of neuronal circuits. For example, GABA-containing cells in the thalamic reticular nucleus (nRt) provide major inhibitory innervation of thalamic relay nuclei that is critical to thalamocortical rhythm generation. To investigate the contribution of individual nRt neurons to the strength of this internuclear inhibition, we obtained whole-cell recordings of unitary inhibitory postsynaptic currents (IPSCs) evoked in ventrobasal thalamocortical (VB) neurons by stimulation of single nRt cells in rat thalamic slices, in conjunction with intracellular biocytin labeling. Two types of monosynaptic IPSCs could be distinguished. "Weak" inhibitory connections were characterized by a significant number of postsynaptic failures in response to presynaptic nRt action potentials and relatively small IPSCs. In contrast, "strong" inhibition was characterized by the absence of postsynaptic failures and significantly larger unitary IPSCs. By using miniature IPSC amplitudes to infer quantal size, we estimated that unitary IPSCs associated with weak inhibition resulted from activation of 1-3 release sites, whereas stronger inhibition would require simultaneous activation of 5-70 release sites. The inhibitory strengths were positively correlated with the density of axonal swellings of the presynaptic nRt neurons, an indicator that characterizes different nRt axonal arborization patterns. These results demonstrate that there is a heterogeneity of inhibitory interactions between nRt and VB neurons, and that variations in gross morphological features of axonal arbors in the central nervous system can be associated with significant differences in postsynaptic response characteristics.

Effect of phosphatidylserine on the binding properties of glutamate receptors in brain sections from adult and neonatal rats

The effects of phosphatidylserine (PS) on the binding properties of the AMPA (α-amino-3-hydroxy-5-methylisoxazolepropionic acid) and NMDA ( N-methyl- d-aspartate) subtypes of glutamate receptors were analyzed by quantitative autoradiography of [ 3H]AMPA, [ 3H]6-cyano-7-nitroquinoxaline-2,3-dione (CNQX) and [ 3H]glutamate binding on at brain tissue sections. Preincubation of brain sections with PS produced an increase in [ 3H]AMPA binding without modifying the binding properties of [3H]CNQX, an antagonist of AMPA receptors. This effect of PS appeared to be specific for the AMPA subtype of glutamate receptors as the same treatment did not modify [ 3H]glutamate binding to the NMDA receptors. Furthermore, the PS-induced increase in [3H]AMPA binding was different in various brain structures, being larger in the molecular layer of the cerebellum and almost absent in the striatum. Preincubation with calcium also augmented [ 3H]AMPA binding, and the lack of additivity of the effects of calcium and PS on [ 3H]AMPA binding strongly suggests that both treatments share a common mechanism(s) for producing increased agonist binding. Finally, the effect of PS on AMPA receptor properties was markedly reduced in rat brain sections prepared from neonatal rats at a developmental stage that is normally characterized by the absence of LTP expression in certain brain regions. The present data are consistent with the hypothesis that alteration in the lipid composition of synaptic membranes may be an important mechanism for regulating AMPA receptor properties. which could be involved in producing long-lasting changes in synaptic operation.

Transients in the inhibitory driving of neurons and their postsynaptic consequences

The presynaptic fiber at an inhibitory synapse on a pacemaker neuron was forced to generate transients, defined here as spike trains with a trend, unceasingly accelerating or slowing. Experiments were on isolated crayfish stretch receptor organs. Spike train analyses used tools and notions from conventional point processes and from non-linear dynamics. Pre- and postsynaptic discharges contrasted clearly in terms of rates and interspike intervals. The inhibitory train evolved monotonically and smoothly, following tightly the simple prescribed curves; it was uniform, exhibiting throughout a single and simple discharge form (i.e. interval patterning). The inhibited postsynaptic train alternately accelerated and slowed, not following tightly any simple curve; it was heterogeneous, exhibiting in succession several different and often complex discharge forms, and switching abruptly from one to another. The inhibited trains depended on the inhibitory transient's span, range and average slope. Accordingly, transients separated (not cuttingly) into categories with prolonged spans (over 1 s) and slow slopes (around 1/s 2) and those with short spans (under 1 s) and fast slopes (around 30/s 2). Special transients elicited postsynaptic discharges that reproduced it faithfully, e.g. accelerated with the transient and proportionately; no transient elicited postsynaptic discharges faithful to its mirror image. Crayfish synapses are prototypes, so these findings should be expected in any other junction, as working hypotheses at least. Implications involve the operation of neural networks, including the role of distortions and their compensation, and the underlying mechanisms. Transients have received little attention, most work on synaptic coding concentrating on stationary discharges. Transients are inherent to the changing situations that pervade everyday life, however, and their biological importance is self-evident. The different discharges encountered during a transient had strong similarities to the stationary forms reported for different pacemaker drivings that are called locking, intermittency, erratic and stammering; they were, in fact, trendy versions of these. Such forms appear with several synaptic drivings in the same order along the presynaptic rate scale; they may constitute basic building blocks for synaptic operation. In terms of non-linear science, it is as if the attractors postulated for stationary drivings remained strongly influential during the transients, though affected by the rate of change.

Potassium-induced long-term potentiation in area CA1 of the hippocampus involves phospholipase activation.

Previous studies have shown that potassium-induced long-term potentiation (LTP) of the Schaffer collateral/commissural synapses in area CA1 of the hippocampus shares common properties with tetanus-induced LTP. In the present investigation, we performed electrophysiological and binding experiments on CA1 hippocampal slices to evaluate the location and nature of the changes underlying potassium-induced LTP. Paired-pulse facilitation, which represents an index of transmitter release, was markedly reduced by potassium-induced LTP. In addition, KCl-induced LTP was associated with an increase in 3H-AMPA ([3H]-amino-3-hydroxy-5-methylisoxazole-4-propionate) binding to CA1 synaptic membranes when measured 40 min after high-potassium exposure; however, no changes were detected in binding of an antagonist ([3H]-6-cyano-7-nitroquinoxaline-2,3-dione; 3H-CNQX) to AMPA receptors in slices expressing KCl-induced LTP. Administration of the phospholipase A2 (PLA2) inhibitor bromophenacyl bromide (BPB) prior to potassium application prevented LTP formation as well as the changes in paired-pulse facilitation and 3H-AMPA binding that characterized this type of potentiation. Taken together, these data indicate that potassium-induced LTP may be related to modifications in both pre- and postsynaptic properties and confirm the hypothesis that PLA2 activation is an important mechanism in long-term changes of synaptic operation.

Dynamic Neural Units with Applications to the Control of Unknown Nonlinear Systems

To emulate higher cognitive functions, such as sensory information processing and the attendant complexities of learning, memory storage, control, and vision, the biological neuron has to be modeled based upon feedback networks. The authors proposed such a neuronal model, called a “dynamic neural unit” DNU based upon the topology of a reverberating circuit in a neuronal pool of the central nervous system. The DNU architecture embodies delay elements, feedforward and feedback synaptic weights, and a nonlinear activation function. In this article, the concept of DNU is extended by incorporating both synaptic and somatic operations. The synaptic operation provides the optimum feed forward and feedback weights, while the somatic operation determines the optimum gain slope of the nonlinear activation function for a given task. The architectural details and the learning and adaptive algorithm to modify adjustable parameters, namely, feed forward and feedback synaptic weights, and slope of nonlinear function of the DNU are presented. The algorithm implementation of an isolated DNU is given. Considering DNU as the basic processing element, a three-stage dynamic neural network is developed. The effectiveness of the proposed dynamic neural network architecture as applied to the control of unknown nonlinear systems is discussed and extensive simulation results are presented.

Understanding the intrinsic circuitry of the cat's lateral geniculate nucleus: electrical properties of the spine-triad arrangement.

Electron-microscopic studies of relay cells in the lateral geniculate nucleus of the cat have shown that the retinal input to X-cells is associated with a special synaptic circuitry, termed the spine-triad. The retinal afferent makes an asymmetrical synapse with both a dendritic appendage of the X-cell and a geniculate interneuron. The interneuron contacts in turn the same dendritic appendage with a symmetrical synaptic profile. The retinal input to geniculate Y-cells is predominantly found on dendritic shafts without any triadic arrangement. We explore the integrative properties of X- and Y-cells resulting from this striking dichotomy in synaptic architecture. The basis of our analysis is the solution of the passive cable equation in a HRP-stained and reconstructed geniculate X-cell with known somatic input resistance. Recent evidence shows that geniculate interneurons stain for glutamic acid decarboxylase, the synthesizing enzyme for the inhibitory neuro-transmitterγ-aminobutyric acid (GABA). Under the assumption that the GABAergic inhibition has a reversal potential close to the resting potential of the cell, activation of the interneuron reduces very efficiently the excitatory postsynaptic potential induced by the retinal afferentwithoutaffecting the electrical activity in the rest of the cell. Therefore, the spine—triad circuit implements the analogue version of an AND—NOT gate. Functionally, this corresponds to a presynaptic, feed-forward type of inhibition of the optic tract terminal although inhibition actually occurs at a postsynaptic site. If the inhibition has a reversal potential well below the resting potential, the inhibitory postsynaptic potential would hyperpolarize large parts of the relay cell, abolishing the local character of the synaptic veto operation. Since Y-cells lack this structure, inhibition acts globally, reducing the general electrical activity of the cell. We propose that geniculate interneurons gate the flow of visual information into the X-system as a function of the behavioural state of the animal, enhancing the centre-surround antagonism and possibly mediating reciprocal lateral inhibition and eye-movement-related suppression.

Pathways and polarities of synaptic interactions in the inner retina of the mudpuppy: I. Synaptic blocking studies

Intracellular recording techniques were applied to the perfused retina-eyecup of the mudpuppy ( Necturus maculosus). The use of input resistance measurements and the application of cobalt as a synaptic blocking agent provided evidence about the mode of synaptic operation for depolarizing and hyperpolarizing bipolar cell input to cells of the inner retina. In addition, cobalt produced a more rapid block of hyperpolarizing bipolars compared to depolarizing bipolars; in some postbipolar neurons, a temporary simplification of neuronal response waveform was observed, and such cells resembled the depolarizing bipolar. These experiments suggest that bipolar cell input is excitatory and that response characteristics of third-order neurons is partially dependent on the relative input from the two types of bipolar cells.