Synapses In Neural Networks Research Articles

Overshoot‐Suppressed Memristor Array with AlN Oxygen Barrier for Low‐Power Operation in the Intelligent Neuromorphic Systems

As the demand for bio‐inspired neuromorphic systems grows, memristor has emerged as a pivotal component in artificial synaptic devices. This study delves into the advantages and limitations of the one‐transistor‐one‐resistor (1T‐1R) and 0T‐1R architectures for memory array configurations. A significant enhancement in the memristor's on/off ratio, surpassing 103, achieved by integrating both an overshoot suppression layer (OSL) and an ultra‐thin AlN oxygen barrier layer (OBL) is also reported. The concurrent insertion of an AlOx OSL is essential for manifesting the self‐compliance attribute in 4F2 memristor arrays. Evaluations of a 24 × 24 array embedded with OSL and OBL reveal a substantial reduction in retention variation. In terms of functionality, a 7.13‐fold decline in vector‐matrix multiplication error is observed, accentuating the potential of this approach for neural network synapse applications. The analog reset characteristics of the OBL memristor facilitate over 25 multilevel states. Furthermore, the adoption of nonnegative weights presents an avenue to potentially double synaptic integration density. Through simulation program with integrated circuit emphasis simulations, the necessity of nonnegative and 16‐level quantized conductance in balancing power consumption without accuracy loss at the image classification is validated.

A liquid metal-based module emulating the intelligent preying logic of flytrap

Plant species like the Venus flytrap possess unique abilities to intelligently respond to various external stimuli, ensuring successful prey capture. Their nerve-devoided structure provides valuable insights for exploring natural intelligence and constructing intelligent systems solely from materials, but limited knowledge is currently available and the engineering realization of such concept remains a significant challenge. Drawing upon the flytrap’s action potential resulting from ion diffusion, we propose a signal accumulation/attenuation model and a corresponding liquid metal-based logic module, which operates on the basis of the shape change of liquid metal within a sodium hydroxide buffer solution. The module itself exhibits memory and counting properties without involving any other electronic components, intelligently responding to various stimulus sequences, and reproducing the flytrap’s most logical function. We also demonstrate and forecast its potential as a moving window integration-based high-pass filter, artificial synapse in neural networks, and other related applications. This research provides a fresh perspective on comprehending the intelligence inherent in nature and its realization through physical structures, which is expected to inspire logic device development in a broad engineering field.

DNA Reaction Network Mimicking the Basic Steps of Synaptic Communication

Comprehensive SummaryBiological systems use intricate networks of chemical reactions to exchange information. How to simulate complex systems with simple strand‐displacement reactions is crucial to broaden the application scenario of the DNA reaction network. Here, we report the artificial DNA reaction network to mimic the operation and function of biological information transfer via strand‐displacement reaction. DNA is used as simple artificial analogs to schematize structures and transmit information. Using chemical synapses in neural networks as an example, we show that the proposed network enables core functions of biological systems, such as the long‐term potential of synapses, which underpin learning and memory. Also, we performed the simple “silicon mimetic” to link electronic circuits to chemical network‐based biological structures. As such, synaptic communication simulated by the DNA reaction network provides a complete demonstration for designing artificial reaction networks based on the essence of information interaction.

Linear conductance update improvement of CMOS-compatible second-order memristors for fast and energy-efficient training of a neural network using a memristor crossbar array.

Memristors are two-terminal memory devices that can change the conductance state and store analog values. Thanks to their simple structure, suitability for high-density integration, and non-volatile characteristics, memristors have been intensively studied as synapses in artificial neural network systems. Memristive synapses in neural networks have theoretically better energy efficiency compared with conventional von Neumann computing processors. However, memristor crossbar array-based neural networks usually suffer from low accuracy because of the non-ideal factors of memristors such as non-linearity and asymmetry, which prevent weights from being programmed to their targeted values. In this article, the improvement in linearity and symmetry of pulse update of a fully CMOS-compatible HfO2-based memristor is discussed, by using a second-order memristor effect with a heating pulse and a voltage divider composed of a series resistor and two diodes. We also demonstrate that the improved device characteristics enable energy-efficient and fast training of a memristor crossbar array-based neural network with high accuracy through a realistic model-based simulation. By improving the memristor device's linearity and symmetry, our results open up the possibility of a trainable memristor crossbar array-based neural network system that possesses great energy efficiency, high area efficiency, and high accuracy at the same time.

Quasi-Volatile MoS2 Barristor Memory for 1T Compact Neuron by Correlative Charges Trapping and Schottky Barrier Modulation.

Artificial neurons as the basic units of spiking neural network (SNN) have attracted increasing interest in energy-efficient neuromorphic computing. 2D transition metal dichalcogenide (TMD)-based devices have great potential for high-performance and low-power artificial neural devices, owing to their unique ion motion, interface engineering, and resistive switching behaviors. Although there are widespread applications of TMD-based artificial synapses in neural networks, TMD-based neurons are seldom reported due to the lack of bio-plausible multi-mechanisms to mimic leaking, integrating, and firing biological behaviors without external assistance. In this work, for the first time, a methodology is developed by introducing the hybrid effect of charge trapping (CT) and Schottky barrier (SB) in MoS2 FETs for barristor memory and one-transistor (1T) compact artificial neuron realization. By correlating the CT and SB processes, quasi-volatile and resistive switching behaviors are realized on the fabricated MoS2 FET and utilized to mimic the accumulating, leaking, and firing biological behaviors of neurons. Therefore, based on a single quasi-volatile CT-SB MoS2 barristor memory, a 1T compact neuron of the basic leaky-integral-and-fire (LIF) function is demonstrated without a peripheral circuit. Furthermore, a spiking neural network (SNN) based on the CT-SB MoS2 barristor neurons is simulated and implemented in pattern classification with high accuracy approaching 95.82%. This work provides a highly integrated and inherently low-energy implementation for neural networks.

Physics inspired compact modelling of hbox {BiFeO}_3 based memristors

With the advent of the Internet of Things, nanoelectronic devices or memristors have been the subject of significant interest for use as new hardware security primitives. Among the several available memristors, BiFemathrm{O}_{3} (BFO)-based electroforming-free memristors have attracted considerable attention due to their excellent properties, such as long retention time, self-rectification, intrinsic stochasticity, and fast switching. They have been actively investigated for use in physical unclonable function (PUF) key storage modules, artificial synapses in neural networks, nonvolatile resistive switches, and reconfigurable logic applications. In this work, we present a physics-inspired 1D compact model of a BFO memristor to understand its implementation for such applications (mainly PUFs) and perform circuit simulations. The resistive switching based on electric field-driven vacancy migration and intrinsic stochastic behaviour of the BFO memristor are modelled using the cloud-in-a-cell scheme. The experimental current–voltage characteristics of the BFO memristor are successfully reproduced. The response of the BFO memristor to changes in electrical properties, environmental properties (such as temperature) and stress are analyzed and consistant with experimental results.

Architectural Features of Neural Network Forecasting Software Systems with Continuous Training

In recent years, a method of neural network event forecasting has been developed, based on the use of a pair of recurrent neural networks with controlled elements. This method allows you to make predictions without interrupting training. However, for its full use, a reasonable software implementation is necessary. This study considers the problem of searching for a software architecture that implements the method of neural network forecasting with continuous learning. Offers an improved prediction method that significantly reduces the required amount of memory. A procedure for accelerated calculation of the weights of neural network synapses has been developed. To assess the effectiveness of the proposed architectural solutions, a comparative analysis of various variants of software implementations was conducted. In systems developed with the proposed innovations, the requirements for memory and computing resources are much lower than in software implementations of the prototype method. For example, the amount of memory required has decreased by an average of 15 times, and system initialization has taken 16 times less time. At the same time, the strategy of maximum memory saving in such systems proved to be unproductive compared to the combined approach. Based on the obtained comparison results, recommendations are given for the use and choice of architectures depending on the specific tasks facing the end user, and the hardware and software environment in which the forecasting systems are supposed to operate.

Memristive Logic Design of Multifunctional Spiking Neural Network with Unsupervised Learning

We report a prospective approach to neural network modeling based on implementation of metal-oxide heterostructures with non-volatile memory behavior and multilevel resistive switching. These structures could be used as artificial synapses in neural networks, providing a modulation of synaptic strength in the range of seven orders of magnitude. Together with leaky integrated-and-fire neurons, it allows to organize a feed-forward spiking neural network with embedded spike-timing-dependent plasticity mechanism at the hardware level. The results of computer simulation demonstrate an ability of reconstructed networks based on electronic multilevel synapses to unsupervised learning and processing of asynchronous stream of spikes. Following the results of computer simulation, the biologically inspired circuit design for artificial neural network with memristive synapses was developed. The idea underlying the circuit design is based on the analogue approach and implies the hybrid CMOS-neurons/memristive synapses network, where operational amplifiers used as elements of current control through the memristive devices.

Method of solving optimal design problems based on flexible tolerance strategy

In the work on solving problems of optimal design, structures working in aggressive external environments, a modified method is proposed, which is based on the flexible tolerance method. The proposed method allows to control the accuracy of solving systems of differential equations when calculating the constraint functions of an optimisation problem. Based on the information about the degree of closeness of the point of the solution space to the local extremum point, which is received by the neural network controller, its parameters change. For this purpose, various matrices of neural network synapses, trained for different precisions of calculating the functions of constraints, are used. This strategy is used to modify the flexible tolerance method, based on the use of a neural network controller. As a criterion of the flexible tolerance, the error of calculating the constraint functions is used. It is shown that the use of a neural network regulator of the accuracy of calculating the restriction functions in the modified flexible tolerance method allows to significantly increase its efficiency while simultaneously obtaining a solution to the problem with a given accuracy and compensating the computational costs connected with using the α-level generalisation principle.

Improved Learning Experience Memristor Model and Application as Neural Network Synapse

This paper proposes a memristor model, named learning experience memristor (LEM), for using as synapse in the associative neural network. The properties of LEM are discussed under different external voltages. And then, we design a new feedback learning rule, all input feedback (AIF). An associative neural network-based the AIF law and LEM synapse is constructed and analyzed, and the associative neural network incorporates learning experience behavior, forgetting, and threshold functions. The properties of LEM are also verified through PSpice simulation. The associative neural network circuit based on AIF law and LEM are constructed and simulated using PSpice, the simulation results are analyzed sufficiently. Finally, different memristors are used as synapses in the associative neural network, and we analyze and compare the simulation results. All simulation results show that the associative neural network incorporating LEM synapses and AIF learning law exhibits good performance, mimicking biological neural networks, and self-learning behavior.

A Compact Device Model for Nanoparticle-organic Memory Transistor’s Characterization

Neuromorphic electronic devices have recently been a candidate for new computing architecture associated with innovative nanotechnologies. A report of the characterization of Nanoparticle organic memory transistor (NOMFET) introduced a similar behavior to a biological spiking synapse in neural networks. In this paper, a refinement model based on the extracted parameters including a hybrid NOMFET/CMOS neuromorphic computing circuit and architecture of synapse to neuron interface by characterizing transistor -- memory and the temporal dynamic function is presented. A compact EKV model refinement serves as a link between nanotechnology process and circuit design for novel CMOS devices.

Ultrasensitive Graphene Optoelectronic Probes for Recording Electrical Activities of Individual Synapses

The complex neuronal circuitry connected by submicron synapses in our brain calls for technologies that can map neural networks with ultrahigh spatiotemporal resolution to decipher the underlying mechanisms for multiple aspects of neuroscience. Here we show that, through combining graphene transistor arrays with scanning photocurrent microscopy, we can detect the electrical activities of individual synapses of primary hippocampal neurons. Through measuring the local conductance change of graphene optoelectronic probes directly underneath neuronal processes, we are able to estimate millivolt extracellular potential variations of individual synapses during depolarization. The ultrafast nature of graphene photocurrent response allows for decoding of activity patterns of individual synapses with a sub-millisecond temporal resolution. This new neurotechnology provides promising potentials for recording of electrophysiological outcomes of individual synapses in neural networks.

Reactive Astrocytes in Glioblastoma Multiforme.

Despite the multidisciplinary integration in the therapeutic management of glioblastoma multiforme (GBM), the prognosis of GBM patients is poor. There is growing recognition that the cells in the tumor microenvironment play a vital role in regulating the progression of glioma. Astrocytes are an important component of the blood-brain barrier (BBB) as well as the tripartite synapse neural network to promote bidirectional communication with neurons under physiological conditions. Emerging evidence shows that tumor-associated reactive astrocytes interact with glioma cells and facilitate the progression, aggression, and survival of tumors by releasing different cytokines. Communication between reactive astrocytes and glioma cells is further promoted through ion channels and ion transporters, which augment the migratory capacity and invasiveness of tumor cells by modifying H+ and Ca2+ concentrations and stimulating volume changes in the cell. This in part contributes to the loss of epithelial polarization, initiating epithelial-mesenchymal transition. Therefore, this review will summarize the recent findings on the role of reactive astrocytes in the progression of GBM and in the development of treatment-resistant glioma. In addition, the involvement of ion channels and transporters in bridging the interactions between tumor cells and astrocytes and their potential as new therapeutic anti-tumor targets will be discussed.

Unsupervised Learning Using Charge-Trap Transistors

Unsupervised learning is demonstrated using a device ubiquitously found in today's technology: a transistor with high-k -metal gate. Specifically, the charge-trapping phenomenon in the high-k gate dielectric is leveraged so that the device can be used as a non-volatile analog memory. Experimental data from 22-nm silicon-on-insulator devices reveal that a charge-trap transistor possesses promising characteristics for implementing synapses in neural networks, such as very fine tunability, weight-dependent plasticity, and low power consumption. A proof-of-concept winner-takes-all neural network is simulated based on experimental data and perfect clustering is achieved within tens of training cycles. This means that the network can be trained for multiple times, and a larger system can be built. The robustness of the procedure to the device variation is also discussed.

Modulating Effect of Peptide Semax on the Level of Synaptic Activity and Short-Term Plasticity in Glutamatergic Synapses of Co-Cultured Dorsal Root Ganglion and Dorsal Horn Neurons of Rats

The effects of a long-term culturing (12 days, in vitro) of the dorsal root ganglion (DRG) and the dorsal horn (DH) neurons with peptide Semax on the level of synaptic activity in co-cultures, as well as short-term plasticity in sensory synapses were studied. It has been revealed that neuronal culturing with peptide at concentrations of 10 and 100 µM leads to increasing the frequency of spontaneous glutamatergic postsynaptic currents in DH neurons by 71.7 ± 1.8% and 93.9 ± 3.1% (n = 6; P < 0.001). Semax does not exert a pronounced effect on the amplitude and frequency of miniature glutamatergic currents, but it causes an increase of the amplitudes of spontaneous postsynaptic currents and an elevation of the quantum content. The data obtained indicate the increase in multivesicular glutamate release efficiency in neural networks of co-cultures following incubation with the peptide. Moreover, Semax (10 and 100 µM) induces changes in the basic parameters of short-term plasticity of sensory synapses, namely, i) an increase in the paired-pulse ratio from (0.53 ± 0.028) (n = 8) to (0.91 ± 0.072) (n = 6, P < 0.01) and (0.95 ± 0.026) (n = 7; P < 0.001); ii) reduced coefficients of variation ratio (CV2/CV1) from (1.49 ± 0.11) (n = 8) to (1.02 ± 0.09) (n = 6; P < 0.05) and (1.11 ± 0.13) (n = 7; P < 0.01) respectively. The results indicate a stimulating effect of Semax on the activity of glutamatergic synapses in neural networks of co-cultures, as well as the ability of the peptide to modulate effectively short-term plasticity of sensory synapses.

비접지형 멤리스터 에뮬레이터 회로

A floating type of memristor emulator which acts like the behavior of memristor has been developed. Most of existing memristor emulators are grounded type which is built disregarding the connectivity with other memristor or other devices. The developed memristor emulator is a floating type whose output does not need to be grounded. Therefore, the emulator is able to be connected with other devices and be utilized for the interoperability test with various other circuits. To prove the floating function of the proposed memristor emulator, a Wheatstone bridge is built by connecting 4 memristor emulators in series and parallel. Also this bridge circuit suggest that it is possible to weight calculation of the neural network synapse.

EFFECT OF PEPTIDE SEMAX ON SYNAPTIC ACTIVITY AND SHORT-TERM PLASTICITY OF GLUTAMATERGIC SYNAPSES OF CO-CULTURED DORSAL ROOT GANGLION AND DORSAL HORN NEURONS

The influence of long-term culturing (12 days in vitro) of dorsal root ganglion (DRG) and dorsal horn (DH) neurons with peptide Semax on the level of synaptic activity at co-cultures, as well as short-term plasticity in sensory synapses were studied. It has been shown that neuronal culturing with peptide at concentrations of 10 and 100 µM led to increasing the frequency of spontaneous glutamatergic postsynaptic currents in DH neurons to 71.7 ± 1.8% and 93.9 ± 3.1% (n = 6; P < 0.001); Semax has a not significant effect on the amplitude and frequency of miniature glutamatergic currents, but causes an increase of the amplitudes of spontaneous postsynaptic currents, as well as elevates the quantum content. The data show the increase of multivesicular glutamate release efficiency in neural networks of co-cultures following incubation with the peptide. Also Semax (10 and 100 µM) induces changes of the basic parameters of short-term plasticity in sensory synapses: (1) increasing the paired-pulse ratio from 0.53 ± 0.028 (n = 8) to 0.91 ± 0.072 (n = 6, P < 0.01) and 0.95 ± 0.026 (n = 7; P < 0.001); (2) reducing the ratio of the coefficients of variation (CV2/ CV1) from 1.49 ± 0.11 (n = 8) to 1.02 ± 0.09 (n = 6; P < 0.05) and 1.11 ± 0.13 (n = 7; P < 0.0) respectively. The results indicate a stimulating effect of Semax on the activity of glutamatergic synapses in neural networks of co-cultures, as well as the ability of the peptide to effectively modulate the short-term plasticity in sensory synapses.

Noise cancellation of memristive neural networks

This paper investigates noise cancellation problem of memristive neural networks. Based on the reproducible gradual resistance tuning in bipolar mode, a first-order voltage-controlled memristive model is employed with asymmetric voltage thresholds. Since memristive devices are especially tiny to be densely packed in crossbar-like structures and possess long time memory needed by neuromorphic synapses, this paper shows how to approximate the behavior of synapses in neural networks using this memristive device. Also certain templates of memristive neural networks are established to implement the noise cancellation.

Retrospective-Prospective Differential Inclusions and Their Control by the Differential Connection Tensors of Their Evolutions: The Trendometer

This study is motivated by two different, yet, connected, motivations. The first one follows the observation that the classical definition of derivatives involves prospective (or forward) difference quotients, not known whenever the time is directed, at least at the macroscopic level. Actually, the available and known derivatives are retrospective (or backward). They co\incide whenever the functions are differentiable in the classical sense, but not in the case of non smooth maps, single-valued or set-valued. The later ones are used in differential inclusions (and thus, in uncertain control systems) governing evolutions in function of time and state. We follow the plea of some physicists for taking also into account the retrospective derivatives to study prospective evolutions in function of time, state and retrospective derivatives, a particular, but specific, example of historical of ''path dependent'' evolutionary systems. This is even more crucial in life sciences, in the absence of experimentation of uncertain evolutionary systems. The second motivation emerged from the study of networks with junctions (cross-roads in traffic networks, synapses in neural networks, banks in financial networks, etc.), an important feature of ''complex systems''. At each junction, the velocities of the incoming (retrospective) and outgoing (prospective) evolutions are confronted. One measure of this confrontation (''jerkiness'') is provided by the product of the retrospective and prospective velocities, negative in ''inhibitory'' junctions, positive for ''excitatory'' ones, for instance. This leads to the introduction of the ''differential connection tensor'' of two evolutions, defined as the tensor product of retrospective and prospective derivatives, which can be used for controlling evolutionary systems governing the evolutions through networks with junctions.

Synaptic dynamics: Linear model and adaptation algorithm

In this research, temporal processing in brain neural circuitries is addressed by a dynamic model of synaptic connections in which the synapse model accounts for both pre- and post-synaptic processes determining its temporal dynamics and strength. Neurons, which are excited by the post-synaptic potentials of hundred of the synapses, build the computational engine capable of processing dynamic neural stimuli. Temporal dynamics in neural models with dynamic synapses will be analyzed, and learning algorithms for synaptic adaptation of neural networks with hundreds of synaptic connections are proposed.The paper starts by introducing a linear approximate model for the temporal dynamics of synaptic transmission. The proposed linear model substantially simplifies the analysis and training of spiking neural networks. Furthermore, it is capable of replicating the synaptic response of the non-linear facilitation–depression model with an accuracy better than 92.5%. In the second part of the paper, a supervised spike-in-spike-out learning rule for synaptic adaptation in dynamic synapse neural networks (DSNN) is proposed. The proposed learning rule is a biologically plausible process, and it is capable of simultaneously adjusting both pre- and post-synaptic components of individual synapses. The last section of the paper starts with presenting the rigorous analysis of the learning algorithm in a system identification task with hundreds of synaptic connections which confirms the learning algorithm’s accuracy, repeatability and scalability. The DSNN is utilized to predict the spiking activity of cortical neurons and pattern recognition tasks. The DSNN model is demonstrated to be a generative model capable of producing different cortical neuron spiking patterns and CA1 Pyramidal neurons recordings. A single-layer DSNN classifier on a benchmark pattern recognition task outperforms a 2-Layer Neural Network and GMM classifiers while having fewer numbers of free parameters and decides with a shorter observation of data. DSNN performance in the benchmark pattern recognition problem shows 96.7% accuracy in classifying three classes of spiking activity.