Synaptic Weight Values Research Articles

Graphene‐Assisting Nonvolatile Vanadium Dioxide Phase Transition for Neuromorphic Machine Vision

AbstractAs a photoinduced and electro‐induced phase change material, VO2 undergoes a transition from an insulating phase to a metallic phase under photoelectric stimulation, accompanied by strong lattice distortion and band changes. This characteristic of simultaneously responding to optical and electrical signals provides a basis for simulating biological vision systems, but the volatility of phase transition challenges the Long‐term memory of neural networks. Here, a phase transition‐regulated artificial photoelectric synapse based on VO2/graphene heterostructure is proposed. Graphene serves as an electron exchange center, amplifying weak signals generated by VO2 phase transitions and achieving nonvolatile properties. Using the energy band change of VO2 before and after the photoinduced phase transition and the electron exchange with graphene, synaptic devices can be regulated by optical signals. The modulation of gate voltage on the Fermi level of graphene leads to the phase transition of VO2, thereby achieving the regulation of synapses by electrical signals. Extract the electrical conductivity difference between two equivalent synapses as synaptic weight values to train a three‐layer neural network. The trained neural network achieves high recognition accuracy and noise resistance for handwritten digits, which is of great significance for the application of artificial optoelectronic synapses in neural morphology calculation.

Unsupervised logic mining with a binary clonal selection algorithm in multi-unit discrete Hopfield neural networks via weighted systematic 2 satisfiability

<p>Evaluating behavioral patterns through logic mining within a given dataset has become a primary focus in current research. Unfortunately, there are several weaknesses in the research regarding the logic mining models, including an uncertainty of the attribute selected in the model, random distribution of negative literals in a logical structure, non-optimal computation of the best logic, and the generation of overfitting solutions. Motivated by these limitations, a novel logic mining model incorporating the mechanism to control the negative literal in the systematic Satisfiability, namely Weighted Systematic 2 Satisfiability in Discrete Hopfield Neural Network, is proposed as a logical structure to represent the behavior of the dataset. For the proposed logic mining models, we used ratio of <italic>r</italic> to control the distribution of the negative literals in the logical structures to prevent overfitting solutions and optimize synaptic weight values. A new computational approach of the best logic by considering both true and false classification values of the learning system was applied in this work to preserve the significant behavior of the dataset. Additionally, unsupervised learning techniques such as Topological Data Analysis were proposed to ensure the reliability of the selected attributes in the model. The comparative experiments of the logic mining models by utilizing 20 repository real-life datasets were conducted from repositories to assess their efficiency. Following the results, the proposed logic mining model dominated in all the metrics for the average rank. The average ranks for each metric were Accuracy (7.95), Sensitivity (7.55), Specificity (7.93), Negative Predictive Value (7.50), and Mathews Correlation Coefficient (7.85). Numerical results and in-depth analysis demonstrated that the proposed logic mining model consistently produced optimal induced logic that best represented the real-life dataset for all the performance metrics used in this study.</p>

Extreme synchronization events in a model neuron-astrocyte network

The examination of synchronization in neural networks is essential to comprehend how the brain functions in normal and pathological states. The synchronization of signals between groups of neurons is paramount for brain activity and associated with various brain functions, including memory, movement, sleep, and attention. However, a balance between synchronization and desynchronization must be maintained for proper brain function. During epilepsy, spontaneous transitions occur between two states of brain activity: synchronous and asynchronous. These transitions can be induced by various factors, including imbalances in chemical signals, changes in network activity, and other mechanisms that are not yet fully understood. Understanding these spontaneous transitions and the underlying mechanisms is a crucial aspect of epilepsy research. It will lead to the development of novel treatment approaches and strategies for inhibiting synchronized activity and averting seizure episodes. A neuron-astrocyte network model was constructed in this study, which displayed spontaneous and induced transitions between asynchronous and synchronous states triggered by external stimuli. The network model utilized 1000 Izhikevich neurons [1] interconnected with excitatory synapses and organized according to a scale-free graph. Each neuron had a bidirectional interaction with a single astrocyte, resulting in a total of 1000 astrocytes in the network. The Ullah model [2] simulated intracellular calcium ion concentration dynamics in a single astrocyte. The action potential generated by a neuron causes the release of neurotransmitters into the synaptic cleft, which triggers the corresponding astrocyte to release calcium ions from its endoplasmic reticulum into the cytoplasm, generating a calcium impulse. The calcium pulse within the astrocyte led to the release of a gliotransmitter. This gliotransmitter has the potential to regulate the synaptic transmission efficiency of both presynaptic and postsynaptic terminals associated with the respective astrocyte. The study simulated the effect of astrocytic suppression on neurotransmitter release. The strength of the synaptic input connection to the neuron that interacts with the astrocyte decreased in proportion to the amplitude of the calcium pulse in the astrocyte. Within the context of the implemented model study, we analyzed the global order parameter’s dependence on the maximum synaptic weight. Our bifurcation analysis showed the presence of hysteresis within a specific range of parameter values for connection weight maximums. In this range, the system under consideration is capable of displaying both synchronous and asynchronous regimes. However, an asynchronous regime was the only observed outcome below the lower boundary of the range, and only a synchronous regime was observed above the upper boundary. A statistical analysis was carried out on the duration of the model’s asynchronous states, specifically for the parameter value of the maximum synaptic weight near the upper boundary of the hysteresis region. The network dynamics were simulated for a prolonged period, and a histogram of the asynchronous state durations was plotted on a double logarithmic scale. The resulting data points were approximated using linear regression, which yielded a power-law exponent of –3/2. Based on the obtained results, the neuron-astrocyte network can exhibit spontaneous transitions between two states: synchronous and asynchronous, similar to pathological processes in the brain. More precisely, the power-law exponent of –3/2 aligns with values discovered in experimental recordings of epileptic activity in rodent brains [3–5]. Detailed modeling of biophysical processes revealed the spontaneous emergence of global order in the network induced by noise. Astrocytic effects mediated the disruption of synchronization in the neural network by reducing the efficiency of synaptic transmission.

Biologically plausible information propagation in a complementary metal-oxide semiconductor integrate-and-fire artificial neuron circuit with memristive synapses

Neuromorphic circuits based on spikes are currently envisioned as a viable option to achieve brain-like computation capabilities in specific electronic implementations while limiting power dissipation given their ability to mimic energy-efficient bioinspired mechanisms. While several network architectures have been developed to embed in hardware the bioinspired learning rules found in the biological brain, such as spike timing-dependent plasticity, it is still unclear if hardware spiking neural network architectures can handle and transfer information akin to biological networks. In this work, we investigate the analogies between an artificial neuron combining memristor synapses and rate-based learning rule with biological neuron response in terms of information propagation from a theoretical perspective. Bioinspired experiments have been reproduced by linking the biological probability of release with the artificial synapse conductance. Mutual information and surprise have been chosen as metrics to evidence how, for different values of synaptic weights, an artificial neuron allows to develop a reliable and biological resembling neural network in terms of information propagation and analysis.

Advantages and Disadvantages of Learning a Multilayer Neural Network Using a Genetic Algorithm

The advantages and disadvantages of learning a multilayer neural network using genetic algorithm are studied. Investigation was conducted on the basis of the developed software, which allows to detect indications of diseases by analyzing photographs of human organs` tissues. The Java programming language was used to implement the project. The software product consists of modules that implement: multilayer neural network, genetic algorithm and mechanism for reading and processing images from the file system. The genetic algorithm is used to calculate the initial values of synaptic weights, which are specified by the error propagation algorithm.

Synchronization of memristor based bidirectionally coupled Hindmarsh-Rose neurons

An ideal memristor is a device whose resistive memory value is determine by its initial conditions and the voltage that has been applied across its terminals. As such, it is a good candidate to model the synaptic plasticity of neural systems. When memristors are included in neural models, they are called memristive neural networks. In this contribution, we investigate the emergence of synchronization in an array of two identical Hindmarsh-Rose neurons bidirectionally coupled through their voltage variables via memristors. We show that, for a sufficiently large positive memductance, synchronization emerges between neurons while the memristors converge to constant synaptic weight values. We illustrate our results with numerical simulations.

A Reconfigurable Graphene-Based Spiking Neural Network Architecture

In the paper we propose a reconfigurable graphene-based Spiking Neural Network (SNN) architecture and a training methodology for initial synaptic weight values determination. The proposed graphene-based platform is flexible, comprising a programmable synaptic array which can be configured for different initial synaptic weights and plasticity functionalities and a spiking neuronal array, onto which various neural network structures can be mapped according to the application requirements and constraints. To demonstrate the validity of the synaptic weights training methodology and the suitability of the proposed SNN architecture for practical utilization, we consider character recognition and edge detection applications. In each case, the graphene-based platform is configured as per the application tailored SNN topology and initial state and SPICE simulated to evaluate its reaction to the applied input stimuli. For the first application, a 2-layer SNN is used to perform character recognition for 5 vowels. Our simulation indicates that the graphene-based SNN can achieve comparable recognition accuracy with the one delivered by a functionally equivalent Artificial Neural Network. Further, we reconfigure the architecture for a 3-layer SNN to perform edge detection on 2 grayscale images. SPICE simulation results indicate that the edge extraction results are close agreement with the one produced by classical edge detection operators. Our results suggest the feasibility and flexibility of the proposed approach for various application purposes. Moreover, the utilized graphene-based synapses and neurons operate at low supply voltage, consume low energy per spike, and exhibit small footprints, which are desired properties for largescale energy-efficient implementations.

A study of conductance update method for Ni/SiNx/Si analog synaptic device

Neuromorphic systems are expected to be a breakthrough beyond the conventional von Neumann architecture when implementing an artificial neural network. In a neuromorphic system, analog synaptic devices store the synaptic weight values of an artificial neural network. Among various memory devices, RRAM-based synaptic device has several advantages such as excellent scaling potential with a simple two-terminal structure and low energy consumption during the read and write operations. However, it has an inherent limitation of abrupt and nonlinear change in the conductance characteristics. Here, we investigate the non-ideal characteristics of conductance modulation using a fabricated RRAM device. We also analyze the impact of non-ideal conductance modulation on pattern recognition accuracy through a device-to-system level simulation. In addition, to solve the drawback of the previous conductance update method (occasional RESET), we propose a new conductance update method (occasional RESET without re-write). This comprehensive experiment and device-to-system level study can facilitate the realization of reliable learning performance on RRAM-based neuromorphic systems.

An Event-Based Neural Network Architecture with Content Addressable Memory

A hybrid analog/digital very large-scale integration (VLSI) implementation of a spiking neural network with programmable synaptic weights was designed. The synaptic weight values are stored in an asynchronous module, which is interfaced to a fast current-mode event-driven DAC for producing synaptic currents with the appropriate amplitude values. It acts as a transceiver, receiving asynchronous events for input, performing neural computations with hybrid analog/digital circuits on the input spikes, and eventually producing digital asynchronous events in output. Input, output, and synaptic weight values are transmitted to/from the chip using a common communication protocol based on the address event representation (AER). Using this representation, it is possible to interface the device to a workstation or a microcontroller and explore the effect of different types of spike-timing dependent plasticity (STDP) learning algorithms for updating the synaptic weights values in the CAM module.

The Role of Geterogeneity in Synchronization of Spiking Neural Networks

The diversity and heterogeneity of neurons and synapses is an important factor in the functioning of the brain. In our work, we investigated the role of heterogeneity of neural populations in the occurrence of synchronous modes in a network connected by exciting links in the presence of an external exciting input. Using Monte-Carlo modeling and the semi-analytical modeling the distribution of the refractory density of neuron integrators and Hodgkin – Huxley neurons, we showed that there is a range of parameters for the stimulating current and the strength of connections in the population where the effects of neuronal heterogeneity on the threshold or on the stimulating current are opposite. For large values ​​of synaptic weights and subthreshold values ​​of the exciting current, heterogeneity contributes to the emergence of a synchronous mode in the neural network, while at the same time reducing the coupling strength and increasing the exciting current. The heterogeneity reduces the tendency of the neural network to synchronize. The results obtained make it possible to reconcile the known data on the effects of heterogeneity in the regulation of the synchronous regimes arising in the neural ensembles of the brain.

Comparative Analysis of Memristor based Synaptic Circuits for Neuromorphic Architectures

Memristor is one of the fundamental electrical elements, which has recently been successfully built. Memristor is being used extensively as synapses in neuromorphic applications. A common method for developing memristor based synaptic circuits is to store synaptic weight values within memristors as resistance values. It requires use of the continuous resistance (memristance) range available in the memristors to store the weights. In this paper, we study how different compact synaptic circuits implemented using Ti02 memristors to perform zero, negative, and positive synaptic weightings. Results of different memristor based circuits, based on pulsed input signals, are compared, analyzed and presented using Ti02 memristors via simulations.

Estimating stock closing indices using a GA-weighted condensed polynomial neural network

Accurate forecasting of changes in stock market indices can provide financial managers and individual investors with strategically valuable information. However, predicting the closing prices of stock indices remains a challenging task because stock price movements are characterized by high volatility and nonlinearity. This paper proposes a novel condensed polynomial neural network (CPNN) for the task of forecasting stock closing price indices. We developed a model that uses partial descriptions (PDs) and is limited to only two layers for the PNN architecture. The outputs of these PDs along with the original features are fed to a single output neuron, and the synaptic weight values and biases of the CPNN are optimized by a genetic algorithm. The proposed model was evaluated by predicting the next day’s closing price of five fast-growing stock indices: the BSE, DJIA, NASDAQ, FTSE, and TAIEX. In comparative testing, the proposed model proved its ability to provide closing price predictions with superior accuracy. Further, the Deibold-Mariano test justified the statistical significance of the model, establishing that this approach can be adopted as a competent financial forecasting tool.

Bifurcation diagrams in estimated parameter space using a pruned extreme learning machine

We propose an algorithm to estimate parameter spaces by using a pruned extreme learning machine, but without using principal component analysis, and we plot bifurcation diagrams in the estimated parameter spaces to visualize changes in system patterns. The estimation of parameter spaces can predict changes in the behavior of a system when its parameters are changed. It can be very helpful to adjust the optimal parameters of unknown systems. In this paper, we estimate the dimension of parameter spaces using the singular values of trained synaptic weights and the parameter spaces based on the method proposed by Bagarinao etal., using a pruned extreme learning machine. We motivate this use of a pruned extreme learning machine through numerical experiments, with the estimation of the dimension of parameter spaces for various systems, and we show that the proposed method can successfully plot bifurcation diagrams in the estimated parameter spaces. In addition, we show the results of a bifurcation diagram in the estimated parameter space for maps that are nonlinear-in-parameters, since Bagarinao etal. limited their method to linear-in-parameters maps. In particular, we estimate the parameter spaces for a mathematical model of induction motor drives and for a model of vegetation biomass in ecosystems, which are both nonlinear-in-parameter maps corresponding to real-world systems.

NEURAL NETWORKS APPLICATION FOR MECHANICAL PARAMETERS IDENTIFICATION OF ASYNCHRONOUS MOTOR

A method for identification of mechanical parameters of asynchronous motor is presented in this paper. The identification method is based on the use of our knowledge about the system. This paper clarifies the method using the example of identifying mechanical parameters of the three-phase squirrel-cage asynchronous motor. Model of mechanical subsystem of the motor is presented as well as results of simulation. The special neural network is used as identification model and its adaptation is based on the gradient descent method. Parameters of mechanical subsystem are derived from the values of synaptic weights of the neural identification model after its adaptation. Deviation of identified mechanical parameters in case moment of inertia was up to 0.03\% and in case of load torque was 1.45 \% of real values.

Stock Market Prediction using Neuro-Genetic Model

Background/Objectives: To design a stock market prediction system using neuro-genetic approach. To predict BSE Sensex closing price using an artificial neural network. To optimize the synaptic weight values using genetic algorithm. Methods/Statistical Analysis: In this research work input parameters related to the BSE Sensex are fed as input dataset to the Multi Layer Perceptron neural network and the future day Sensex closing value is predicted as the output. Various training algorithms are implemented and the results are compared. The best neural network model is further is subjected to synaptic weight optimization using Genetic Algorithm. The various models are then subjected to testing over a period of 15 days, to obtain the most accurate model. Findings: The proposed system applies variants of Back Propagation (BP) learning algorithm on a Multi Layer Perceptron network (MLP) which is trained using four years’ BSE Sensex data. The performance of the network is measured by Normalized Mean Squared Error (NMSE). It is observed that resilient back propagation algorithm with log sigmoid activation function gives the lowest NMSE of 0.003745. The research work also uses Genetic Algorithm (GA) for weight optimization. BP suffers from the danger of getting stuck in local minima. This is avoided by using GA to select the best synaptic weights and node thresholds initially and then proceeding with the training of MLP using BP. It is observed that this hybrid model gives optimized results. In order to substantiate the model proposed, experiments are first conducted without using GA. The results of this general BP MLP model are then compared with that of GA-BP MLP model and analyzed. NMSE for the GA-BP MLP model is 0.003092121. Artificial Neural Network has evolved out to be a better technique in capturing the structural relationship between a stock’s performance and its determinant factors more accurately than many other statistical techniques. Although neural network models are used for stock prediction very little work is done on BSE SENSEX data. The proposed work is unique as experiments are done using different variants of back propagation learning algorithm and different activation functions. Therefore these experimental results add value to the existing work. The proposed work also demonstrates that performance improvement can be achieved by using genetic algorithm. Application/Improvements: The proposed model can also be used for forecasting index returns of markets like New York Stock Exchange, Hang Seng Stock Exchange, Korea Stock Exchange, Taiwan Stock Exchange etc., by using appropriate data set. Keywords: Artificial Neural Network, Closing Value, Genetic Algorithm, Optimization, Sensex Prediction, Synaptic Weight

A Reconfigurable Digital Neuromorphic Processor with Memristive Synaptic Crossbar for Cognitive Computing

This article presents a brain-inspired reconfigurable digital neuromorphic processor (DNP) architecture for large-scale spiking neural networks. The proposed architecture integrates an arbitrary number of N digital leaky integrate-and-fire (LIF) silicon neurons to mimic their biological counterparts and on-chip learning circuits to realize spike-timing-dependent plasticity (STDP) learning rules. We leverage memristor nanodevices to build an N × N crossbar array to store not only multibit synaptic weight values but also network configuration data with significantly reduced area overhead. Additionally, the crossbar array is designed to be accessible both column- and row-wise to expedite the synaptic weight update process for learning. The proposed digital pulse width modulator (PWM) produces binary pulses with various durations for reading and writing the multilevel memristive crossbar. The proposed column based analog-to-digital conversion (ADC) scheme efficiently accumulates the presynaptic weights of each neuron and reduces silicon area overhead by using a shared arithmetic unit to process the LIF operations of all N neurons. With 256 silicon neurons, learning circuits and 64K synapses, the power dissipation and area of our DNP are 6.45 mW and 1.86 mm 2 , respectively, when implemented in a 90-nm CMOS technology. The functionality of the proposed DNP architecture is demonstrated by realizing an unsupervised-learning based character recognition system.

An Event-Based Neural Network Architecture With an Asynchronous Programmable Synaptic Memory

We present a hybrid analog/digital very large scale integration (VLSI) implementation of a spiking neural network with programmable synaptic weights. The synaptic weight values are stored in an asynchronous Static Random Access Memory (SRAM) module, which is interfaced to a fast current-mode event-driven DAC for producing synaptic currents with the appropriate amplitude values. These currents are further integrated by current-mode integrator synapses to produce biophysically realistic temporal dynamics. The synapse output currents are then integrated by compact and efficient integrate and fire silicon neuron circuits with spike-frequency adaptation and adjustable refractory period and spike-reset voltage settings. The fabricated chip comprises a total of 32 × 32 SRAM cells, 4 × 32 synapse circuits and 32 × 1 silicon neurons. It acts as a transceiver, receiving asynchronous events in input, performing neural computation with hybrid analog/digital circuits on the input spikes, and eventually producing digital asynchronous events in output. Input, output, and synaptic weight values are transmitted to/from the chip using a common communication protocol based on the Address Event Representation (AER). Using this representation it is possible to interface the device to a workstation or a micro-controller and explore the effect of different types of Spike-Timing Dependent Plasticity (STDP) learning algorithms for updating the synaptic weights values in the SRAM module. We present experimental results demonstrating the correct operation of all the circuits present on the chip.

APPLICATION OF NEURAL NETWORK MODEL FOR PARAMETERS IDENTIFICATION OF NON-LINEAR DYNAMIC SYSTEM

A method for identification of parameters of a non-linear dynamic system, such as an induction motor with saturation effect taken into account, is presented in this paper. Adaptive identifier with structure similar to model of the system performs identification. This identifier can be regarded as a special neural network, therefore its adaptation is based on the gradient descent method and Back-Propagation well known in the neural networks theory. Parameters of electromagnetic subsystems were derived from the values of synaptic weights of the estimator after its adaptation. Testing was performed with simulations taking into account noise in measured quantities. Deviations of identified parameters in case of electrical parameters of the system were up to 1% of real values. Parameters of non-linear magnetizing curve were identified with deviations up to 6 % of real values. Identifier was able to follow sudden changes of rotor resistance, load torque and moment of inertia.

Blood pressure long term regulation: A neural network model of the set point development

BackgroundThe notion of the nucleus tractus solitarius (NTS) as a comparator evaluating the error signal between its rostral neural structures (RNS) and the cardiovascular receptor afferents into it has been recently presented. From this perspective, stress can cause hypertension via set point changes, so offering an answer to an old question. Even though the local blood flow to tissues is influenced by circulating vasoactive hormones and also by local factors, there is yet significant sympathetic control. It is well established that the state of maturation of sympathetic innervation of blood vessels at birth varies across animal species and it takes place mostly during the postnatal period. During ontogeny, chemoreceptors are functional; they discharge when the partial pressures of oxygen and carbon dioxide in the arterial blood are not normal.MethodsThe model is a simple biological plausible adaptative neural network to simulate the development of the sympathetic nervous control. It is hypothesized that during ontogeny, from the RNS afferents to the NTS, the optimal level of each sympathetic efferent discharge is learned through the chemoreceptors' feedback. Its mean discharge leads to normal oxygen and carbon dioxide levels in each tissue. Thus, the sympathetic efferent discharge sets at the optimal level if, despite maximal drift, the local blood flow is compensated for by autoregulation. Such optimal level produces minimum chemoreceptor output, which must be maintained by the nervous system. Since blood flow is controlled by arterial blood pressure, the long-term mean level is stabilized to regulate oxygen and carbon dioxide levels. After development, the cardiopulmonary reflexes play an important role in controlling efferent sympathetic nerve activity to the kidneys and modulating sodium and water excretion.ResultsStarting from fixed RNS afferents to the NTS and random synaptic weight values, the sympathetic efferents converged to the optimal values. When learning was completed, the output from the chemoreceptors became zero because the sympathetic efferents led to normal partial pressures of oxygen and carbon dioxide.ConclusionsWe introduce here a simple simulating computational theory to study, from a neurophysiologic point of view, the sympathetic development of cardiovascular regulation due to feedback signals sent off by cardiovascular receptors. The model simulates, too, how the NTS, as emergent property, acts as a comparator and how its rostral afferents behave as set point.

Self-organization of feed-forward structure and entrainment in excitatory neural networks with spike-timing-dependent plasticity

Spike-timing dependent plasticity (STDP) is an organizing principle of biological neural networks. While synchronous firing of neurons is considered to be an important functional block in the brain, how STDP shapes neural networks possibly toward synchrony is not entirely clear. We examine relations between STDP and synchronous firing in spontaneously firing neural populations. Using coupled heterogeneous phase oscillators placed on initial networks, we show numerically that STDP prunes some synapses and promotes formation of a feedforward network. Eventually a pacemaker, which is the neuron with the fastest inherent frequency in our numerical simulations, emerges at the root of the feedforward network. In each oscillatory cycle, a packet of neural activity is propagated from the pacemaker to downstream neurons along layers of the feedforward network. This event occurs above a clear-cut threshold value of the initial synaptic weight. Below the threshold, neurons are self-organized into separate clusters each of which is a feedforward network.