Neuromorphic Networks Research Articles

Delay learning architectures for memory and classification

We present a neuromorphic spiking neural network, the DELTRON, that can remember and store patterns by changing the delays of every connection as opposed to modifying the weights. The advantage of this architecture over traditional weight-based ones is simpler hardware implementation without multipliers or digital–analog converters (DACs) as well as being suited to time-based computing. The name is derived due to similarity in the learning rule with an earlier architecture called tempotron. The DELTRON can remember more patterns than other delay-based networks by modifying a few delays to remember the most ‘salient’ or synchronous part of every spike pattern. We present simulations of memory capacity and classification ability of the DELTRON for different random spatio-temporal spike patterns. The memory capacity for noisy spike patterns and missing spikes is also shown. Finally, we present SPICE simulation results of the core circuits involved in a reconfigurable mixed signal implementation of this architecture.

A neuromorphic network for generic multivariate data classification

Computational neuroscience has uncovered a number of computational principles used by nervous systems. At the same time, neuromorphic hardware has matured to a state where fast silicon implementations of complex neural networks have become feasible. En route to future technical applications of neuromorphic computing the current challenge lies in the identification and implementation of functional brain algorithms. Taking inspiration from the olfactory system of insects, we constructed a spiking neural network for the classification of multivariate data, a common problem in signal and data analysis. In this model, real-valued multivariate data are converted into spike trains using "virtual receptors" (VRs). Their output is processed by lateral inhibition and drives a winner-take-all circuit that supports supervised learning. VRs are conveniently implemented in software, whereas the lateral inhibition and classification stages run on accelerated neuromorphic hardware. When trained and tested on real-world datasets, we find that the classification performance is on par with a naïve Bayes classifier. An analysis of the network dynamics shows that stable decisions in output neuron populations are reached within less than 100 ms of biological time, matching the time-to-decision reported for the insect nervous system. Through leveraging a population code, the network tolerates the variability of neuronal transfer functions and trial-to-trial variation that is inevitably present on the hardware system. Our work provides a proof of principle for the successful implementation of a functional spiking neural network on a configurable neuromorphic hardware system that can readily be applied to real-world computing problems.

Research progress of memristors and memristive mechanism

Memristors are the fourth basic circuit element in addition to the three classical elements: resistor, capacitor, and inductor, which have great application prospects in the fields of information storage, logic operations and neuromorphic networks. The recent development of memristors and memristive mechanism is reviewed, including connotations and characteristics of memristors, memristive mechanism, types of memristive mateirals, and application prospects of memristors. Finally, the key problems and development proposals are presented and a prospect on the development trend is also given.

A theoretical and experimental study of neuromorphic atomic switch networks for reservoir computing

Atomic switch networks (ASNs) have been shown to generate network level dynamics that resemble those observed in biological neural networks. To facilitate understanding and control of these behaviors, we developed a numerical model based on the synapse-like properties of individual atomic switches and the random nature of the network wiring. We validated the model against various experimental results highlighting the possibility to functionalize the network plasticity and the differences between an atomic switch in isolation and its behaviors in a network. The effects of changing connectivity density on the nonlinear dynamics were examined as characterized by higher harmonic generation in response to AC inputs. To demonstrate their utility for computation, we subjected the simulated network to training within the framework of reservoir computing and showed initial evidence of the ASN acting as a reservoir which may be optimized for specific tasks by adjusting the input gain. The work presented represents steps in a unified approach to experimentation and theory of complex systems to make ASNs a uniquely scalable platform for neuromorphic computing.

Pattern classification by memristive crossbar circuits using ex situ and in situ training

Memristors are memory resistors that promise the efficient implementation of synaptic weights in artificial neural networks. Whereas demonstrations of the synaptic operation of memristors already exist, the implementation of even simple networks is more challenging and has yet to be reported. Here we demonstrate pattern classification using a single-layer perceptron network implemented with a memrisitive crossbar circuit and trained using the perceptron learning rule by ex situ and in situ methods. In the first case, synaptic weights, which are realized as conductances of titanium dioxide memristors, are calculated on a precursor software-based network and then imported sequentially into the crossbar circuit. In the second case, training is implemented in situ, so the weights are adjusted in parallel. Both methods work satisfactorily despite significant variations in the switching behaviour of the memristors. These results give hope for the anticipated efficient implementation of artificial neuromorphic networks and pave the way for dense, high-performance information processing systems.

Principles of Neural Science. Fifth Edition. Edited by Eric R. Kandel, James H. Schwartz, Thomas M. Jessell, Steven A. Siegelbaum, and A. J. Hudspeth; Art Editor:, Sarah Mack. New York: McGraw-Hill. $135.00. l + 1709 p.; ill.; index. ISBN: 978-0-07-139011-8. 2013.

Previous articleNext article No AccessReviews and Brief NoticesPrinciples of Neural Science. Fifth Edition. Edited by Eric R. Kandel, James H. Schwartz, Thomas M. Jessell, Steven A. Siegelbaum, and A. J. Hudspeth; Art Editor:, Sarah Mack. New York: McGraw-Hill. $135.00. l + 1709 p.; ill.; index. ISBN: 978-0-07-139011-8. 2013.Paul E. TibbettsPaul E. TibbettsPhilosophy and Cognitive Science, University of Dayton, Dayton, Ohio Search for more articles by this author PDFPDF PLUSFull Text Add to favoritesDownload CitationTrack CitationsPermissionsReprints Share onFacebookTwitterLinkedInRedditEmail SectionsMoreDetailsFiguresReferencesCited by The Quarterly Review of Biology Volume 88, Number 2June 2013 Published in association with Stony Brook University Article DOIhttps://doi.org/10.1086/670559 Views: 1609Total views on this site Citations: 11Citations are reported from Crossref For permission to reuse, please contact [email protected]PDF download Crossref reports the following articles citing this article:Ahmad Rujoie, Ole Kæseler Andersen, Ken Steffen Frahm A novel temperature-controlled laser system to uniformly activate cutaneous thermal receptors during movable thermal stimulation, Journal of Neural Engineering 20, no.11 (Feb 2023): 016040.https://doi.org/10.1088/1741-2552/acb2f9S. J. Mahendra, Viswanath Talasila, Abhilash G. Dutt, Mukund Balaji, Abhishek C. Mouli A Functional Electrical Stimulator to Enable Grasping Through Wrist Flexion, International Journal of Biology and Biomedical Engineering 16 (Jan 2022): 19–29.https://doi.org/10.46300/91011.2022.16.4Agustín Bou, Juan Bisquert Impedance Spectroscopy Dynamics of Biological Neural Elements: From Memristors to Neurons and Synapses, The Journal of Physical Chemistry B 125, no.3535 (Aug 2021): 9934–9949.https://doi.org/10.1021/acs.jpcb.1c03905Uday S. Goteti, Ivan A. Zaluzhnyy, Shriram Ramanathan, Robert C. Dynes, Alex Frano Low-temperature emergent neuromorphic networks with correlated oxide devices, Proceedings of the National Academy of Sciences 118, no.3535 (Aug 2021).https://doi.org/10.1073/pnas.2103934118Ya Yang, Lichao Xiu, Guoming Yu Emotional Information in News Reporting on Audience Cognitive Processing in the Age of Posttruth: An Electroencephalogram and Functional Connectivity Approach, Frontiers in Psychology 12 (Aug 2021).https://doi.org/10.3389/fpsyg.2021.734147 Decoding of the Skin Sensations to the Low Intensive Electric and Magnetic Fields, Journal of Bioscience & Biomedical Engineering (Jan 2021).https://doi.org/10.47485/2693-2504.1032Ahmad Shalbaf, Sara Bagherzadeh, Arash Maghsoudi Transfer learning with deep convolutional neural network for automated detection of schizophrenia from EEG signals, Physical and Engineering Sciences in Medicine 43, no.44 (Sep 2020): 1229–1239.https://doi.org/10.1007/s13246-020-00925-9Jun Li, Yao-Hua Yang, Qi Chen, Wen-Qing Zhu, Jian-Hua Zhang Aqueous-solution-processed proton-conducting carbon nitride/polyvinylpyrrolidone composite electrolytes for low-power synaptic transistors with learning and memory functions, Journal of Materials Chemistry C 8, no.1212 (Jan 2020): 4065–4072.https://doi.org/10.1039/C9TC05765JFrancesco Di Ciò, Francesco Garaci, Silvia Minosse, Luca Passamonti, Alessio Martucci, Simona Lanzafame, Francesca Di Giuliano, Eliseo Picchi, Massimo Cesareo, Maria Giovanna Guerrisi, Roberto Floris, Carlo Nucci, Nicola Toschi Reorganization of the structural connectome in primary open angle Glaucoma, NeuroImage: Clinical 28 (Jan 2020): 102419.https://doi.org/10.1016/j.nicl.2020.102419Shu Lih Oh, Jahmunah Vicnesh, Edward J Ciaccio, Rajamanickam Yuvaraj, U Rajendra Acharya Deep Convolutional Neural Network Model for Automated Diagnosis of Schizophrenia Using EEG Signals, Applied Sciences 9, no.1414 (Jul 2019): 2870.https://doi.org/10.3390/app9142870Adolfo M. García, Ezequiel Mikulan, Agustín Ibáñez A neuroscientific toolkit for translation studies, (Sep 2016): 21–46.https://doi.org/10.1075/btl.128.02gar

The ripple pond: enabling spiking networks to see

We present the biologically inspired Ripple Pond Network (RPN), a simply connected spiking neural network which performs a transformation converting two dimensional images to one dimensional temporal patterns (TP) suitable for recognition by temporal coding learning and memory networks. The RPN has been developed as a hardware solution linking previously implemented neuromorphic vision and memory structures such as frameless vision sensors and neuromorphic temporal coding spiking neural networks. Working together such systems are potentially capable of delivering end-to-end high-speed, low-power and low-resolution recognition for mobile and autonomous applications where slow, highly sophisticated and power hungry signal processing solutions are ineffective. Key aspects in the proposed approach include utilizing the spatial properties of physically embedded neural networks and propagating waves of activity therein for information processing, using dimensional collapse of imagery information into amenable TP and the use of asynchronous frames for information binding.

VLSI implementation of a bio-inspired olfactory spiking neural network.

This paper presents a low-power, neuromorphic spiking neural network (SNN) chip that can be integrated in an electronic nose system to classify odor. The proposed SNN takes advantage of sub-threshold oscillation and onset-latency representation to reduce power consumption and chip area, providing a more distinct output for each odor input. The synaptic weights between the mitral and cortical cells are modified according to an spike-timing-dependent plasticity learning rule. During the experiment, the odor data are sampled by a commercial electronic nose (Cyranose 320) and are normalized before training and testing to ensure that the classification result is only caused by learning. Measurement results show that the circuit only consumed an average power of approximately 3.6 μW with a 1-V power supply to discriminate odor data. The SNN has either a high or low output response for a given input odor, making it easy to determine whether the circuit has made the correct decision. The measurement result of the SNN chip and some well-known algorithms (support vector machine and the K-nearest neighbor program) is compared to demonstrate the classification performance of the proposed SNN chip.The mean testing accuracy is 87.59% for the data used in this paper.

CrossNets: Neuromorphic Hybrid CMOS/Nanoelectronic Networks

Hybrid CMOS/nanoelectronic circuits, combining CMOS chips with simple nanoelectronic crossbar add-ons, may extend the exponential Moore-Law progress of microelectronics beyond the 10-nm frontier. This paper reviews the development of neuromorphic networks (“CrossNets”) based on this prospective technology. In these networks, the neural cell bodies (“somas”) are implemented in the CMOS subsystem, crossbar nanowires are used as axons and dendrites, while two-terminal crosspoint devices are used as elementary synapses. Extensive analysis and simulations have shown that such networks may perform virtually all information processing tasks demonstrated with software-implemented neural networks, with much higher performance. Estimates show that CrossNets may eventually overcome bio-cortical circuits in density, at comparable connectivity, while operating 4 to 6 orders of magnitude faster, at manageable power dissipation.

Real-time classification of datasets with hardware embedded neuromorphic neural networks

Neuromorphic artificial neural networks attempt to understand the essential computations that take place in the dense networks of interconnected neurons making up the central nervous systems in living creatures. This article demonstrates that artificial spiking neural networks--built to resemble the biological model--encoding information in the timing of single spikes, are capable of computing and learning clusters from realistic data. It shows how a spiking neural network based on spike-time coding can successfully perform unsupervised and supervised clustering on real-world data. A temporal encoding procedure of continuously valued data is developed, together with a hardware implementation oriented new learning rule set. Solutions that make use of embedded soft-core microcontrollers are investigated, to implement some of the most resource-consuming components of the artificial neural network. Details of the implementations are given, with benchmark application evaluation and test bench description. Measurement results are presented, showing real-time and adaptive data processing capabilities, comparing these to related findings in the specific literature.

Challenges for implementations of delay-coded neuromorphic neural networks on embedded digital hardware

Challenges for implementations of delay-coded neuromorphic neural networks on embedded digital hardware

STDP implementation using memristive nanodevice in CMOS-Nano neuromorphic networks

Implementation of a correlation-based learning rule, Spike-Timing-Dependent-Plasticity (STDP), for asynchronous neuromorphic networks is demonstrated using `memristive' nanodevice. STDP is performed using locally available information at the specific moment of time, for which mapping to crossbar-based CMOS-Nano architectures, such as CMOS-MOLecular (CMOL), is done rather easily. The learning method is dynamic and online in which the synaptic weights are modified based on neural activity. The performance of the proposed method is analyzed for specifically shaped spikes and simulation results are provided for a synapse with STDP properties.

Hybrid CMOS/Nanoelectronic Circuits: Opportunities and Challenges

This is a concise review of the recent work on devices, circuits and architectures for possible hybrid semiconductor/nanodevice integrated circuits based on nanowire crossbars, with similar, simple, two-terminal devices formed at each crosspoint. Special attention is given to the so-called CMOL' variety of the hybrids, in which the crossbar is connected to the underlying CMOS circuit, using an area-distributed, pin-based interface. Recent detailed studies have shown that digital hybrid circuits may extend the Moore's Law progress of integrated circuits for 10 to 15 years. Even more impressive, mixed-signal neuromorphic networks (CrossNets) may provide unparalleled performance for some important information processing tasks, and in future may become the first hardware basis for challenging the mammal cerebral cortex in density, far exceeding it in speed, at manageable power consumption. Recently, the hybrid circuit concept received a strong boost from the experimental demonstrations of reproducible crosspoint devices (latching switches) based on amorphous silicon and metal oxides, and of nanowire crossbars with 15-nm-scale half-pitch. However, CMOL technology still faces several significant challenges, briefly discussed in the last section.

Self-Assembled Nanowire Arrays of Metal–Insulator–Semiconductor Diodes Exhibiting S-Type Nonlinearity

We have electrochemically self-assembled regimented arrays of vertically standing nanowire metal-insulator-semiconductor diodes (MISD) embedded in an insulating matrix. Each diode is a 10-nm-diameter semiconductor (CdS) nanowire in contact with an underlying thin insulator ( ~20 nm thick), flanked by two metal electrodes. The density of these MISD devices exceeds 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">11</sup> /cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> . The dc current-voltage characteristic of several devices in parallel displays an S-type nonlinearity, typical of an MISD. The negative differential resistance associated with the S-type nonlinearity can have a peak to valley ratio as large as 19:1 at room temperature. Individually, such devices can be used as thyristors, logic switches, and high-frequency oscillators. Collectively, they constitute a system of interacting nonlinear elements that could realize a nanoelectronic neuromorphic network model proposed more than a decade ago.

Compact silicon neuron circuit with spiking and bursting behaviour

A silicon neuron circuit that produces spiking and bursting firing patterns, with biologically plausible spike shape, is presented. The circuit mimics the behaviour of known classes of cortical neurons: regular spiking (RS), fast spiking (FS), chattering (CH) and intrinsic bursting (IB). The paper describes the operation of the circuit, provides simulation results, a simplified analytical model, and a phase-plane analysis of its behaviour. The functionality of the circuit has been verified experimentally. The paper introduces a proof-of-concept analogue integrated circuit, implemented in a 0.35 microm CMOS technology, and presents preliminary measurement results. The neuron cell provides an area and energy efficient implementation of the silicon cortical neuron, and could be used as a universal neuron circuit in VLSI neuromorphic networks that closely resemble the circuits of the cortex.

Defect‐tolerant nanoelectronic pattern classifiers

AbstractMixed‐signal neuromorphic networks (‘CrossNets’), based on hybrid CMOS/nanodevice circuits, may provide unprecedented performance for important pattern classification tasks. The synaptic weights necessary for such tasks may be imported from an external ‘precursor’ network with either continuous or discrete synaptic weights (in the former case, with the quantization—‘clipping’—due to the binary character of the elementary synaptic nanodevices—latching switches.) Alternatively, the weights may be adjusted ‘in situ’ (inside the CrossNet) using a pseudo‐stochastic method, or set‐up using a mixed‐mode method partly employing external circuitry. Our calculations have shown that CrossNet pattern classifiers, using any of these synaptic weight adjustment methods, may be remarkably resilient. For example, in a CrossNet with synapses in the form of two small square arrays with 4 × 4 nanodevices each, the resulting weight discreteness may have a virtually negligible effect on the classification fidelity, while the fraction of defective devices which affects the performance substantially ranges from ∼20% to as high as 90% (!), depending on the training method. Copyright © 2007 John Wiley &amp; Sons, Ltd.

Defect-tolerant nanoelectronic pattern classifiers

Mixed-signal neuromorphic networks (CrossNets), based on hybrid CMOS/nanodevice circuits, may provide unprecedented performance for important pattern classification tasks. The synaptic weights nece...

CMOL: Second life for silicon?

This is a brief review of the recent work on the prospective hybrid CMOS/nanowire/nanodevice (“CMOL”) circuits including digital memories, reconfigurable Boolean-logic circuits, and mixed-signal neuromorphic networks. The basic idea of CMOL circuits is to combine the advantages of CMOS technology (including its flexibility and high fabrication yield) with the extremely high potential density of molecular-scale two-terminal nanodevices. Relatively large critical dimensions of CMOS components and the “bottom-up” approach to nanodevice fabrication may keep CMOL fabrication costs at affordable level. At the same time, the density of active devices in CMOL circuits may be as high as 1012cm2 and that they may provide an unparalleled information processing performance, up to 1020 operations per cm2 per second, at manageable power consumption.

Analogue CMOS prototype vision chip with fuzzy Kohonen network processing for grey level image segmentation

In this paper, the design and realization of an analogue CMOS prototype vision chip for grey level image segmentation for two regions is presented with Fuzzy-Kohonen Network processing as an alternative to neuromorphic and cellular neural networks approaches for vision chips. This alternative is due to the compensation that fuzziness gives in order to utilize simple analogue circuits with small layout areas instead of robust analogue circuits with bigger layout areas required. This helps to relieve the signal to noise ratio when crisp computations are present. The vision chip is based on the elaboration of an array of basic cells that work simultaneously. Every basic cell has embedded one vertical phototransistor and one network implemented under weak inversion techniques. The chip processes one image in 35 us with consumption of 733 uW per cell, signal to noise ratio of 43 dB and dynamic range of 49 dB.

An analytical approach to neuronal connectivity

This paper describes how realistic neuromorphic networks can have their connectivity properties fully characterized in analytical fashion. By assuming that all neurons have the same shape and are regularly distributed along the two-dimensional orthogonal lattice with parameter $\Delta$, it is possible to obtain the accurate number of connections and cycles of any length from the autoconvolution function as well as from the respective spectral density derived from the adjacency matrix. It is shown that neuronal shape plays an important role in defining the spatial spread of network connections. In addition, most such networks are characterized by the interesting phenomenon where the connections are progressively shifted along the spatial domain where the network is embedded. It is also shown that the number of cycles follows a power law with their respective length. Morphological measurements for characterization of the spatial distribution of connections, including the adjacency matrix spectral density and the lacunarity of the connections, are suggested. The potential of the proposed approach is illustrated with respect to digital images of real neuronal cells.