Bienenstock-Cooper-Munro Research Articles

An Electrolyte‐Gated InGaZnO Phototransistor that Emulates Visual Experience‐Dependent Plasticity

AbstractInspired by neurobiological learning rules, bionic devices that simulate the fundamental functions of synapses and neurons provide a highly effective approach to neuromorphic computing. Among various learning rules, the Bienenstock‐Cooper‐Munro (BCM) learning rule can explain the threshold sliding effect of synaptic weight modification in the visual cortex, which is difficult to explain with the classical Hebb's rule. Existing research mainly focuses on exploiting electrical stimulation to implement the BCM rule, while the optical implementation is still unexplored. In this paper, the light‐history‐dependent BCM learning rule is implemented with electrolyte‐gated InGaZnO (IGZO) transistors. The channel conductance can be modulated through light illumination and electrical stimulation. By utilizing the light‐history‐dependent property of the IGZO electrolyte‐gated transistor and following the triplet‐spike‐timing‐dependent plasticity (STDP) rules, the BCM learning rule is successfully emulated in a single device. Moreover, the light‐history‐dependent property enables a variety of bionic vision functions including image edge detection and associative memory. This work provides a paradigm for the novel implementation of the BCM rule and paves the way for further development of machine vision systems.

A Hybrid CMOS-Memristor Spiking Neural Network Supporting Multiple Learning Rules.

Artificial intelligence (AI) is changing the way computing is performed to cope with real-world, ill-defined tasks for which traditional algorithms fail. AI requires significant memory access, thus running into the von Neumann bottleneck when implemented in standard computing platforms. In this respect, low-latency energy-efficient in-memory computing can be achieved by exploiting emerging memristive devices, given their ability to emulate synaptic plasticity, which provides a path to design large-scale brain-inspired spiking neural networks (SNNs). Several plasticity rules have been described in the brain and their coexistence in the same network largely expands the computational capabilities of a given circuit. In this work, starting from the electrical characterization and modeling of the memristor device, we propose a neuro-synaptic architecture that co-integrates in a unique platform with a single type of synaptic device to implement two distinct learning rules, namely, the spike-timing-dependent plasticity (STDP) and the Bienenstock-Cooper-Munro (BCM). This architecture, by exploiting the aforementioned learning rules, successfully addressed two different tasks of unsupervised learning.

BCM‐Inspired Synapses Constructed with Barrier‐Modulated Coupling Junctions for Enhancing Speech Recognition

AbstractBio‐inspired synaptic devices have garnered considerable interest in neuromorphic computing. The Bienenstock‐Cooper‐Munro (BCM) learning rule stands out as one of the most accurate synaptic models, featuring non‐monotonic behavior and threshold sliding effect, crucial for stable learning processes. The direct device strategy for completely mimicking the BCM rule is a tough issue since the current devices lack two competitive working modes within one device. In this work, a dual‐junction synaptic device with opposite built‐in electric fields using a W/WO2/WO3‐x/Au structure is demonstrated. The devices directly mimic two fundamental features of the BCM rule via a delicately‐designed bandgap engineering strategy. Furthermore, the working mechanisms are investigated and the promising potential of dual‐junction synaptic devices is demonstrated for enhancing speech recognition through Convolutional Neural Network (CNN)‐based digital speech recognition with a remarkable accuracy of 98% through a synaptic array. Even for speech recognition with 13% Gaussian noise, the accuracy remained at 83%. These findings provide a promising strategy for developing BCM‐based synaptic devices for neuromorphic computing applications.

Evolving Dual-Threshold Bienenstock-Cooper-Munro Learning Rules in Echo State Networks.

The strengthening and the weakening of synaptic strength in existing Bienenstock-Cooper-Munro (BCM) learning rule are determined by a long-term potentiation (LTP) sliding modification threshold and the afferent synaptic activities. However, synaptic long-term depression (LTD) even affects low-active synapses during the induction of synaptic plasticity, which may lead to information loss. Biological experiments have found another LTD threshold that can induce either potentiation or depression or no change, even at the activated synapses. In addition, existing BCM learning rules can only select a set of fixed rule parameters, which is biologically implausible and practically inflexible to learn the structural information of input signals. In this article, an evolved dual-threshold BCM learning rule is proposed to regulate the reservoir internal connection weights of the echo-state-network (ESN), which can contribute to alleviating information loss and enhancing learning performance by introducing different optimal LTD thresholds for different postsynaptic neurons. Our experimental results show that the evolved dual-threshold BCM learning rule can result in the synergistic learning of different plasticity rules, effectively improving the learning performance of an ESN in comparison with existing neural plasticity learning rules and some state-of-the-art ESN variants on three widely used benchmark tasks and the prediction of an esterification process.

Oxide-Based Bionic Hetero-Dendritic Neuron with Capabilities of Bienenstock-Cooper-Munro Learning Activities

The close replication of synaptic function is critically important for achieving cognitive computing based on neuromorphic devices. As an advanced neural learning rule, Bienenstock–Cooper–Munro (BCM) learning rules have been attracted...

An Au25 nanocluster/MoS2 vdWaals heterojunction phototransistor for chromamorphic visual-afterimage emulation.

Color vision relies on three cone photoreceptors that are sensitive to different wavelengths of light. The interaction of three incident light wavelengths over time creates a fascinating color coupling perception, termed chromamorphic computing. However, the realization of this fascinating characteristic in semiconductor devices remains a great challenge. Herein, a mixed-dimensional optoelectronic transistor based on a novel metal nanocluster Au25(SC12H25)18 and two-dimensional MoS2 van der Waals (vdWaals) heterojunction is proposed for chromamorphic visual-afterimage emulation with red-green-blue three-color spatiotemporal coupling perception. This distinguished molecular-like electronic level of Au25 nanoclusters allows the transistor to have visible light-sensitive properties, endowing it with the ability to perceive color information. Moreover, the chromamorphic functions are realized using a color spatiotemporal coupling approach. By utilizing the photogating effect of light stimulus, the device exhibits visual experience-dependent plasticity in accordance with the Bienenstock-Cooper-Munro (BCM) learning rule. Most importantly, for the first time, intriguing visual afterimages could be implemented using a color sensitization approach based on a close relationship between visual persistence and negative afterimages. These results represent an important step towards a new generation of intelligent visual color perception systems for human-computer interaction, bionic robots, etc.

Circuit implementation of on-chip trainable spiking neural network using CMOS based memristive STDP synapses and LIF neurons

Computation on a large volume of data at high speed and low power requires energy-efficient architectures for edge computing applications. As a result, scientists focus on memristive circuits and systems for area and energy efficiency. Spiking neural network (SNN) with bio-inspired spike-timing-dependent plasticity learning (STDP) is a promising solution for energy-efficient neuromorphic systems than conventional artificial neural network (ANN). Previous works on SNN with STDP learning primarily use memristor macro models, which are software-based and cannot give complete insight into circuit implementation challenges. Some reported works on SNN use memristive devices, which require additional fabrication steps. This article presents a full circuit-level implementation of the SNN system featuring on-chip training and classification using memristive STDP synapse in standard CMOS technology. A new learning rule using the modified STDP is implemented to simplify the weight modification process. It does not involve FPGAs, CPUs, or GPUs to train the neural network. The approach used in this paper to modify the weights does not require any additional combinational or digital circuits attached to the memristive synapse resulting in less consumption of area, energy and time. We demonstrated the complete circuit-level design, implementation and simulation of SNN with on-chip training and pattern classification using 180 nm CMOS technology. A comprehensive comparison of the proposed SNN circuit with the previous related work is also presented. To demonstrate the versatility of the CMOS synapse circuit for application scenarios requiring rate-based learning, we have tuned the pair-based STDP circuit to obtain Bienenstock–Cooper–Munro (BCM) characteristics and applied it to heart rate classification.

Normal and abnormal BCM rules realized in BaTiO3/Nb:SrTiO3 heterojunction

Recently, the ferroelectric memristors have attracted much attention for neuromorphic computing and artificial intelligence because of fast reversal speed, excellent fatigue, nondestructive readout. In this paper, the basic functions of biological synapses including paired-pulse facilitation/depression (PPF/PPD), spike-rate/amplitude-dependent plasticity (SRDP/SADP) have been successfully simulated in BaTiO3/Nb:SrTiO3 epitaxial heterojunction. Furthermore, normal and abnormal Bienenstock-Cooper-Munro (BCM) learning rules with sliding frequency threshold have been found from SRDP under positive and negative bias, respectively. This provides support for encoding and deep learning in the application of neuromorphic computing.

Synaptic and Gradual Conductance Switching Behaviors in CeO2/Nb–SrTiO3 Heterojunction Memristors for Electrocardiogram Signal Recognition

The synaptic properties of memristors have been widely studied. However, researchers are still committed to solving various challenges, including the study of highly reliable memristors with comprehensive synaptic functions and memristors that simulate highly complex neurological learning rules. In this work, we report a CeO2/Nb-SrTiO3 heterojunction memristor whose conductance could be gradually tuned under both positive and negative pulse trains. Due to the gradual conductance switching behavior and the high switching ratio (105), the CeO2/Nb-SrTiO3 heterojunction memristor could dutifully mimic biosynaptic functions, including excitatory/inhibitory postsynaptic current (EPSC/IPSC), paired-pulse facilitation and depression (PPF/PPD), spike amplitude-dependent plasticity (SADP), spike duration-dependent plasticity (SDDP), spike rate-dependent plasticity (SRDP), paired/triplet spiking-time-dependent plasticity (STDP), and Bienenstock-Cooper-Munro (BCM) rules. Moreover, a convolutional neural network based on the memristors is constructed to identify the electrocardiogram (ECG) data sets to realize the diagnosis of diseases with a recognition accuracy of 93%. Besides, the recognition accuracy of the handwriting digit reaches 96%. These studies broaden the research scope of high-level synaptic behavior and lay a foundation for the future full synaptic memristor networks.

Tuning Bienenstock-Cooper-Munro learning rules in a two-terminal memristor for neuromorphic computing.

In memristors, the implementation of the Bienenstock-Cooper-Munro (BCM) learning rule plays a significant role in the modulation balance of artificial synapses and the reduction of energy consumption owing to their sliding frequency threshold. At present, the BCM learning rule is mostly achieved by adjusting gating voltage or channel current in field effect transistors. However, owing to the lack of the tunable degrees of freedom, the progress of two-terminal memristors is limited to simulating the BCM learning rule. In this study, by adjusting the series resistance, three types of BCM-like learning rules are found in a two-terminal BaTiO3 memristor. Specifically, the abnormal BCM learning rule with high-frequency depression and low-frequency potentiation is obtained for a small series resistance, the monotonous BCM learning rule with high-frequency potentiation and low-frequency depression is achieved for a large series resistance, and the type of BCM learning rule with the enhanced depression effect is obtained for a moderate series resistance. These three BCM learning rules are related to the non-monotonous conductance modulation caused by the migration of ionized oxygen vacancies and are proved by X-ray photoelectron spectroscopy. Moreover, spike rate-dependent plasticity (SRDP) and history-dependent plasticity are achieved. This study offers promising prospects for neuromorphic computing.

BCM Learning Rules Emulated by a-IGZO-Based Photoelectronic Neuromorphic Transistors

Hardware implementation of Bienenstock–Cooper–Munro (BCM) learning rules would be of great implications toward artificial intelligent systems. In this work, amorphous indium gallium zinc oxide (a-IGZO)-based photoelectronic neuromorphic transistors were proposed for mimicking BCM learning rules. A SiO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> electrolyte film with a large electric-double-layer capacitance ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.33~\mu \text{F}$ </tex-math></inline-formula> /cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) was used for the gate dielectric film. Light induced short-term synaptic plasticity can be mimicked by such device, including excitatory postsynaptic current (EPSC), paired-pulse facilitation (PPF), and high-pass temporal filtering. More importantly, BCM learning rules are realized based on this photoelectronic neuromorphic transistor. These results would provide a step forward the development of photoelectronic neuromorphic systems with sophisticated learning rules.

Weight dependence in BCM leads to adjustable synaptic competition

Models of synaptic plasticity have been used to better understand neural development as well as learning and memory. One prominent classic model is the Bienenstock-Cooper-Munro (BCM) model that has been particularly successful in explaining plasticity of the visual cortex. Here, in an effort to include more biophysical detail in the BCM model, we incorporate 1) feedforward inhibition, and 2) the experimental observation that large synapses are relatively harder to potentiate than weak ones, while synaptic depression is proportional to the synaptic strength. These modifications change the outcome of unsupervised plasticity under the BCM model. The amount of feed-forward inhibition adds a parameter to BCM that turns out to determine the strength of competition. In the limit of strong inhibition the learning outcome is identical to standard BCM and the neuron becomes selective to one stimulus only (winner-take-all). For smaller values of inhibition, competition is weaker and the receptive fields are less selective. However, both BCM variants can yield realistic receptive fields.

Effectiveness of Biologically Inspired Neural Network Models in Learning and Patterns Memorization.

Purpose: In this work, we propose an implementation of the Bienenstock–Cooper–Munro (BCM) model, obtained by a combination of the classical framework and modern deep learning methodologies. The BCM model remains one of the most promising approaches to modeling the synaptic plasticity of neurons, but its application has remained mainly confined to neuroscience simulations and few applications in data science. Methods: To improve the convergence efficiency of the BCM model, we combine the original plasticity rule with the optimization tools of modern deep learning. By numerical simulation on standard benchmark datasets, we prove the efficiency of the BCM model in learning, memorization capacity, and feature extraction. Results: In all the numerical simulations, the visualization of neuronal synaptic weights confirms the memorization of human-interpretable subsets of patterns. We numerically prove that the selectivity obtained by BCM neurons is indicative of an internal feature extraction procedure, useful for patterns clustering and classification. The introduction of competitiveness between neurons in the same BCM network allows the network to modulate the memorization capacity of the model and the consequent model selectivity. Conclusions: The proposed improvements make the BCM model a suitable alternative to standard machine learning techniques for both feature selection and classification tasks.

Constraint-induced intervention as an emergent phenomenon from synaptic competition in biological systems

The principle of constraint-induced therapy is widely practiced in rehabilitation. In hemiplegic cerebral palsy (CP) with impaired contralateral corticospinal projection due to unilateral injury, function improves after imposing a temporary constraint on limbs from the less affected hemisphere. This type of partially-reversible impairment in motor control by early brain injury bears a resemblance to the experience-dependent plastic acquisition and modification of neuronal response selectivity in the visual cortex. Previously, such mechanism was modeled within the framework of BCM (Bienenstock-Cooper-Munro) theory, a rate-based synaptic modification theory. Here, we demonstrate a minimally complex yet sufficient neural network model which provides a fundamental explanation for inter-hemispheric competition using a simplified spike-based model of information transmission and plasticity. We emulate the restoration of function in hemiplegic CP by simulating the competition between cells of the ipsilateral and contralateral corticospinal tracts. We use a high-speed hardware neural simulation to provide realistic numbers of spikes and realistic magnitudes of synaptic modification. We demonstrate that the phenomenon of constraint-induced partial reversal of hemiplegia can be modeled by simplified neural descending tracts with 2 layers of spiking neurons and synapses with spike-timing-dependent plasticity (STDP). We further demonstrate that persistent hemiplegia following unilateral cortical inactivation or deprivation is predicted by the STDP-based model but is inconsistent with BCM model. Although our model is a highly simplified and limited representation of the corticospinal system, it offers an explanation of how constraint as an intervention can help the system to escape from a suboptimal solution. This is a display of an emergent phenomenon from the synaptic competition.

Bienenstock-Cooper-Munro Learning Rule Realized in Polysaccharide-Gated Synaptic Transistors with Tunable Threshold.

With reference to the organization of the human brain nervous system, a hardware-based approach that builds massively parallel neuromorphic circuits is of great significance to neuromorphic computing. The Bienenstock-Cooper-Munro (BCM) learning rule, which describes that the synaptic weight modulation exhibits frequency-dependent and tunable frequency threshold characteristics, is more compatible with the working principle of neuromorphic computing systems than spike-timing-dependent plasticity. Therefore, it is interesting to simulate the BCM learning rule on solid-state synaptic devices. Here, we have prepared λ-carrageenan (λ-car) electrolyte-gated oxide synaptic transistors, which exhibit good transistor performances, including a low subthreshold swing of 125 mV/dec, an on/off ratio larger than 106, and a mobility of 9.5 cm2 V-1 s-1. By modulating the initial channel current and spike frequency, the simulation of the BCM rule was successfully realized. The competitive relationship between the drift of protons under an electric field and the spontaneous diffusion of protons can explain this mechanism. The proposed λ-car-gated synaptic transistor has a great significance to neuromorphic computing.

Fault diagnosis model based on Granular Computing and Echo State Network

In order to improve the efficiency and accuracy of electronic equipment fault diagnosis, a fault diagnosis model based on Granular Computing and Echo State Network (ESN) is proposed. Firstly, the attribute reduction of test index is carried out based on granular computing model. An attribute distinguishing ability index is defined based on attribute value influence degree. As the basis of similarity measure, a number of attribute granules of similar distinguish are obtained through affinity propagation clustering algorithm, then fault attribute reduction was completed by selecting clustering center attributes. In the stage of fault identification by ESN, in order to improve the dynamic adaptability of ESN reservoir to samples, Bienenstock–Cooper–Munro(BCM) rule is introduced into the reservoir construction to train the connection weight matrix. Meanwhile, the L1∕2-norm penalty term is added to the objective function in order to improve the sparsification efficiency, and a smoothing L1∕2-norm regularization term is used to overcome the iterative numerical oscillation problem, the model is solved by using the half threshold iteration method at last. The effectiveness and superiority of the proposed method are verified by a fault diagnosis example of terminal guidance radar signal processing module.

Effects of HCN2 Mutations on Dendritic Excitability and Synaptic Plasticity: A Computational Study

Several reports of augmented hyperpolarisation-activated cyclic nucleotide-gated (HCN) currents in seizures have suggested a pro-convulsive identity for HCN channels. The mutations identified in one or more of the four HCN channel subunits are found to be contributing to different epileptic phenotypes. S126L, S632W, V246M and E515K are four different mutations affecting the HCN2 subunit and have been reported in febrile seizures and partial/generalised idiopathic epilepsies. From the visible outcomes in subjects with these mutations, it is evident that they must play important roles in altering dendritic excitability. Through this simulation study using NEURON, we created a three-compartmental, hippocampal CA1 pyramidal neuron synapse model expressing seven different ion channels (fast sodium (NaF), T-type calcium (CaT), R-type calcium (CaR), delayed rectifier potassium (KDR), A-type potassium (KA), small conductance potassium (SK), and HCN channels) and two glutamate receptors (AMPAR and NMDAR). We modelled an HCN2 channel and incorporated changes in it to obtain mutation kinetics. Their effects on excitability were studied by observing resting membrane potentials, input resistances and plasticity profiles for measuring the sliding modification threshold (SMT) of Bienenstock–Cooper–Munro (BCM) theory. Virtual knockouts of ion channels other than HCN were also performed to assess their role in altering excitability when they act alongside HCN2 mutations. Our results show that HCN2 mutations can potentially be a primary causative factor for excessive action potential firing through their effect on resting membrane potentials and input resistance.

Commentary: Cumulative effects of anodal and priming cathodal tDCS on pegboard test performance and motor cortical excitability.

Commentary: Cumulative effects of anodal and priming cathodal tDCS on pegboard test performance and motor cortical excitability.

Computational modeling of heterosynaptic plasticity in the hippocampus

Hippocampal long-term potentiation (LTP) and long-term depression (LTD) are central synaptic mechanisms of learning and memory. Here we use compartmental models of hippocampal granule cells to better understand LTP and heterosynaptic LTD which have been reported in the dentate gyrus of awake rats [1]. Our simulations indicate that LTP and heterosynaptic LTD can be explained by a spike-timing-dependent plasticity (STDP) rule combined with a fast Bienenstock-Cooper-Munro (BCM)-like metaplasticity rule [2-5]. We study the interaction between these plasticity rules and ongoing pre- and postsynaptic activity. Our models are able to account for the experimentally observed degree of LTP and heterosynaptic LTD induced by various plasticity-inducing protocols.

Modeling neural plasticity in echo state networks for classification and regression

Echo state networks (ESNs) are one of two major neural network models belonging to the reservoir computing framework. Traditionally, only the weights connecting to the output neuron, termed read-out weights, are trained using a supervised learning algorithm, while the weights inside the reservoir of the ESN are randomly determined and remain unchanged during the training. In this paper, we investigate the influence of neural plasticity applied to the weights inside the reservoir on the learning performance of the ESN. We examine the influence of two plasticity rules, anti-Oja's learning rule and the Bienenstock–Cooper–Munro (BCM) learning rule on the prediction and classification performance when either offline or online supervised learning algorithms are employed for training the read-out connections. Empirical studies are conducted on two widely used classification tasks and two time series prediction problems. Our experimental results demonstrate that neural plasticity can more effectively enhance the learning performance when offline learning is applied. The results also indicate that the BCM rule outperforms the anti-Oja rule in improving the learning performance of the ENS in the offline learning mode.