Intrinsic Plasticity Mechanisms Research Articles

Fast and robust learning in Spiking Feed-forward Neural Networks based on Intrinsic Plasticity mechanism

In this paper, the computational performance of a Spiking Feed-forward Neural Network (SFNN) is investigated based on a brain-inspired Intrinsic Plasticity (IP) mechanism, which is a membrane potential adaptive tuning scheme used to change the intrinsic excitability of individual neuron. This learning rule has the ability of regulating neural activity in a relative homeostatic level even if the external input of a neuron is extremely low or extremely high. The effectiveness of IP on SFNN model has been studied and evaluated through the MNIST handwritten digits classification. The training of network weights starts from a conventional artificial neural network by backpropagation and then the rate-based neurons are transformed into spiking neuron models with IP learning. Our results show that both over-activation and under-activation of neuronal response which commonly exist during the computation of neural networks can be effectively avoided. Without loss of accuracy, the calculation speed of SFNN with IP learning is extremely higher than that of the other models. Besides, when the input intensity and data noise are taken into account, both of the learning speed and accuracy of the model can be greatly improved by the application of IP learning. This biologically inspired SFNN model is simple and effective which may give insights for the optimization of neural computation.

Sparseness, antisparseness and anything in between: the operating point of a neuron determines its computational repertoire.

A recent model of intrinsic plasticity coupled to Hebbian synaptic plasticity proposes that adaptation of a neuron's threshold and gain in a sigmoidal response function to achieve a sparse, exponential output firing rate distribution facilitates the discovery of heavy-tailed or super- gaussian sources in the neuron's inputs. We show that the exponential output distribution is irrelevant to these dynamics and that, furthermore, while sparseness is sufficient, it is not necessary. The intrinsic plasticity mechanism drives the neuron's threshold large and positive, and we prove that in such a regime, the neuron will find supergaussian sources; equally, however, if the threshold is large and negative (an antisparse regime), it will also find supergaussian sources. Away from such extremes, the neuron can also discover subgaussian sources. By examining a neuron with a fixed sigmoidal nonlinearity and considering the synaptic strength fixed-point structure in the two-dimensional parameter space defined by the neuron's threshold and gain, we show that this space is carved up into sub- and supergaussian-input-finding regimes, possibly with regimes of simultaneous stability of sub- and supergaussian sources or regimes of instability of all sources; a single gaussian source may also be stabilized by the presence of a nongaussian source. A neuron's operating point (essentially its threshold and gain coupled with its input statistics) therefore critically determines its computational repertoire. Intrinsic plasticity mechanisms induce trajectories in this parameter space but do not fundamentally modify it. Unless the trajectories cross critical boundaries in this space, intrinsic plasticity is irrelevant and the neuron's nonlinearity may be frozen with identical receptive field refinement dynamics.

Adaptive connectionist systems for engineering applications

Adaptive Connectionist Systems refer to systems that change with time through interaction with the environment, learning through the utilization of artificial neural networks (ANN). Many engineering applications require the use of real-time decision-making and adaptation to new input. In this context a number of researchers have explored the use of adaptive connectionist systems and their applicability to solve engineering problems. This special issue includes selected contributions from the 13th Engineering Applications of Neural Network conference held in London, UK in 2012 (EANN 2012). The EANN conferences promote neural networks and associated techniques and the significant benefits that can be derived from their use. The conferences are not only for reporting advances, but also for showing how neural networks provide practical solutions in a wide range of applications. The first paper ‘‘Neurocomputing techniques to dynamically forecast spatiotemporal air pollution data’’ by Antonios Papaleonidas and Lazaros Iliadis addresses the application of ANNs to the problem of forecasting dangerous levels of Ozone (O3) concentrations in the city center of Athens and more specifically in the ‘‘Athinas’’ air quality monitoring station. While most existing approaches develop static ANN models optimal only under specific conditions, this work suggests a dynamic flexible and adjustable neural spatiotemporal model. The proposed approach takes in account not only the values of the primitive air pollutants and meteorological measurements, but it is also evaluates the influence of the existing O3 values in monitoring stations close to the one under examination. The results suggest that the optimal ANN model is using data from a small number of measurement’s stations and it is capable of estimating O3 concentrations in real time mod. It also has the capacity of forecasting the same values for future time intervals. The second paper ‘‘Information dynamics based selfadaptive reservoir for delay temporal memory tasks’’ by Sakyasingha Dasgupta, Florentin Worgotter and Poramate Manoonpong, investigates the application of recurrent neural networks of the reservoir computing (RC) type for time-series processing tasks and more specifically for delayed response tasks involving the transient memorization of information (temporal memory). The authors combine a generalized intrinsic plasticity rule with a local information dynamics based schema of reservoir neuron leak adaptation. Utilizing this combination they propose a self-adaptive reservoir (RNN) framework that maintains homeostasis through the generic intrinsic plasticity mechanism. The suggested framework has been evaluated on two standard benchmark tasks and the results demonstrate that the adaptation mechanism clearly outperforms static reservoirs. The authors also demonstrate the application of the self-adaptive reservoir (RNN) to the control of autonomous robotic agents through maze navigation experiments. The results show that the proposed network is effective for reconstructing the original trajectories and has the ability to cope with the variable temporal delay memory problem. The third paper ‘‘Exploratory analysis of mobile phone traffic patterns using 1-dimensional Self Organizing Map (SOM), clustering and anomaly detection’’ by Pekka Kumpulainen, considers the problem of recognizing traffic patterns in continuously evolving mobile networks. This is C. Jayne (&) Faculty of Engineering and Computing, Coventry University, Room EC306, Coventry CV1 5FB, UK e-mail: ab1527@coventry.ac.uk; Chrisina.Jayne2@coventry.ac.uk

Syntax processing properties of generic cortical circuits

Higher cognitive functioning is assumed to be largely symbolic/representational and compositional in nature. At various processing stages, from perceptual to motor, discrete structural elements with intricate temporal dependencies are combined into increasingly complex constructs [1]. Mapping such complex computational processes to the underlying neuronal infrastructure and assessing the properties of the neuronal system responsible for their implementation is not straightforward, but it is likely to yield important insights into the nature of neural computation. In order to address these issues, we adopt ideas and formalisms developed by theoretical linguistics to study the nature of rule-like or compositional behavior in the language domain, namely the acquisition of formal (artificial) grammars. The Artificial Grammar Learning (AGL) paradigm has a long tradition in psycholinguistic research (see, e.g. [2] for an overview), as a means to study the nature of syntactic processing and implicit sequence learning. With mere exposure and without performance feedback, human beings implicitly acquire knowledge about the structural regularities implemented by complex rule systems. In this work, we investigate to which extent generic cortical microcircuits can support formally explicit symbolic computations, instantiated by the same grammars used in the human AGL literature and implementing various types of local and non-adjacent dependencies between the sequence elements, thus requiring varying degrees of computational complexity and online processing memory to be adequately learned. We use concrete implementations of input-driven recurrent networks composed of noisy, spiking neurons, built according to the reservoir computing framework and dynamically shaped by a variety of synaptic and intrinsic plasticity mechanisms operating concomitantly [3]. Additionally, we compare supervised and unsupervised learning rules for the decoding algorithms, with varying degrees of biological plausibility. We show that, when shaped by plasticity, these models are capable of acquiring the structure of simple (regular) grammars. When asked to judge string legality (in a manner similar to human subjects), the networks perform at a qualitatively comparable level. We uncover which plasticity mechanisms are crucial for the task, with the aim of specifying a minimal model. Furthermore, the capability of the networks to process (bounded) recursive constructions including multiple patterns of non-adjacent dependencies accurately reflects recent results of human performance, highlighting inherent limitations imposed by the nature of neuronal processing.

Regional and Temporal Specificity of Intrinsic Plasticity Mechanisms in Rodent Primary Visual Cortex

Different neocortical regions are functionally specialized, but whether this specialization is reflected in the forms of plasticity present during developmental critical periods (CPs) is largely unknown. In rodent visual cortex, we recently showed that a form of intrinsic plasticity [LTP of intrinsic excitability (LTP-IE)] in the monocular region of the primary visual cortex (V1M) plays an important role in modulating cortical responsiveness following visual deprivation. Here we ask whether LTP-IE is present and similarly regulated by visual experience in the binocular region of the primary visual cortex (V1B), where inputs from the two eyes compete during the CP. In contrast to V1M, where LTP-IE is present throughout the CP, in V1B LTP-IE was only transiently expressed at the onset of the CP. Also distinct from V1M, brief monocular deprivation (MD) was unable to modulate LTP-IE magnitude in V1B, and even binocular deprivation (the equivalent of MD in V1M) could only influence LTP-IE expression during a narrow time window at the peak of the CP. Finally, we asked whether these differences depend on differences in sensory activation of the two areas during development. MD of ipsilateral inputs from before eye opening (to reduce competitive interactions) did not affect the pattern of LTP-IE expression in V1B. Further, the differences in plasticity in the two cortical areas persisted when animals were reared in the dark to remove all patterned visual input. Thus neocortical LTP-IE expression shows dramatic regional and temporal differentiation, and these differences are not driven by differences in sensory experience.

A Model of Neuronal Intrinsic Plasticity

Recent experimental results have accumulated evidence that the neurons can change their response characteristics to adapt to the variations of the synaptic inputs, which is the so-called neuronal intrinsic plasticity mechanism. In this paper, we present a new model on neuronal intrinsic plasticity. We first show that the probability distribution of the neuronal firing rates is more suitable to be represented as a Weibull distribution than an exponential distribution. Then, we derive the intrinsic plasticity model based on information theory. This study provides a more realistic model for further research on the effects of intrinsic plasticity on various brain functions and dynamics.

Cellular Plasticity for Group I mGluR-Mediated Epileptogenesis

Stimulation of group I metabotropic glutamate receptors (mGluRs) by the agonist (S)-dihydroxyphenylglycine in the hippocampus transforms normal neuronal activity into prolonged epileptiform discharges. The conversion is long lasting in that epileptiform discharges persist after washout of the inducing agonist and serves as a model of epileptogenesis. The group I mGluR model of epileptogenesis took on special significance because epilepsy associated with fragile X syndrome (FXS) may be caused by excessive group I mGluR signaling. At present, the plasticity mechanism underlying the group I mGluR-mediated epileptogenesis is unknown. I(mGluR(V)), a voltage-gated cationic current activated by group I mGluR agonists in CA3 pyramidal cells in the hippocampus, is a possible candidate. I(mGluR(V)) activation is associated with group I mGluR agonist-elicited epileptiform discharges. For I(mGluR(V)) to play a role in epileptogenesis, long-term activation of the current must occur after group I mGluR agonist exposure or synaptic stimulation. We observed that I(mGluR(V)), once induced by group I mGluR agonist stimulation in CA3 pyramidal cells, remained undiminished for hours after agonist washout. In slices prepared from FXS model mice, repeated stimulation of recurrent CA3 pyramidal cell synapses, effective in eliciting mGluR-mediated epileptiform discharges, also induced long-lasting I(mGluR(V)) in CA3 pyramidal cells. Similar to group I mGluR-mediated prolonged epileptiform discharges, persistent I(mGluR(V)) was no longer observed in preparations pretreated with inhibitors of tyrosine kinase, of extracellular signal-regulated kinase 1/2, or of mRNA protein synthesis. The results indicate that I(mGluR(V)) is an intrinsic plasticity mechanism associated with group I mGluR-mediated epileptogenesis.