Neuron-like Elements Research Articles

Grazing Bifurcations, Homoclinic Orbits and Chaos in a Ring Network of Linear Neuron-Like Elements with a Single Nonmonotonic Piecewise Constant Output Function

The bifurcations and chaos of a ring of three unidirectionally coupled neuron-like elements are examined as a minimal chaotic neural network. The output function of one neuron is nonmonotonic and piecewise constant while those of the other two neurons are linear. Two kinds of nonmonotonic output functions are considered and it is shown that periodic solutions undergo grazing bifurcations owing to discontinuity in the nonmonotonic functions. Chaotic attractors are created directly through a grazing bifurcation and homoclinic orbits based at pseudo steady states are generated. It is shown that homoclinic/heteroclinic orbits satisfying the condition of Shil’nikov chaos are caused by overshoot in the nonmonotonic functions.

Minimal Chaotic Networks of Linear Neuron-Like Elements with Single Rectification: Three Prototypes

Chaotic oscillations induced by single rectification in networks of linear neuron-like elements are examined on three prototype models: one nonautonomous system and two autonomous systems. The first is a system of coupled neurons with periodic input; the second is a system of three coupled neurons with six couplings; the third is a ring of four unidirectionally coupled neurons with one reverse coupling. In each system, the output function of one neuron is ramp and that of the others is linear. Each system is piecewise linear and the phase space is separated into two domains by a single border. Steady states, periodic solutions and homoclinic orbits are derived rigorously and their stability is evaluated with the eigenvalues of the Jacobian matrices. The bifurcation analysis of the three systems shows that chaotic attractors could be generated through cascades of period-doubling bifurcations of periodic solutions.

Fixed-Time Stable Neurodynamic Flow to Sparse Signal Recovery via Nonconvex L1-β2-Norm.

This letter develops a novel fixed-time stable neurodynamic flow (FTSNF) implemented in a dynamical system for solving the nonconvex, nonsmooth model L1-β2, β∈[0,1] to recover a sparse signal. FTSNF is composed of many neuron-like elements running in parallel. It is very efficient and has provable fixed-time convergence. First, a closed-form solution of the proximal operator to model L1-β2, β∈[0,1] is presented based on the classic soft thresholding of the L1-norm. Next, the proposed FTSNF is proven to have a fixed-time convergence property without additional assumptions on the convexity and strong monotonicity of the objective functions. In addition, we show that FTSNF can be transformed into other proximal neurodynamic flows that have exponential and finite-time convergence properties. The simulation results of sparse signal recovery verify the effectiveness and superiority of the proposed FTSNF.

Superconducting neural networks with disordered Josephson junction array synaptic networks and leaky integrate-and-fire loop neurons

Fully coupled randomly disordered recurrent superconducting networks with additional open-ended channels for inputs and outputs are considered the basis to introduce a new architecture to neuromorphic computing in this work. Various building blocks of such a network are designed around disordered array synaptic networks using superconducting devices and circuits as an example, while emphasizing that a similar architectural approach may be compatible with several other materials and devices. A multiply coupled (interconnected) disordered array of superconducting loops containing Josephson junctions [equivalent to superconducting quantum interference devices (SQUIDs)] forms the aforementioned collective synaptic network that forms a fully recurrent network together with compatible neuron-like elements and feedback loops, enabling unsupervised learning. This approach aims to take advantage of superior power efficiency, propagation speed, and synchronizability of a small world or a random network over an ordered/regular network. Additionally, it offers a significant factor of increase in scalability. A compatible leaky integrate-and-fire neuron made of superconducting loops with Josephson junctions is presented, along with circuit components for feedback loops as needed to complete the recurrent network. Several of these individual disordered array neural networks can further be coupled together in a similarly disordered way to form a hierarchical architecture of recurrent neural networks that is often suggested as similar to a biological brain.

Criticality or Supersymmetry Breaking?

In many stochastic dynamical systems, ordinary chaotic behavior is preceded by a full-dimensional phase that exhibits 1/f-type power spectra and/or scale-free statistics of (anti)instantons such as neuroavalanches, earthquakes, etc. In contrast with the phenomenological concept of self-organized criticality, the recently found approximation-free supersymmetric theory of stochastics (STS) identifies this phase as the noise-induced chaos (N-phase), i.e., the phase where the topological supersymmetry pertaining to all stochastic dynamical systems is broken spontaneously by the condensation of the noise-induced (anti)instantons. Here, we support this picture in the context of neurodynamics. We study a 1D chain of neuron-like elements and find that the dynamics in the N-phase is indeed featured by positive stochastic Lyapunov exponents and dominated by (anti)instantonic processes of (creation) annihilation of kinks and antikinks, which can be viewed as predecessors of boundaries of neuroavalanches. We also construct the phase diagram of emulated stochastic neurodynamics on Spikey neuromorphic hardware and demonstrate that the width of the N-phase vanishes in the deterministic limit in accordance with STS. As a first result of the application of STS to neurodynamics comes the conclusion that a conscious brain can reside only in the N-phase.

Synchronization in coupled neural network with inhibitory coupling

A theoretical model of a network of neuron-like elements was constructed. The network included several subnetworks. The first subnetwork was used to translate a constant-amplitude signal into a spike sequence (conversion of amplitude to frequency). A similar process occurs in the brain when perceiving visual information. With an increase in the flow of information, the generation frequency of the neural ensemble participating in the processing increases. Further, the first subnetwork transmitted excitation to two large interconnected subnetworks. These subnetworks simulated the dynamics of the cortical neuronal populations. It was shown that in the presence of inhibitory coupling, the neuronal ensembles demonstrate antiphase dynamics. Various connectivity topologies and various types of neuron-like oscillators were investigated. We compare the results obtained in a discrete neuron model (Rulkov model) and a continuous-time model (Hodgkin-Huxley). It is shown that in the case of a discrete neuron model, the periodic dynamics is manifested in the alternate excitation of various neural ensembles. In the case of the continuous-time model, periodic modulation of the synchronization index of neural ensembles is observed.

The dynamics of ensemble of neuron-like elements with excitatory couplings

We investigated the phenomenological model of ensemble of two FitzHugh–Nagumo neuron-like elements with symmetric excitatory couplings. The main advantage of proposed model is the new approach to model the coupling which is implemented by smooth function that approximates rectangular function and reflects main important properties of biological synaptic coupling. The proposed coupling depends on three parameters that define: a) the beginning of activation of an element α, b) the duration of the activation δ and c) the strength of the coupling g. We observed a rich diversity of different types of neuron-like activity, including regular in-phase, anti-phase and sequential spiking. In the phase space of the system, these regular regimes correspond to specific asymptotically stable periodic motions (limit cycles). We also observed the canard in-phase solutions and the chaotic anti-phase activity, which corresponds to a strange attractor that appears via the cascade of period doubling bifurcations of limit cycles.In addition, we investigated an interesting phenomenon when two different chaotic attractive regimes corresponding for two different types of chaotic anti-phase activity merge in a single strange attractor. As a result, a new type of chaotic anti-phase regime appears by explosion from the collision of these two strange attractors.We also provided the detailed study of bifurcations which lead to the transitions between all these regimes. We detected on the (α, δ) parameter plane regions that correspond to the above-mentioned regimes. We also showed numerically the existence of bistability regions where various non-trivial regimes coexist. For example, in some regions, one can observe either anti-phase or in-phase oscillations depending on initial conditions. We also specified regions corresponding to coexisting various types of sequential activity.

On born of in-phase limit cycle in ensemble of excitatory coupled FitzHugh-Nagumo elements

We proposed and studied numerically efficient phenomenological model of ensemble of two FitzHugh-Nagumo neuron-like elements that are coupled by symmetric synaptic excitatory coupling. This coupling is defined by function that depends on phase of active element and that is smooth approximation of rectangular impulse function. Above-mentioned coupling depends on three parameters that define the beginning of element activation, the duration of the activation and the coupling strength. We show analytically that in the phase space of the model there exists stable in-phase limit cycle that corresponds to regular oscillations with equal phases and frequencies of elements. It is proved that this limit cycle is a result of supercritical Andronov-Hopf bifurcation. The chart of activity regimes is depicted on the plane of parameters that define beginning and duration of activation. The boundaries of bifurcations that lead to birth of this cycle are found.

On born of in-phase limit cycle in ensemble of excitatory coupled FitzHugh-Nagumo elements

We proposed and studied numerically efficient phenomenological model of ensemble of two FitzHugh-Nagumo neuron-like elements that are coupled by symmetric synaptic excitatory coupling. This coupling is defined by function that depends on phase of active element and that is smooth approximation of rectangular impulse function. Above-mentioned coupling depends on three parameters that define the beginning of element activation, the duration of the activation and the coupling strength. We show analytically that in the phase space of the model there exists stable in-phase limit cycle that corresponds to regular oscillations with equal phases and frequencies of elements. It is proved that this limit cycle is a result of supercritical Andronov-Hopf bifurcation. The chart of activity regimes is depicted on the plane of parameters that define beginning and duration of activation. The boundaries of bifurcations that lead to birth of this cycle are found.

High-frequency forced oscillations in neuronlike elements.

We analyzed a generic relaxation oscillator under moderately strong forcing at a frequency much greater that the natural intrinsic frequency of the oscillator. Additionally, the forcing is of the same sign and, thus, has a nonzero average, matching neuroscience applications. We found that, first, the transition to high-frequency synchronous oscillations occurs mostly through periodic solutions with virtually no chaotic regimes present. Second, the amplitude of the high-frequency oscillations is large, suggesting an important role for these oscillations in applications. Third, the 1:1 synchronized solution may lose stability, and, contrary to other cases, this occurs at smaller, but not at higher frequency differences between intrinsic and forcing oscillations. We analytically built a map that gives an explanation of these properties. Thus, we found a way to substantially "overclock" the oscillator with only a moderately strong external force. Interestingly, in application to neuroscience, both excitatory and inhibitory inputs can force the high-frequency oscillations.

Влияние электрической связи на динамику ансамбля нейроноподобных элементов с синаптическими тормозящими связями

Влияние электрической связи на динамику ансамбля нейроноподобных элементов с синаптическими тормозящими связями

Sequential dynamics in the motif of excitatory coupled elements

In this article a new model of motif (small ensemble) of neuron-like elements is proposed. It is built with the use of the generalized Lotka–Volterra model with excitatory couplings. The main motivation for this work comes from the problems of neuroscience where excitatory couplings are proved to be the predominant type of interaction between neurons of the brain. In this paper it is shown that there are two modes depending on the type of coupling between the elements: the mode with a stable heteroclinic cycle and the mode with a stable limit cycle. Our second goal is to examine the chaotic dynamics of the generalized three-dimensional Lotka–Volterra model.

Machine learning for neuroscience

What is machine learning? Machine learning is a type of statistics that places particular emphasis on the use of advanced computational algorithms. As computers become more powerful, and modern experimental methods in areas such as imaging generate vast bodies of data, machine learning is becoming ever more important for extracting reliable and meaningful relationships and for making accurate predictions. Key strands of modern machine learning grew out of attempts to understand how large numbers of interconnected, more or less neuron-like elements could learn to achieve behaviourally meaningful computations and to extract useful features from images or sound waves. By the 1990s, key approaches had converged on an elegant framework called ‘graphical models’, explained in Koller and Friedman, in which the nodes of a graph represent variables such as edges and corners in an image, or phonemes and words in speech. The probabilistic relationships between nodes are represented by conditional probability tables or simple functions whose parameters are learned from the data. There are three main problems in fitting graphical models to data: inference, parameter learning and structure learning. The inference problem is how to infer the probable values of unobserved variables when the values of a subset of the variables have been observed, and is a problem that perceptual systems need to solve if they are to infer the hidden causes of their sensory input. The parameter-learning problem is how to adjust the parameters governing the way in which one variable influences another, so that the graphical model is a better fit to some observed data. In the brain, this is presumably done by changing synapse strengths. The structure-learning problem is how to decide which unobserved variables are needed and how they must be connected to model the correlations between observed variables. In the brain, evolution and early pruning of connections presumably have a large role to play in determining the structure. Could you provide a brief description of the methods of machine learning? Machine learning can be divided into three parts: 1) in supervised learning, the aim is to predict a class label or a real value from an input (classifying objects in images or predicting the future value of a stock are examples of this type of learning); 2) in unsupervised learning, the aim is to discover good features for representing the input data; and 3) in reinforcement learning, the aim is to discover what action should be performed next in order to maximize the eventual payoff.

Digital data transmission between neuronlike elements

The problem of digital data transmission between two neuronlike systems has been investigated. The former of a sequence of rectangular pulses is used as an information signal source. The behavior of neuronlike elements is described with the help of the Hodgkin-Huxley mathematical model. Such a system is shown to form the communications channel of fairly high quality in terms of the classical theory of information transmission.

Dynamics of two neuronlike elements with inhibitory feedback

An analog electronic model is proposed to qualitatively reproduce the main dynamic regimes of inferior-olive (IO) neurons. An analog simulation is used to study the dynamics of a single IO neuron and a pair of neurons involved in inhibitory feedback. The experimental results that demonstrate that such feedback leads to a partial spike synchronization of the interacting neurons are presented. It is shown that the electronic model can be used for the simulation of the burst activity of oscillatory neurons.

Sparse Coding via Thresholding and Local Competition in Neural Circuits

While evidence indicates that neural systems may be employing sparse approximations to represent sensed stimuli, the mechanisms underlying this ability are not understood. We describe a locally competitive algorithm (LCA) that solves a collection of sparse coding principles minimizing a weighted combination of mean-squared error and a coefficient cost function. LCAs are designed to be implemented in a dynamical system composed of many neuron-like elements operating in parallel. These algorithms use thresholding functions to induce local (usually one-way) inhibitory competitions between nodes to produce sparse representations. LCAs produce coefficients with sparsity levels comparable to the most popular centralized sparse coding algorithms while being readily suited for neural implementation. Additionally, LCA coefficients for video sequences demonstrate inertial properties that are both qualitatively and quantitatively more regular (i.e., smoother and more predictable) than the coefficients produced by greedy algorithms.

A neuronal model of the language cortex

We modelled language-learning processes in a brain-inspired model of the language cortex. The network consisted of neuron-like elements (graded-response units) and mimicked the neuroanatomical areas in the perisylvian language cortex and the intrinsic and mutual connections within and between them. Speaking words creates correlated activity in motor and auditory cortical systems. This correlated activity might play an important role in setting up word representations at the neuronal level [D.B. Fry, The development of the phonological system in the normal and deaf child, in: F. Smith, F.A. Miller, (Eds.), The genesis of language MIT Press, Cambridge, MA, 1966, pp. 187–206; F. Pulvermüller, Words in the Brain's Language, Behav. Brain Sci. 22 (1999) 253–279]. We simulated this language-learning process and used the network to simulate neurophysiological brain responses to words and meaningless “pseudowords” as they have been documented using EEG and MEG.

Are our senses critical?

The sensitivity and dynamic range of a network made of neuron-like elements is now shown to be maximized at the critical point of a phase transition. This raises the question of whether critical senses might improve survival in a critical world.

Control of synchronous dynamics of chaotic bursts in ensembles of neuron-like elements

The mutual phase locking of chaotic oscillations in ensembles of globally coupled neuron-like elements is demonstrated. The possibility to control such synchronous dynamics by a periodic signal applied to an arbitrarily taken element is shown. An interpretation of the observed effects is proposed and the possibility of their practical use in neuron-like ensembles is discussed.

A stochastic population approach to the problem of stable recruitment hierarchies in spiking neural networks

Synchrony-driven recruitment learning addresses the question of how arbitrary concepts, represented by synchronously active ensembles, may be acquired within a randomly connected static graph of neuron-like elements. Recruitment learning in hierarchies is an inherently unstable process. This paper presents conditions on parameters for a feedforward network to ensure stable recruitment hierarchies. The parameter analysis is conducted by using a stochastic population approach to model a spiking neural network. The resulting network converges to activate a desired number of units at each stage of the hierarchy. The original recruitment method is modified first by increasing feedforward connection density for ensuring sufficient activation, then by incorporating temporally distributed feedforward delays for separating inputs temporally, and finally by limiting excess activation via lateral inhibition. The task of activating a desired number of units from a population is performed similarly to a temporal k-winners-take-all network.