Analog Neuron Model Research Articles

Fast backpropagation learning using optimization of learning rate for pulsed neural networks

AbstractNeural networks are widely applied to information processing because of their nonlinear processing capability. Digital hardware implementation of neural networks seems to be effective in the construction of neural network systems in which real‐time operation and much broader applications are possible. However, the digital hardware implementation of analog neural networks is very difficult because of the need to satisfy restrictions concerning circuit resources such as circuit scale, arrangement, and wiring. A technique that uses a pulsed neuron model instead of an analog neuron model as a method for solving this problem has been proposed, and its effectiveness has been confirmed. To construct pulsed neural networks (PNN), backpropagation (BP) learning has been proposed. However, BP learning takes considerable time to construct a PNN compared with the learning of an analog neural network. Therefore, some method of speeding up BP learning in PNN is necessary. In this paper, we propose a fast BP learning method using optimization of the learning rate for a PNN. In the proposed method, the learning rate is optimized so as to speed up learning in every learning cycle. To evaluate the proposed method, we apply it to pattern recognition problems, such as XOR, 3‐bit parity, and digit recognition. The results of the computer‐based experiments demonstrate the validity of the proposed method. © 2011 Wiley Periodicals, Inc. Electron Comm Jpn, 94(7): 27–34, 2011; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ecj.10249

Fast Back Propagation Learning Using Optimization of Learning Rate for Pulsed Neural Networks

Neural Networks (NN) are widely applied to information processing because of its nonlinear processing capability. Digital hardware implementation of NN seems to be effective in construction of NN systems in which real-time operation and much further wide applications are possible. However, the digital hardware implementation of analogue NN is very difficult because we have to fulfill the restrictions about circuit resource, such as circuit scale, arrangement, and wiring. A technique that uses pulsed neuron model instead of analogue neuron model as a method of solving this problem has been proposed, and its effectiveness has been confirmed. To construct Pulsed Neural Networks (PNN), Back Propagation (BP) learning has been proposed. However, BP learning takes much time to construct PNN compared with the learning of analogue NN. Therefore some method to speed up BP learning of PNN is necessary. In this paper, we propose a fast BP learning using optimization of learning rate for PNN. In the proposed method, the learning rate is optimized so as to speed up the learning at every learning epoch. To evaluate the proposed method, we apply it to some pattern recognition problems, such as XOR, 3-bits parity, and digit recognition. Results of computational experiments indicate the validity of the proposed method.

A Chaos Associative Model with a Sinusoidal Activation Function

In this paper, we shall propose a novel chaos neural network model applied to the chaotic autoassociation memory. The present artificial neuron model is properly characterized in terms of a time-dependent periodic activation function to involve chaotic dynamics as well as the energy steepest descent strategy. It is elucidated that the present neural network has a remarkable ability of dynamic memory retrieval beyond the conventional models with the nonmonotonous activation function as well as a monotonous activation function as the sigmoidal one. This advantage is found to result from the property of the analogue periodic mapping accompanied with a chaotic behaviour of the neurons. It is also concluded that the present analogue neuron model, with the periodicity control, has an apparently large memory capacity in comparison with the previously proposed association models.

Chaos Associative Memory with a Periodic Activation Function

In this paper we shall propose a novel chaos neural network model applied to the chaotic autoassociation memory. The present artificial neuron model is properly characterized in terms of a time-dependent periodic activation function to involve a chaotic dynamics as well as the energy steepest descent strategy. It is elucidated that the present neural network has a remarkable ability of the dynamic memory retrievals beyond the conventional models with the nonmonotonous activation function as well as such a monotonous activation function as sigmoidal one. This advantage is found to result from the property of the analogue periodic mapping accompanied with a chaotic behaviour of the neurons. It is also concluded that the present analogue neuron model with the periodicity control has an apparently large memory capacity in comparison with the previously proposed association models.

A Parameter Controlled Chaos Neural Network

In this paper we shall propose a simple chaos neural network model applied to autonomous chaotic memory searching and the autoassociation. The present artificial neuron model is properly characterized in terms of a sinusoidal activation function to involve a chaotic dynamics as well as an autonomous control of the periodicity. It is elucidated that the present neural network has an ability of the dynamic memory retrievals beyond the conventional chaotic model with such a monotonous mapping as a sigmoid function. This advantage is found to result from the nonmonotonous property of the analogue periodic mapping accompanied with a chaotic behaviour of the neurons. It is also found that the present analogue neuron model with the autonomous periodicity control has a remarkably large memory capacity in comparison with the conventional monotonous dynamics.

An Artificial Neuron Model with a Periodic Activation Function

In this paper we shall propose a novel chaos neuron model with a periodic activation function and investigate the dynamic properties in a chaotic memory retrieval mode. The present artificial neuron model is characterized by a sinusoidal activation function. It is elucidated that the present neural network has an ability of the dynamic memory retrievals beyond the conventional chaotic model with such a monotonous mapping as a sigmoid function. This advantage is considered to result from the nonmonotonous property of the analogue periodic mapping which may be accompanied with a chaotic behaviour of the neurons. It is also found that the present analogue neuron model has a relatively larger memory capacity than the conventional association model even for the case of the monotonous mapping as a special case.

A modular analog neuron-model for research and teaching.

An analog electronic neuron for modeling neuronal networks and for teaching purposes is presented. The circuitry represents a new approach, since ionic channels are all modeled by fast relay switches in combination with resistors. Thus, the synapse design includes nonlinear PSP superposition and reversal potentials. The spike is modeled with Na+ and K+ currents. Detailed circuit diagrams and examples of performance are given. Besides its use in network research, the model has been successfully used over 3 years of teaching practice.

On periodic responses generated by a degenerate analog neuron model with periodic inputs.

Periodic responses to periodically varying stimulating pulse sequences are mathematically described for a degenerate analog neuron model. The model used was derived by Yoshizawa et al. (1982) in their investigation of the state transition of an electronic model using a tunnel diode in a degenerate case. Periodic responses to constant pulse sequences for the model were described by them. In this paper, it is shown that periodic responses to periodically varying pulse sequences for the model are identical with those which the author previously described for a discrete neuron model.

Difference equation model of ventricular parasystole as an interaction between cardiac pacemakers based on the phase response curve

A model of extended ventricular parasystole proposed by Moe et al. (1977) was formulated as a system of nonlinear difference equations by using the phase response curve of myocardial pacemakers. A number of ECG patterns of ventricular arrhythmia such as bigeminy, trigeminy etc. were explained from the property of periodic solutions of the equation. Characteristic properties of special kinds of arrhythmia called “concealed bigeminy” and “concealed trigeminy” were derived mathematically by assuming the model, in relation to the equation of the analog neuron model. The present study was considered to be of clinical significance as a theoretical foundation for the study of genesis of cardiac arrhythmias.

Pulse sequences generated by a degenerate analog neuron model.

The response characteristics of an electronic neuron model proposed by the authors are investigated. Periodic stimulating pulse sequences with a fixed frequency are applied to the analog neuron model and the response pulse sequences are studied. In the degenerate case, the state transition of the neuron model during one period of the stimulating pulse sequense is described by a first order piecewise linear difference equation with a jump. It is shown that the periodic response pulse sequences of the neuron model belong to a special class of pulse sequences generated by a simple algorithm, and that the relation between the pulse width (or amplitude) of the stimulating pulse and the firing rate of the neuron model takes the form of an extended Cantor function.

Cyclic sequences generated by neuron model

AbstractA neuron can be considered as a threshold element with refractory period. In order to arrive at a generalized treatment of several neuron models proposed in the past, this paper considers a certain type of threshold element network model which has, in addition to exponential decay of refractoriness with time, the effect of past excitations included in the exponent. The periodic behavior of such a network is analyzed and the existence condition for arbitrarily assumed periodic sequence and its generation rule are derived. The procedure is closely related to the continued fraction expansion of a rational number and the sequence is uniquely determined from the network dynamics. The entire set of periodic sequences produced by a network can be derived by the proposed procedure. Lastly, the relation between the proposed network model and the analog neuron model is discussed.