Backpropagation Through Time Algorithm Research Articles

Efficient training of spiking neural networks with temporally-truncated local backpropagation through time.

Directly training spiking neural networks (SNNs) has remained challenging due to complex neural dynamics and intrinsic non-differentiability in firing functions. The well-known backpropagation through time (BPTT) algorithm proposed to train SNNs suffers from large memory footprint and prohibits backward and update unlocking, making it impossible to exploit the potential of locally-supervised training methods. This work proposes an efficient and direct training algorithm for SNNs that integrates a locally-supervised training method with a temporally-truncated BPTT algorithm. The proposed algorithm explores both temporal and spatial locality in BPTT and contributes to significant reduction in computational cost including GPU memory utilization, main memory access and arithmetic operations. We thoroughly explore the design space concerning temporal truncation length and local training block size and benchmark their impact on classification accuracy of different networks running different types of tasks. The results reveal that temporal truncation has a negative effect on the accuracy of classifying frame-based datasets, but leads to improvement in accuracy on event-based datasets. In spite of resulting information loss, local training is capable of alleviating overfitting. The combined effect of temporal truncation and local training can lead to the slowdown of accuracy drop and even improvement in accuracy. In addition, training deep SNNs' models such as AlexNet classifying CIFAR10-DVS dataset leads to 7.26% increase in accuracy, 89.94% reduction in GPU memory, 10.79% reduction in memory access, and 99.64% reduction in MAC operations compared to the standard end-to-end BPTT. Thus, the proposed method has shown high potential to enable fast and energy-efficient on-chip training for real-time learning at the edge.

Preferential training of neurodynamical model based on predictability of target dynamics

Intrinsic motivation is one of the keys in implementing the mechanism of interest to robots. In this paper, we present a method to apply intrinsic motivation in dynamics learning with predictable and unpredictable targets in view. The robot’s arm is used for the predictable target and the human’s arm is used for the unpredictable target in the experiment. The learning algorithm based on intrinsic motivation will automatically set a larger weight to targets that would contribute to decreasing the training error, while setting a smaller weight to others. A neurodynamical model, namely multiple timescales recurrent neural network (MTRNN), is utilized for studying the robot’s arm/external object dynamics. Training of MTRNN is done using the back propagation through time (BPTT) algorithm. We modify the BPTT algorithm by the following two steps. (1) Evaluate predictability of robot’s arm/objects using training error of MTRNN. (2) Assign a preference ratio, which represents the weight of the training, to each object based on predictability. The proposed training method would focus more on reducing training error of predictable objects compared to normal BPTT, where training error is equally treated for every object. Experiments were conducted using an actual robot platform, moving the robot’s arm while a human moves his arm in the robot’s camera view. The results of the experiment showed that the proposed training method could achieve smaller training error of the robot’s arm visuomotor dynamics, which is predictable from the robot’s motor command, compared to general training with BPTT. Evaluation of MTRNN as a forward model to predict untrained data showed that the proposed model is capable of predicting the robot’s hand motion, specifically with larger number of nodes.

Multiple Time Scales Recurrent Neural Network for Complex Action Acquisition

This paper presents novel results of complex action learning experiments based on the use of extended multiple timescales recurrent neural networks (MTRNN). The experiments were carried out with the iCub humanoid robot, as a model of the developmental learning of motor primitives as the basis of sensorimotor and linguistic compositionality. The model was implemented through the Aquila cognitive robotics toolkit, which supports the CUDA architecture and makes use of massively parallel GPUs (graphics processing units). The results presented herein show that the model was able to learn and successfully reproduce multiple behavioural sequences of actions in an object manipulation task scenario using large-scale MTRNNs. This forms the basis on ongoing experiments on action and language compositionality

Real time implementation of CTRNN and BPTT algorithm to learn on-line biped robot balance: Experiments on the standing posture

This paper describes experimental results regarding the real time implementation of continuous time recurrent neural networks (CTRNN) and the dynamic back-propagation through time (BPTT) algorithm for the on-line learning control laws. Experiments are carried out to control the balance of a biped robot prototype in its standing posture. The neural controller is trained to compensate for external perturbations by controlling the torso’s joint motions. Algorithms are embedded in the real time electronic unit of the robot. On-line learning implementations are presented in detail. The results on learning behavior and control performance demonstrate the strength and the efficiency of the proposed approach.

Multicylinder engine pressure reconstruction using NARX neural networks and crank kinematics

A NARX neural network is adapted for cylinder pressure trace reconstruction on a multicylinder engine. Following a systematic study to establish the required NARX input information (using measured pressure traces and simulated crank kinematics), two fully recurrent training algorithms are developed and applied to real engine data. These include a back-propagation-through-time algorithm (BPTT) and an extended Kalman filter (EKF). For multi-cylinder engines, two cases are examined, both assuming crank kinematics is obtained from a single shaft-encoder fitted at the forward end of the crankshaft. In one case, a NARX model is constructed to provide an inverse relationship between the kinematics at the encoder location and the pressure trace in an arbitrary cylinder. In the second case, by transforming the kinematics (to emulate a local encoder), a different NARX model is constructed to relate the kinematics at the crank location of a particular cylinder to the corresponding pressure trace. The accuracy and efficiency of both NARX models is examined for application to a three-cylinder in-line DISI engine (in which pressure traces are measured on all cylinders). The paper shows that the computational requirements of training are substantial and, although the efficiency of the EKF algorithm is better than the BPTT, the fitting accuracies are similarly good. For generalization, however (to unseen data), neither method is yet sufficiently accurate (even for steady state engine operation) unless substantially more training data are used to achieve the target accuracy of ± 4 per cent. The overall conclusion of the paper is that the NARX model has the correct architecture for multicylinder pressure reconstruction.

Backpropagation Algorithms for a Broad Class of Dynamic Networks

This paper introduces a general framework for describing dynamic neural networks--the layered digital dynamic network (LDDN). This framework allows the development of two general algorithms for computing the gradients and Jacobians for these dynamic networks: backpropagation-through-time (BPTT) and real-time recurrent learning (RTRL). The structure of the LDDN framework enables an efficient implementation of both algorithms for arbitrary dynamic networks. This paper demonstrates that the BPTT algorithm is more efficient for gradient calculations, but the RTRL algorithm is more efficient for Jacobian calculations.

Neural network approach for identification of Hammerstein systems

In this paper, four different on-line gradient-based learning algorithms for training neural network Hammerstein models are presented in a unified framework. These algorithms, namely the backpropagation for series–parallel models, the backpropagation, sensitivity method, and truncated backpropagation through time algorithm (BPTT) for parallel models are derived, analysed, and compared. For the truncated BPTT, it is shown that determination of the number of unfolding time steps, necessary to calculate the gradient with an assumed degree of accuracy, can be made on the basis of impulse response functions of sensitivity models. The algorithms are shown to differ in their computational complexity, gradient approximation accuracy, and convergence rates. Numerical examples are also included to compare the performance of the algorithms.

Recurrent Neural Networks for Predicting Outcomes after Liver Transplantation: Representing Temporal Sequence of Clinical Observations

This paper investigates a version of recurrent neural network with the backpropagation through time (BPTT) algorithm for predicting liver transplant graft failure based on a time series sequence of clinical observations. The objective is to improve upon the current approaches to liver transplant outcome prediction by developing a more complete model that takes into account not only the preoperative risk assessment, but also the early postoperative history. A 6-fold cross-validation procedure was used to measure the performance of the networks. The data set was divided into a learning set and a test set by maintaining the same proportion of positive and negative cases in the original set. The effects of network complexity on overfitting were investigated by constructing two types of networks with different numbers of hidden units. For each type of network, 10 individual networks were trained on the learning set and used to form a committee. The performance of the networks was measured exhaustively with respect to both the entire training and test sets. The networks were capable of learning the time series problem and achieved good performances of 90% correct classification on the learning set and 78% on the test set. The prediction accuracy increases as more information becomes progressively available after the operation with the daily improvement of 10% on the learning set and 5% on the test set. Recurrent neural networks trained with BPTT algorithm are capable of learning to represent temporal behavior of the time series prediction task. This model is an improvement upon the current model that does not take into account postoperative temporal information.

A simplification of the backpropagation-through-time algorithm for optimal neurocontrol

Backpropagation-through-time (BPTT) is the temporal extension of backpropagation which allows a multilayer neural network to approximate an optimal state-feedback control law provided some prior knowledge (Jacobian matrices) of the process is available. In this paper, a simplified version of the BPTT algorithm is proposed which more closely respects the principle of optimality of dynamic programming. Besides being simpler, the new algorithm is less time-consuming and allows in some cases the discovery of better control laws. A formal justification of this simplification is attempted by mixing the Lagrangian calculus underlying BPTT with Bellman-Hamilton-Jacobi equations. The improvements due to this simplification are illustrated by two optimal control problems: the rendezvous and the bioreactor.