Weight Convergence Analysis Research Articles

Weight convergence analysis of DV-hop localization algorithm with GA.

The distance vector-hop (DV-hop) is a typical localization algorithm. It estimates sensor nodes location through detecting the hop count between nodes. To enhance the positional precision, the weight is used to estimate position, and the conventional wisdom is that the more hop counts are, the smaller value of weight will be. However, there has been no clear mathematical model among positioning error, hop count, and weight. This paper constructs a mathematical model between the weights and hops and analyzes the convergence of this model. Finally, the genetic algorithm is used to solve this mathematical weighted DV-hop (MW-GADV-hop) positioning model, the simulation results illustrate that the model construction is logical, and the positioning error of the model converges to 1/4R.

Robustness to Training Disturbances in SpikeProp Learning.

Stability is a key issue during spiking neural network training using SpikeProp. The inherent nonlinearity of Spiking Neuron means that the learning manifold changes abruptly; therefore, we need to carefully choose the learning steps at every instance. Other sources of instability are the external disturbances that come along with training sample as well as the internal disturbances that arise due to modeling imperfection. The unstable learning scenario can be indirectly observed in the form of surges, which are sudden increases in the learning cost and are a common occurrence during SpikeProp training. Research in the past has shown that proper learning step size is crucial to minimize surges during training process. To determine proper learning step in order to avoid steep learning manifolds, we perform weight convergence analysis of SpikeProp learning in the presence of disturbance signals. The weight convergence analysis is further extended to robust stability analysis linked with overall system error. This ensures boundedness of the total learning error with minimal assumption of bounded disturbance signals. These analyses result in the learning rate normalization scheme, which are the key results of this paper. The performance of learning using this scheme has been compared with the prevailing methods for different benchmark data sets and the results show that this method has stable learning reflected by minimal surges during learning, higher success in training instances, and faster learning as well.

Robust learning in SpikeProp

Training a Spiking Neural Network using SpikeProp and its derivatives faces stability issues. Surges, marked by a sudden rise in learning cost, are a common occurrence during the learning process. They disrupt the learning process and often destabilize the process resulting in failure. A proper learning rate, which is neither too small nor too big, is important to minimize surges. Furthermore, external disturbances due to imperfection in sample data as well as internal disturbances are additional destabilizing source during the learning process. In this paper, we perform error system analysis incorporating external disturbance, followed by weight convergence analysis along with detailed robust stability analysis of SpikeProp learning process to ensure error bound of the learning process. Based on these results, we propose a robust adaptive learning rate scheme that aligns with the results of theoretical analysis. The performance of the proposed method has been compared with other prevalent methods based on different benchmark datasets and the results demonstrate that our method indeed has better performance in terms of convergence and learning speed as well.

Adaptive learning rate of SpikeProp based on weight convergence analysis

A Spiking Neural Network (SNN) training using SpikeProp and its variants is usually affected by sudden rise in learning cost called surges. These surges cause diversion in the learning process and often cause it to fail as well. Researches have shown that proper learning rate is crucial to avoid these surges. In this paper, we perform weight convergence analysis to determine the proper step size in each iteration of weight update and derive an adaptive learning rate extension to SpikeProp that assures convergence of the learning process. We have analyzed the performance of this learning rate adaptation with existing methods via simulations on different benchmarks. The results show that using adaptive learning rate significantly improves the weight convergence and speeds up learning as well.

Robust Initialization of a Jordan Network With Recurrent Constrained Learning

In this paper, we propose a robust initialization of a Jordan network with a recurrent constrained learning (RIJNRCL) algorithm for multilayered recurrent neural networks (RNNs). This novel algorithm is based on the constrained learning concept of the Jordan network with a recurrent sensitivity and weight convergence analysis, which is used to obtain a tradeoff between the training and testing errors. In addition to using classical techniques for the adaptive learning rate and the adaptive dead zone, RIJNRCL employs a recurrent constrained parameter matrix to switch off excessive contributions from the hidden layer neurons based on weight convergence and stability conditions of the multilayered RNNs. It is well known that a good response from the hidden layer neurons and proper initialization play a dominant role in avoiding local minima in multilayered RNNs. The new RIJNRCL algorithm solves the twin problems of weight initialization and selection of the hidden layer neurons via a novel recurrent sensitivity ratio analysis. We provide the detailed steps for using RIJNRCL in a few benchmark time-series prediction problems and show that the proposed algorithm achieves superior generalization performance.

Weight-convergence analysis of adaptive antenna arrays based on SMI algorithm

Weight-convergence analysis of adaptive antenna arrays based on the sample matrix inversion (SMI) algorithm is presented. First, we derive a required number of samples for weight to achieve an output signal-to-interference-plus-noise ratio -10 log/sub 10/ r dB (r /spl les/ 1) below an optimum value. Next, we evaluate weight convergence of code-division multiple-access adaptive antenna arrays based on the SMI algorithm, where two types of weight computational methods are compared between chip-level computation and symbol-level computation. Analytical and simulation results reveal that chip-level computation, before despreading process, enables faster convergence than symbol-level computation, after the despreading process.