On Momentum and Learning Rate of the Generalized ADLINE Neural Network for Time Varying System Identification

Abstract

On-line System identification of linear time-varying (LTV) systems whose system parameters change in time has been studied lately. One neural network based such on-line identification method was studied by the author with a generalized ADAptive LINear Element (ADALINE). Since the ADALINE is slow in convergence, which is not suitable for identification of LTV system, two techniques were proposed to speed up convergence of learning. One idea was to introduce a momentum term to the weight adjustment during convergence period. The other technique was to train the generalized ADALINE network with data from a sliding window of the system’s input output data. The second technique took multiple epochs to train which was considered as a shortcoming. In this paper, simulation study towards optimizing the momentum term and learning rate parameter will be presented. Simulation results show that once the momentum factor and learning rate are tuned properly, time varying parameters of LTV systems can be identified quite effectively; which, in turn, sows that the fined tuned GADLINE is quite suitable for online system identification and real time adaptive control applications due to its low computational demand.KeywordsSystem identificationNeural networkADALINETapped delay line feedback