Design and Comparison Base Analysis of Adaptive Estimator for Completely Unknown Linear Systems in the Presence of OE Noise and Constant Input Time Delay

Abstract

AbstractIn this paper, an adaptive estimator (AE) is introduced to learn the approximate Q‐function and control policy by only using the noisy states and control signals of the unknown linear, discrete time systems having constant input time delay. The system measurements are uncertain owing to output error (OE)‐type noise acting randomly on the system measurements. Therefore, this research differs from the designed AE in the literature since previous research ignores the role of the external random disturbances on AE‐based learning. In order to compare the AE‐based learning results with traditional system identification (SI) approaches, a modified version of the OE model structure for unstable systems is reviewed and parameters of a second‐order unstable system with constant input time delay are identified. The simulation results demonstrate that the designed AE efficiently minimizes the stochastic cost function and the temporal difference error by learning the approximate solution for the Hamilton‐Jacobi‐Bellman (HJB) equation. It is noted that the error in the Q‐function obtained with the AE is slightly larger than the Q‐function attained with the identified OE parameters. However, as the noise standard deviation increases, the error in the AE‐based learning results reduces whereas the error in the OE‐based learning increases. This indicates that even though the added random noise deteriorates the performance of the OE predictor, it improves the learning efficiency of the AE since it acts like exploration noise.