Abstract
In this paper, the Q-learning method for quadratic optimal control problem of discrete-time linear systems is reconsidered. The theoretical results prove that the quadratic optimal controller cannot be solved directly due to the linear correlation of the data sets. The following corollaries have been made: (1) The correlation of data is the key factor in the success for the calculation of quadratic optimal control laws by Q-learning method; (2) The control laws for linear systems cannot be derived directly by the existing Q-learning method; (3) For nonlinear systems, there are some doubts about the data independence of current method. Therefore, it is necessary to discuss the probability of the controllers established by the existing Q-learning method. To solve this problem, based on the ridge regression, an improved model-free Q-learning quadratic optimal control method for discrete-time linear systems is proposed in this paper. Therefore, the computation process can be implemented correctly, and the effective controller can be solved. The simulation results show that the proposed method can not only overcome the problem caused by the data correlation, but also derive proper control laws for discrete-time linear systems.
Highlights
This article reconsiders the Q-learning method for quadratic optimal control problem of discrete-time linear systems
To solve the multi-collinearity problem caused by matrix Φ T Φ, an improved Q-learning method adopting ridge regression has been proposed in this paper
The design of optimal quadratic controllers for linear discrete-time systems based on Q-learning method is claimed as a kind of model-free method [37,38]
Summary
This article reconsiders the Q-learning method for quadratic optimal control problem of discrete-time linear systems. It is necessary to explain the quadratic optimal control problem of discrete-time linear systems. The optimal feedback matrix K can be solved It can be seen from the above process that the precondition of the calculation process is the precise acquaintance of controlled systems’. Q-learning method for linear discrete-time systems, while the independence of data during the computation process is not get enough attention. Processes 2020, 8, 368 is discussed, aiming to the computation processes of optimal controllers for linear discrete-time systems adopting Q-learning method. The data sets sampled from the linear discrete-time systems is destined relevant, which means that the controllers cannot be solved by the existing Q-learning method. To design proper controllers for linear discrete-time systems, an improved Q-learning method is proposed in this paper. Results of simulations show the effectiveness of the proposed method
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
