Adaptive Double Fuzzy Systems Based Q-Learning for Pursuit-Evasion Game

Abstract

The paper addresses the problem of obtaining the optimal solutions of pursuit and evasion game in continuous time system. A novel technique that combines double fuzzy systems with Q-learning is proposed. The system consists of two segments: fuzzy Q-learning (FQL) module and Q-value table fuzzy inference system (QTFIS). The linguistic rules are updated adaptively by Q-learning in FQL module. Local Q-value table which takes advantages of the sparsity of Q-learning is the kernel data structure in FQL. The Controllers of pursuer and evader are provided by LQT while the parameters of fuzzy inference system are updated adaptively. Q-value function is approximated by the fuzzy inference system according to QTFIS. The innovative method has an advantage of not relying on accurate controlled models. The difficult drawback of continuous state space and continuous actions for Q-learning is overcome by the novel technique. The limitation of low dimensional space for Q-learning is broken by the system. The proposed technique is applied to an imperfect information pursuit-evasion game between UAVs. Finally, computer simulations show the usefulness of the creative approach.