Fuzzy Actor-critic Learning Research Articles

Accelerating Fuzzy Actor–Critic Learning via Suboptimal Knowledge for a Multi-Agent Tracking Problem

Multi-agent differential games usually include tracking policies and escaping policies. To obtain the proper policies in unknown environments, agents can learn through reinforcement learning. This typically requires a large amount of interaction with the environment, which is time-consuming and inefficient. However, if one can obtain an estimated model based on some prior knowledge, the control policy can be obtained based on suboptimal knowledge. Although there exists an error between the estimated model and the environment, the suboptimal guided policy will avoid unnecessary exploration; thus, the learning process can be significantly accelerated. Facing the problem of tracking policy optimization for multiple pursuers, this study proposed a new form of fuzzy actor–critic learning algorithm based on suboptimal knowledge (SK-FACL). In the SK-FACL, the information about the environment that can be obtained is abstracted as an estimated model, and the suboptimal guided policy is calculated based on the Apollonius circle. The guided policy is combined with the fuzzy actor–critic learning algorithm, improving the learning efficiency. Considering the ground game of two pursuers and one evader, the experimental results verified the advantages of the SK-FACL in reducing tracking error, adapting model error and adapting to sudden changes made by the evader compared with pure knowledge control and the pure fuzzy actor–critic learning algorithm.

An algorithm of pretrained fuzzy actor–critic learning applying in fixed-time space differential game

Solving space differential game in an unknown environment remains a challenging problem. This article proposes a pretrained fuzzy actor–critic learning algorithm for dealing with the space pursuit-evasion game in fixed time. It is supposed that the research objects are two agents including one pursuer and one evader in space. A virtual environment, which is defined as the known part of the real environment, is utilized for deriving optimal strategies of the pursuer and the evader, respectively. Through employing the fuzzy inference system, a pretrained process, which is based on the genetic algorithm, is designed to obtain the initial consequent set of the pursuer and the evader. Besides, an actor–critic framework is applied to finely learn the suitable consequent set of the pursuer and evader in the real environment. Numerical experimental results validate the effectiveness of the proposed algorithms on improving the ability of the agents to adapt to the real environment.

A Residual Gradient Fuzzy Reinforcement Learning Algorithm for Differential Games

In this work, we propose a new fuzzy reinforcement learning algorithm for differential games that have continuous state and action spaces. The proposed algorithm uses function approximation systems whose parameters are updated differently from the updating mechanisms used in the algorithms proposed in the literature. Unlike the algorithms presented in the literature which use the direct algorithms to update the parameters of their function approximation systems, the proposed algorithm uses the residual gradient value iteration algorithm to tune the input and output parameters of its function approximation systems. It has been shown in the literature that the direct algorithms may not converge to an answer in some cases, while the residual gradient algorithms are always guaranteed to converge to a local minimum. The proposed algorithm is called the residual gradient fuzzy actor–critic learning (RGFACL) algorithm. The proposed algorithm is used to learn three different pursuit–evasion differential games. Simulation results show that the performance of the proposed RGFACL algorithm outperforms the performance of the fuzzy actor–critic learning and the Q-learning fuzzy inference system algorithms in terms of convergence and speed of learning.

Obstacle Avoidance of a Mobile Robot Using Hybrid Learning Approach

in this paper, a hybrid learning approach for obstacle avoidance of a mobile robot is presented. the key features of the approach are, firstly, innate hardwired behaviors which are used to bootstrap learning in the mobile robot system. a neuro-fuzzy controller is developed from a pre-wired or innate controller based on supervised learning in a simulation environment. the fuzzy inference system has been constructed based on the generalized dynamic fuzzy neural networks learning algorithm of Wu and Er, whereby structure and parameters identification are carried out automatically and simultaneously. Secondly, the neuro-fuzzy controller is capable of re-adapting in a new environment. After carrying out the learning phase on a simulated robot, the controller is implemented on a real robot. A reinforcement learning method based on the fuzzy actor-critic learning algorithm is employed so that the system can re-adapt to a new environment without human intervention. In this phase, the structure of the fuzzy inference system and the parameters of the antecedent parts of fuzzy rules are frozen, and reinforcement learning is applied to further tune the parameters in the consequent parts of the fuzzy rules. Through the hybrid learning approach, an efficient and compact neuro-fuzzy system is generated for obstacle avoidance of a mobile robot in the real world.

Fuzzy inference system learning by reinforcement methods

Fuzzy Actor-Critic Learning (FACL) and Fuzzy Q-Learning (FQL) are reinforcement learning methods based on dynamic programming (DP) principles. In the paper, they are used to tune online the conclusion part of fuzzy inference systems (FIS). The only information available for learning is the system feedback, which describes in terms of reward and punishment the task the fuzzy agent has to realize. At each time step, the agent receives a reinforcement signal according to the last action it has performed in the previous state. The problem involves optimizing not only the direct reinforcement, but also the total amount of reinforcements the agent can receive in the future. To illustrate the use of these two learning methods, we first applied them to a problem that involves finding a fuzzy controller to drive a boat from one bank to another, across a river with a strong nonlinear current. Then, we used the well known Cart-Pole Balancing and Mountain-Car problems to be able to compare our methods to other reinforcement learning methods and focus on important characteristic aspects of FACL and FQL. We found that the genericity of our methods allows us to learn every kind of reinforcement learning problem (continuous states, discrete/continuous actions, various type of reinforcement functions). The experimental studies also show the superiority of these methods with respect to the other related methods we can find in the literature.