Fuzzy Reinforcement Learning Algorithm Research Articles

Fuzzy Reinforcement Learning Algorithm for Efficient Task Scheduling in Fog-Cloud IoT-Based Systems

Fuzzy Reinforcement Learning Algorithm for Efficient Task Scheduling in Fog-Cloud IoT-Based Systems

JAYA-Optimized Fuzzy Reinforcement Learning Classifier for COVID-19

In the proposed work, Reinforcement Learning has been used to identify Coronavirus disease (COVID-19) amongst viral pneumonia and normal conditions using chest X-ray images. Here, the JAYA-Optimized Fuzzy Reinforcement Learning algorithm, in conjunction with Wavelet Transform-based feature extraction and Principal Component Analysis-based feature reduction technique, has been applied to construct the proposed classifier. Predication accuracy achieved with the proposed approach for COVID-19, viral pneumonia, and normal cases is 87.75%, 95.63%, and 96.89%, respectively. The average classification accuracy is 93.43%. The results show that the JAYA-Optimized Fuzzy Reinforcement Learning approach attains high accuracy and is superior to other contemporary classifiers, i.e. Support Vector Machine, Artificial Neural Network, Tree Algorithm, and K-Nearest Neighbor. The work aims to develop a classifier that can segregate COVID-19 cases from normal and viral pneumonia cases based on the chest X-ray images and is online, adaptive, accurate, and fast. The classifier could serve as a diagnostic tool for healthcare professionals in the current pandemic.

A Pre-Trained Fuzzy Reinforcement Learning Method for the Pursuing Satellite in a One-to-One Game in Space.

In order to help the pursuer find its advantaged control policy in a one-to-one game in space, this paper proposes an innovative pre-trained fuzzy reinforcement learning algorithm, which is conducted in the x, y, and z channels separately. Compared with the previous algorithms applied in ground games, this is the first time reinforcement learning has been introduced to help the pursuer in space optimize its control policy. The known part of the environment is utilized to help the pursuer pre-train its consequent set before learning. An actor-critic framework is built in each moving channel of the pursuer. The consequent set of the pursuer is updated through the gradient descent method in fuzzy inference systems. The numerical experimental results validate the effectiveness of the proposed algorithm in improving the game ability of the pursuer.

Decentralized Incremental Fuzzy Reinforcement Learning for Multi-Agent Systems

We present a new incremental fuzzy reinforcement learning algorithm to find a sub-optimal policy for infinite-horizon Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs). The algorithm addresses the high computational complexity of solving large Dec-POMDPs by generating a compact fuzzy rule-base for each agent. In our method, each agent uses its own fuzzy rule-base to make the decisions. The fuzzy rules in these rule-bases are incrementally created and tuned according to experiences of the agents. Reinforcement learning is used to tune the behavior of each agent in such a way that maximum global reward is achieved. In addition, we propose a method to construct the initial rule-base for each agent using the solution of the underlying MDP. This drastically improves the performance of the algorithm in comparison with random initialization of the rule-base. We assess the performance of our proposed method using several benchmark problems in comparison with some state-of-the-art methods. Experimental results show that our algorithm achieves better or similar reward when compared with other methods. However, from the runtime point of view, our method is superior to all previous methods. Using a compact fuzzy rule-base not only decreases the amount of memory used but also significantly speeds up the learning phase.

A Residual Gradient Fuzzy Reinforcement Learning Algorithm for Differential Games

In this work, we propose a new fuzzy reinforcement learning algorithm for differential games that have continuous state and action spaces. The proposed algorithm uses function approximation systems whose parameters are updated differently from the updating mechanisms used in the algorithms proposed in the literature. Unlike the algorithms presented in the literature which use the direct algorithms to update the parameters of their function approximation systems, the proposed algorithm uses the residual gradient value iteration algorithm to tune the input and output parameters of its function approximation systems. It has been shown in the literature that the direct algorithms may not converge to an answer in some cases, while the residual gradient algorithms are always guaranteed to converge to a local minimum. The proposed algorithm is called the residual gradient fuzzy actor–critic learning (RGFACL) algorithm. The proposed algorithm is used to learn three different pursuit–evasion differential games. Simulation results show that the performance of the proposed RGFACL algorithm outperforms the performance of the fuzzy actor–critic learning and the Q-learning fuzzy inference system algorithms in terms of convergence and speed of learning.

A Decentralized Fuzzy Learning Algorithm for Pursuit-Evasion Differential Games with Superior Evaders

In this paper, we consider multi-pursuer single-superior-evader pursuit-evasion differential games where the evader has a speed that is similar to or higher than the speed of each pursuer. A new fuzzy reinforcement learning algorithm is proposed in this work. The proposed algorithm uses the well-known Apollonius circle mechanism to define the capture region of the learning pursuer based on its location and the location of the superior evader. The proposed algorithm uses the Apollonius circle with a developed formation control approach in the tuning mechanism of the fuzzy logic controller (FLC) of the learning pursuer so that one or some of the learning pursuers can capture the superior evader. The formation control mechanism used by the proposed algorithm guarantees that the pursuers are distributed around the superior evader in order to avoid collision between pursuers. The formation control mechanism used by the proposed algorithm also makes the Apollonius circles of each two adjacent pursuers intersect or be at least tangent to each other so that the capture of the superior evader can occur. The proposed algorithm is a decentralized algorithm as no communication among the pursuers is required. The only information the proposed algorithm requires is the position and the speed of the superior evader. The proposed algorithm is used to learn different multi-pursuer single-superior-evader pursuit-evasion differential games. The simulation results show the effectiveness of the proposed algorithm.

Dynamic traffic steering based on fuzzy Q-Learning approach in a multi-RAT multi-layer wireless network

The infrastructure of current cellular networks must evolve to cope with the increasing demand for mobile-broadband services. Heterogeneous networks are an attractive solution for operators to expand network capacity, based on deploying different Radio Access Technologies, cell sizes and carrier frequencies in the same environment. As a result, operators gain flexibility to distribute traffic across the different networks (or layers) in order to make a more efficient use of resources and enhance network performance. In this work, a dynamic traffic steering technique in multi-RAT multi-layer wireless networks is proposed. In particular, a fuzzy rule-based reinforcement learning algorithm modifies handover parameters according to a specific policy set by the operator, which typically searches for a trade-off between key performance indicators. Results show that the proposed optimization algorithm provides good flexibility to support different policies by simply adjusting some weighting factors. In addition, the Q-Learning algorithm is shown as an effective solution to adapt the network to context variations, such as those produced in the user spatial distribution.

A fuzzy reinforcement learning algorithm for inventory control in supply chains

In the real world, applications with very large state and action spaces and unknown state transition probability, classical reinforcement learning algorithms usually show poor performance. One way to address the performance problem is to approximate the policy or value function. Fuzzy rule-based systems are amongst the well-known function approximators. This paper presents a Flexible Fuzzy Reinforcement Learning algorithm, in which value function is approximated by a fuzzy rule-based system. The proposed algorithm has a separate module for tuning the structure of fuzzy rules. Moreover, the parameters of the system are tuned during the learning phase. Next, the proposed algorithm is applied to the problem of inventory control in supply chains. In this problem, a fuzzy agent (supplier) should determine the amount of orders for each retailer based on their utility for supplier, by considering its limited supply capacity. Finally, a simulation is performed to show the capability of the proposed algorithm.

Fuzzy Sarsa Learning and the proof of existence of its stationary points

AbstractThis paper provides a new Fuzzy Reinforcement Learning (FRL) algorithm based on critic‐only architecture. The proposed algorithm, called Fuzzy Sarsa Learning (FSL), tunes the parameters of conclusion parts of the Fuzzy Inference System (FIS) online. Our FSL is based on Sarsa, which approximates the Action Value Function (AVF) and is an on‐policy method. In each rule, actions are selected according to the proposed modified Softmax action selection so that the final inferred action selection probability in FSL is equivalent to the standard Softmax formula. We prove the existence of fixed points for the proposed Approximate Action Value Iteration (AAVI). Then, we show that FSL satisfies the necessary conditions that guarantee the existence of stationary points for it, which coincide with the fixed points of the AAVI. We prove that the weight vector of FSL with stationary action selection policy converges to a unique value. We also compare by simulation the performance of FSL and Fuzzy Q‐Learning (FQL) in terms of learning speed, and action quality. Moreover, we show by another example the convergence of FSL and the divergence of FQL when both algorithms use a stationary policy. Copyright © 2008 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society

A simple goal seeking navigation method for a mobile robot using human sense, fuzzy logic and reinforcement learning

This paper proposes a new fuzzy logic-based navigation method for a mobile robot moving in an unknown environment. This method allows the robot obstacles avoidance and goal seeking without being stuck in local minima. A simple Fuzzy controller is constructed based on the human sense and a fuzzy reinforcement learning algorithm is used to fine tune the fuzzy rule base parameters. The advantages of the proposed method are its simplicity, its easy implementation for industrial applications, and the robot joins its objective despite the environment complexity. Some simulation results of the proposed method and a comparison with previous works are provided.

A fuzzy reinforcement learning approach to power control in wireless transmitters

We address the issue of power-controlled shared channel access in wireless networks supporting packetized data traffic. We formulate this problem using the dynamic programming framework and present a new distributed fuzzy reinforcement learning algorithm (ACFRL-2) capable of adequately solving a class of problems to which the power control problem belongs. Our experimental results show that the algorithm converges almost deterministically to a neighborhood of optimal parameter values, as opposed to a very noisy stochastic convergence of earlier algorithms. The main tradeoff facing a transmitter is to balance its current power level with future backlog in the presence of stochastically changing interference. Simulation experiments demonstrate that the ACFRL-2 algorithm achieves significant performance gains over the standard power control approach used in CDMA2000. Such a large improvement is explained by the fact that ACFRL-2 allows transmitters to learn implicit coordination policies, which back off under stressful channel conditions as opposed to engaging in escalating "power wars."