Abstract
ABSTRACTPolicy search algorithms have facilitated application of Reinforcement Learning (RL) to dynamic systems, such as control of robots. Many policy search algorithms are based on the policy gradient, and thus may suffer from slow convergence or local optima complications. In this paper, we take a Bayesian approach to policy search under RL paradigm, for the problem of controlling a discrete time Markov decision process with continuous state and action spaces and with a multiplicative reward structure. For this purpose, we assume a prior over policy parameters and aim for the ‘posterior’ distribution where the ‘likelihood’ is the expected reward. We propound a Markov chain Monte Carlo algorithm as a method of generating samples for policy parameters from this posterior. The proposed algorithm is compared with certain well-known policy gradient-based RL methods and exhibits more appropriate performance in terms of time response and convergence rate, when applied to a nonlinear model of a Cart-Pole benchmark.
Highlights
Reinforcement learning (RL) can be considered as a topic of interest among the researchers in the field of machine learning, control and robotics; see Sutton and Barto (1998) for an introduction
We considered the control problem of an Markov decision processes (MDP) with continuous state and action spaces in finite time horizon and presented a novel policy search method for a class of parameterized polices in RL under a Bayesian learning framework
The proposed learning approach is an MCMC algorithm that uses estimates of the expected multiplicative total reward which are attained with an sequential Monte Carlo (SMC) algorithm
Summary
SYSTEMS SCIENCE & CONTROL ENGINEERING: AN OPEN ACCESS JOURNAL 2018, VOL. Vahid Tavakol Aghaeia, Ahmet Onata and Sinan Yıldırımb aFaculty of Engineering and Natural Sciences, Mechatronics Engineering, Sabancı University, Istanbul, Turkey; bFaculty of Engineering and Natural Sciences, Industrial Engineering, Sabancı University, Istanbul, Turkey
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
