Abstract
In this paper, we consider the problem of deploying a robot from a specification given as a temporal logic statement about some properties satisfied by the regions of a large, partitioned environment. We assume that the robot has noisy sensors and actuators and model its motion through the regions of the environment as a Markov Decision Process (MDP). The robot control problem becomes finding the control policy maximizing the probability of satisfying the temporal logic task on the MDP. For a large environment, obtaining transition probabilities for each state-action pair, as well as solving the necessary optimization problem for the optimal policy are usually not computationally feasible. To address these issues, we propose an approximate dynamic programming framework based on a least-square temporal difference learning method of the actor-critic type. This framework operates on sample paths of the robot and optimizes a randomized control policy with respect to a small set of parameters. The transition probabilities are obtained only when needed. Hardware-in-the-loop simulations confirm that convergence of the parameters translates to an approximately optimal policy.
Highlights
One major goal in robot motion planning and control is to specify a mission task in an expressive and high-level language and convert the task automatically to a control strategy for the robot
This paper extends from [19], in which we proposed an actor-critic method for maximal reachability (MRP) problems, i.e., maximizing the probability of reaching a set of states, to a computational framework that finds a control policy such that the probability of its paths satisfying an arbitrary Linear Temporal Logic (LTL) formula is locally optimal over a set of parameters
We presented a framework that brings together an approximate dynamic programming computational method of the actor critic type, with formal control synthesis for Markov Decision Processes (MDPs) from temporal logic specifications
Summary
One major goal in robot motion planning and control is to specify a mission task in an expressive and high-level language and convert the task automatically to a control strategy for the robot. This paper extends from [19], in which we proposed an actor-critic method for maximal reachability (MRP) problems, i.e., maximizing the probability of reaching a set of states, to a computational framework that finds a control policy such that the probability of its paths satisfying an arbitrary LTL formula is locally optimal over a set of parameters. This set of parameters is designed to tailor to this class of approximate dynamical programming problems. Transpose of a vector x is denoted by xT. · stands for the Euclidean norm. |S| denotes the cardinality of a set S
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.