Continuous‐Time Reinforcement Learning for Force Control

Abstract

This chapter discusses the large and continuous space problems in reinforcement learning (RL) and aims to apply them in the position/force control. A novel hybrid solution is given to take both advantages of discrete‐time and continuous‐time RLs. One method to obtain the centers of the radial basis functions (RBFs) is to use K‐means clustering to partition the input state into K clusters. The hybrid RL achieves sub‐optimal performance without knowledge of the environment dynamics. The chapter utilizes the K‐means algorithm to partition the input space and RBF neural networks to approximate the Q‐function. Hybrid reinforcement learning is used to join the qualities of discrete‐time and continuous‐time reinforcement learning approaches. This new approach finds the sub‐optimal control policy with less learning time. Simulations and experiments are given to verify the reinforcement learning approximation in a position/force control task.