Reinforcement Learning Control by Direct Heuristic Dynamic Programming

Abstract

AbstractWe present a class of reinforcement learning (RL) controllers that is of actor-critic type. We focus on the direct heuristic dynamic programming (dHDP) method. Over the past several years, new analysis and synthesis of the dHDP as a reinforcement learning controller as well as impressive applications of the dHDP have emerged. In this chapter we provide a summary on how the dHDP works, what analytical properties it possesses, and how it was applied and implemented in a wearable robot for the automatic tuning of the prosthesis control parameters with a human user in the loop.KeywordsReinforcement learningAdaptive/approximation dynamic programOptimal controlStochastic gradient descentAdaptive optimal controlDynamic programmingBellman optimalityDirect heuristic dynamic programmingRobotic prosthesis control