The use of pontryagin estimators for on-line optimal control sequence estimation: The truck backer-upper case study

Abstract

The Pontryagin optimality principle can be used in conjunction with on-line (or real-time) measurements of state data to build a local model of the control law. In this paper, we discuss and refine the use of this technique in the context of the simple truck backer-upper problem. We first compare the use of feedforward, associative and CMAC neural architectures for the local control model encoding. Algorithm implementation is then done using the CMAC architecture because of its speed of learning and local scoping. We build temporal difference state prediction models for the truck dynamics and then use these predictions to build an estimate of the best control action to take. This control action is constructed from a depth first tree search used in conjunction with optimal control information obtained by solving locally scoped control problems via the Pontryagin optimality principle. The state to control model can then be encoded into a variety of function approximation models.