Relationship between maximum principle and dynamic programming in presence of intermediate and final state constraints

Olivier Bokanowski,Hasnaa Zidani,Anya Désilles

doi:10.1051/cocv/2021084

Abstract

In this paper, we consider a class of optimal control problems governed by a differential system. We analyze the sensitivity relations satisfied by the co-state arc of the Pontryagin maximum principle and the value function that associates the optimal value of the control problem to the initial time and state. Such a relationship has been already investigated for state-constrained problems under some controllability assumptions to guarantee Lipschitz regularity property of the value function. Here, we consider the case with intermediate and final state constraints, without any controllability assumption on the system, and without Lipschitz regularity of the value function. Because of this lack of regularity, the sensitivity relations cannot be expressed with the sub-differentials of the value function. This work shows that the constrained problem can be reformulated with an auxiliary value function which is more regular and suitable to express the sensitivity of the adjoint arc of the original state-constrained control problem along an optimal trajectory. Furthermore, our analysis covers the case of normal optimal solutions, and abnormal solutions as well.

Highlights

The aim of this paper is to study the relationship between the Pontryagin maximum principle and dynamic programming for control problems in presence of final and/or intermediate state constraints
(1.1a) (1.1b) (1.1c) (1.1d) where the dynamics f : [0, T ] × Rd × U → Rd, the distributed cost : [0, T ] × Rd × U → R, the final cost φ : Rd → R, the constraints gi : Rd → R are smooth functions
An admissible process is a pair of functions (x, u) where u is an admissible control function and x is an absolutely continuous solution of

Summary

Introduction

The aim of this paper is to study the relationship between the Pontryagin maximum principle and dynamic programming for control problems in presence of final and/or intermediate state constraints. It is known that when the control problem does not include any state constraint (for instance if gi ≡ 0, ∀i), under assumptions (H0)-(H3), the value function θ is locally Lipschitz continuous and can be characterized as unique solution of a Hamilton-Jacobi equation, see Chapter III of [3] and [7, 19, 20]. We are interested in the case when the control problem is in presence of final or intermediate state constraints (m ≥ 1) In this case the value function fails to be continuous unless some controllability assumptions are satisfied We will use the maximized Hamiltonian function H : [0, T ] × Rd × Rd+1 × R defined, for (s, x, px, pz) ∈ [0, T ] × Rd × Rd × R, as: H(s, x, px, pz) := max − f (s, x, u) · px + (s, x, u) pz

Main results

The case of Mayer problem with final state constraints

Findings

Sensitivity relations for a control problem with intermediate costs