Optimal control of compartmental models: The exact solution

Abstract

We formulate a control problem for positive compartmental systems formed by nodes (buffers) and arcs (flows). Our main result is that, on a finite horizon, we can solve the Pontryagin equations in one shot without resorting to trial and error via shooting. As expected, the solution is bang–bang and the switching times can be easily determined. We are also able to find a cost-to-go-function, in an analytic form, by solving a simple nonlinear differential equation. On an infinite horizon, we consider the Hamilton–Jacobi–Bellman theory and we show that the HJB equation can be solved exactly. Moreover, we show that the optimal solution is constant and the cost-to-go function is linear and copositive. This function is the solution of a nonlinear equation. We propose an iterative scheme for solving this equation, which converges in finite time. We also show that an exact solution can be found if there is a positive external disturbance affecting the process and the problem is formulated in a minsup framework. We finally provide illustrative examples related to flood control and epidemiology.