Computation of the gradient and sensitivity coefficients in sum of squares minimization problems with differential equation models

Abstract

In parameter estimation problems where the system model consists of differential equations, methods for minimizing a sum of squares of residuals objective function require derivatives of the residuals with respect to the parameters being estimated (sensitivity coefficients) or the gradient of the objective function (depending on the numerical optimization method). This paper considers two methods for generating such derivatives: (1) the adjoint equation — gradient formula; and (2) complimentary sensitivity coefficient differential equations. Particular attention is given to the consistency between the method used to solve the model equations and the proper formulation of the additional equations required by the two methods. Two example problems illustrate computational experience using a modified quasi-Newton method with the adjoint method used to generate gradients and applying a modified Gauss-Newton approach with the sensitivity coefficient equations to calculate both the Gauss-Newton matrix and the objective function gradient. Results indicate the superiority of the sensitivity coefficient approach. When comparing the computational effort required by the two methods and the results from the simple examples, it appears that the use of complimentary sensitivity coefficient equations is much more efficient than using only the gradient of the sum of squares function.