Optimization and sensitivity analysis for multiresponse parameter estimation in systems of ordinary differential equations

Abstract

Methodology for the simultaneous solution of ordinary differential equations (ODEs) and associated parametric sensitivity equations using the Decoupled Direct Method (DDM) is presented with respect to its applicability to multiresponse parameter estimation for systems described by nonlinear ordinary differential equations. The DDM is extended to provide second order sensitivity coefficients and incorporated in multiresponse parameter estimation algorithms utilizing a modified Newton scheme as well as a hybrid Newton/Gauss—Newton optimization algorithm. Significant improvements in performance are observed with use of both the second order sensitivities and hybrid optimization method. In this work, our extension of the DDM to evaluate second order sensitivities and development of new hybrid estimation techniques provide ways to minimize the well-known drawbacks normally associated with second-order optimization methods and expand the possibility of realizing their benefits, particularly for multiresponse parameter estimation in systems of ODEs. The Decoupled Direct Method (DDM) for the simultaneous solution of ODEs and the associated first order parametric sensitivity equations was extended to include solution of the second order parametric sensitivity equations. The extended DDM was then used to develop a full Newton scheme for multiresponse parameter estimation in systems of ODEs. A hybrid method was also developed which allowed switching between this Newton scheme and a generalized Gauss—Newton scheme. The switching procedure was based on the magnitude of the Bates—Watts convergence criterion (Bates and Watts, 1985). Both methods were compared to a generalized Gauss—Newton method and also to a hybrid method which incorporated the DGW update method and the generalized Gauss—Newton method. This latter method was based on a modification of the single reponse nonlinear least squares algorithm of Dennis et al. (1981). In comparing the four estimation methods as applied to four test examples it was observed that methods which incorporated second order sensitivity coefficients in the evaluation of the Hessian matrix were more robust and reliable. The modified full Newton method and the hybrid (Newton/Gauss-Newton) method successfully converged despite problems with high correlation between parameters, large residuals and poor initial estimates. The use of a hybrid method incorporating the DGW update formula to estimate the Hessian matrix was also observed to be robust but not to the extent observed for the full second order methods. Of the two methods employing second order sensitivities, the new hybrid method was significantly more efficient. There are well known drawbacks to requiring second-order derivatives in parameter estimation algorithms. Additional coding for calculating the derivatives, increased computational time and storage are prime factors which increase dramatically with the size of the problem. Some of these problems can be circumvented. For example, the use of symbolic computation packages greatly reduced the amount of coding required. In fact, with packages such as MAPLE, only coding of the model equations was required. Our extension of the DDM to evaluate second-order sensitivities and development of new hybrid estimation techniques provide ways to minimize these drawbacks and expand the possibility of realizing the benefits of second order methods, particularly for multirepsonse parameter estimation in systems of ODEs. It is clear that the efficient and accurate evaluation of second order sensitivity coefficients enhances convergence. In addition, this allows evaluation of the extent of nonlinearity that the model experiences upon variation of the model parameters. In particular, we are currently exploring for the evaluation of curvature measures similar to those developed by Bates and Watts (1985) for complex nonlinear multiresponse regression models. Such curvature measures have been shown to be useful diagnostic tools in nonlinear regression analysis for single responses. As a final, supplemental comment, we note that many of the problems in multiresponse parameter estimation arise as a consequence of limitations in the model building strategy. In particular, starting with too large or complex a model form with limited information in the data inevitably leads to problems in estimation. We strongly recommend beginning with a simplified model with informative data, collected from carefully designed experiments, followed by subsequent modification of the model and further data collection until an adequate fit is achieved. Suitable transformation of the parameters can also improve the estimation. With such a strategy, a simple first order estimation method will usually suffice. Nevertheless, situations do arise where second order methods are required making hybrid methods like those developed here valuable.