Efficient (Partial) Determination of Derivative Matrices via Automatic Differentiation

Abstract

In many scientific computing applications involving nonlinear systems or methods of optimization, a sequence of Jacobian or Hessian matrices is required. Automatic differentiation (AD) technology can be used to accurately determine these matrices, and it is well known that if these matrices exhibit a sparsity pattern (for all iterates), then not only can AD take advantage of this sparsity for significant efficiency gains, AD can also determine the sparsity pattern itself, with some additional work in the first iteration. Practical nonlinear systems and optimization problems often exhibit patterns beyond just “zero-nonzero.” For example, some elements may be duplicates of other elements at all iterates; some elements may be constant (not necessarily zero) for all iterates. Here we show how the popular graph-coloring approach to AD can be adapted to account for these cases as well, with resulting gains in efficiency. In addition, we address the problem of determining, by AD technology, a prescribed set of the entries of the Jacobian (or Hessian, in the optimization context) matrix.