Multithreaded derivative computation with generated libraries

Abstract

The computation of derivatives via automatic differentiation is a valuable technique in many science and engineering applications. While the implementation of automatic differentiation via source transformation yields the highest efficiency results, particularly for gradient computations, the implementation via operator overloading remains a viable alternative for some application contexts. Examples include the computation of higher order derivatives or cases where C++ as the language of choice still proves to be too complicated for the currently available source transformation tools. In this paper, we utilize a code generator to create libraries that overload intrinsics for derivative computation, and discuss approaches to improve the efficiency of the generated code. We first discuss the use of limited loop unrolling, but the main focus of the paper is multithreaded derivative computation, in particular an asynchronous scheme for C++ and a synchronous scheme for Fortran. We present test results obtained with a proof-of-concept implementation.