Efficient computation of derivatives for solving optimization problems in R and Python using SWIG-generated interfaces to ADOL-C

Abstract

Scripting languages are gaining acceptance because of their ease of use and value for rapid prototyping in many fields, including machine learning and statistics. In the context of algorithmic differentiation, however, the main development effort continues to be concentrated on traditional compiled languages such as Fortran and C/C++, whether source transformation tools or operator overloading tools. There is therefore a need for AD tools for computing derivatives efficiently within scripting languages. ADOL-C is an operator overloading-based C++ library that provides accurate first- and higher order derivatives for applications in C++. SWIG is a preprocessor that uses the C/C++ header files to wrap the API of a library to be callable from scripting languages such as R and Python and several other high-level programming languages. Although every language has its caveats, the overall process of making the C/C++ API available via SWIG is the same for all scripting languages. After an initial effort required per language, because SWIG is an automated interface generator based on the library's actual header files, only minimal effort is required to maintain the scripting interface in sync with upstream developments in the original C/C++ library. In addition to achieving our original goal of creating an interface for R, we were able to generate an interface for Python that proved an order of magnitude faster than the previously implemented interface. This paper gives an overview of the interface generation process, the challenges we encountered with both scripting languages and some numerical results to demonstrate both usefulness and efficiency.