Adjoining Strategies for Multi-layered Programs

Abstract

Some papers present the rules to apply to a straight line program to differentiate it in reverse mode, as well as theoretical complexity measures. This paper recalls these rules, but also summarises the different possible strategies for the differentiation of a multi-level program in reverse mode. We focus on the reverse mode, because the computation of derivatives in reverse order (w.r.t. the computation of original variables) makes the problem much more complicated. A lot of strategies can be applied to generate an adjoint code: these strategies are applied within hand-coded discrete adjoints or within automatically generated adjoints. But they are not necessarily known by both communities, this is why we describe several of them in this paper. Until now, the comparison of these strategies on a code is difficult because no complexity measure has been associated to them. This paper presents complexity measures of the adjoint codes generated using these strategies in terms of execution time and memory requirement. These complexities are based on some elementary measures such as execution time and memory requirement of their sub-programs. For sake of simplicity, in a first phase we neglect the memory management within time costs. In a second phase, we explain how the extra cost first neglected can be accommodated for in the complexity measures.