Chain Rule Of Differential Calculus Research Articles

Automatic Differentiation in ROOT

In mathematics and computer algebra, automatic differentiation (AD) is a set of techniques to evaluate the derivative of a function specified by a computer program. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, etc.), elementary functions (exp, log, sin, cos, etc.) and control flow statements. AD takes source code of a function as input and produces source code of the derived function. By applying the chain rule repeatedly to these operations, derivatives of arbitrary order can be computed automatically, accurately to working precision, and using at most a small constant factor more arithmetic operations than the original program. This paper presents AD techniques available in ROOT, supported by Cling, to produce derivatives of arbitrary C/C++ functions through implementing source code transformation and employing the chain rule of differential calculus in both forward mode and reverse mode. We explain its current integration for gradient computation in TFormula. We demonstrate the correctness and performance improvements in ROOT’s fitting algorithms.

A forward operator and its adjoint for GPS slant total delays

AbstractIn a recent study we developed a fast and accurate algorithm to compute Global Positioning System (GPS) Slant Total Delay (STDs) utilizing numerical weather model data. Having developed a forward operator we construct in this study the tangent linear (adjoint) operator by application of the chain rule of differential calculus in forward (reverse) mode. Armed with these operators we show in a simulation study the potential benefit of GPS STDs in inverse modeling. We conclude that the developed operators are tailored for three (four)‐dimensional variational data assimilation and/or travel time tomography.

On optimality preserving eliminations for the minimum edge count and optimal Jacobian accumulation problems in linearized DAGs

The minimum edge count (MEC) and optimal Jacobian accumulation problems in linearized directed acyclic graphs (DAGs) result from the combinatorics induced by the associativity of the chain rule of differential calculus. This paper discusses a suitable graph formalism followed by proving a number of results that yield a considerable search space reduction for both problems. An algorithmic link between numerical analysis and theoretical computer science is established. Although both problems are believed to be NP-hard in general, a linear-time algorithm is presented solving the MEC problem for a specified subclass of l-DAGs.

Automatic differentiation for portable process systems models

AbstractThe development of complicated mathematical models in process systems engineering requires a lot of human effort. The high development cost of complex models can be reduced by reusing submodels created by other modelers. The exchange of ready‐to‐use components requires a common standardized data format. CapeML is an XML‐based meta language defined for describing equation‐based models in process system engineering. This domain specific language provides a convenient level of abstraction to which auxiliary code transformations can be applied. Automatic differentiation is a semantic transformation of programs that describe mathematical functions. It applies the chain rule of differential calculus and generates a new program that additionally calculates the derivatives of the underlying function. ADiCape is an XSLT‐based tool for automatic augmentation of CapeML models with the derivative information. A platform‐independent CapeML representation of a model together with additional code to calculate derivatives is available in a bundle and can be reused by the engineers. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Looking for narrow interfaces in automatic differentiation using graph drawing

Automatic differentiation is a powerful technique for evaluating derivatives of functions given in the form of a high-level programming language such as Fortran, C, or C++. This technique is superior, in terms of accuracy, to numerical differentiation because it avoids the truncation error involved in divided difference approximations. In automatic differentiation, the program is treated as a potentially very long composition of elementary functions to which the chain rule of differential calculus is applied over and over again. Because of the associativity of the chain rule, there is room for different strategies computing the same numerical results but whose computational cost may vary significantly. Several strategies exploiting high-level structure of the underlying computer code are known to reduce computational cost as opposed to blindly applying automatic differentiation. An example includes “interface contraction” where one takes advantage of the fact that the number of variables passed between subroutines is small compared with the number of propagated directional derivatives. Unfortunately, these so-called narrow interfaces are not immediately available. The present study investigates the use of the VCG graph drawing tool to recognize narrow interfaces in the computational graph, a certain directed acyclic graph used to represent data dependences of variables in the underlying computer code.

TIME-PARALLEL COMPUTATION OF PSEUDO-ADJOINTS FOR A LEAPFROG SCHEME

The leapfrog scheme is a commonly used second-order difference scheme for solving differential equations. If Z(t) denotes the state of a system at a particular time step t, the leapfrog scheme computes the state at the next time step as Z(t+1)=H(Z(t),Z(t-1),W), where H is the nonlinear time-stepping operator and W represents parameters that are not time-dependent. In this note, we show how the associativity of the chain rule of differential calculus can be used to compute a so-called adjoint, the derivative of a scalar-valued function applied to the final state Z(T) with respect to some chosen parameters, efficiently in a parallel fashion. To this end, we (1) employ the reverse mode of automatic differentiation at the outermost level, (2) use a sparsity-exploiting version of the forward mode of automatic differentiation to compute derivatives of H at every time step, and (3) exploit chain rule associativity to compute derivatives at individual time steps in parallel. We report on experimental results with a 2-D shallow water equations model problem on an IBM SP parallel computer and a network of Sun SPARCstations.

Exploiting Intermediate Sparsity in Computing Derivatives for a Leapfrog Scheme

The leapfrog scheme is a commonly used second-order method for solving differential equations. If Z(t) denotes the state of a system at a particular time step t, this integration scheme computes the state at the next time step as Z(t + 1) e H(Z(t), Z(t − 1), W), where H is the nonlinear timestepping operator and W are parameters that are not time-dependent. In this note, we show how the associativity of the chain rule of differential calculus can be used to expose and exploit intermediate derivative sparsity arising from the typical localized nature of the operator H. We construct a computational harness that capitalizes on this structure while employing automatic differentiation tools to automatically generate the derivative code corresponding to the evaluation of one time step. Experimental results with a 2-D shallow water equations model on IBM RS/6000 and Sun SPARCstations illustrate these issues.

Response to Michel, Kleijnen and Permut

Professors Michel and Permut have identified an important application of metamodels. Metamodels may be used to determine the need for accuracy in a forecasting model that provides input to a decision model. This suggests an interesting possibility—an analyst may wish to construct a metamodel of the forecasting model as well. This metamodel can be combined with the metamodel of the decision model (using the chain rule of differential calculus) to provide a single metamodel for the complex of models.