Symbolic Differentiation Research Articles

On computational differentiation

Numerical derivatives play an important role in many computations. In many applications, the cost associated with evaluation of numerical derivatives may be significant. Dramatic improvements in the speed of such calculations can be obtained through careful consideration of how these derivatives are computed. This paper reviews several ways in which numerical derivatives can be evaluated: hand-coding, finite difference approximations, reverse polish notation evaluation, symbolic differentiation, and automatic differentiation. It is concluded that automatic differentiation has significant advantages over all other approaches. Several ways of improving the efficiency of obtaining derivatives in an interpretive, symbolic environment are discussed. Example problems are compared to illustrate these improvements.

Automatic derivation of thermodynamic property functions using computer algebra

The commercial computer algebra system known as Maple has been extended in order to make possible the automatic symbolic derivation and differentiation of thermodynamic property functions of mixtures with arbitrary numbers of components. The tools described here could be useful in the rapid development of new models for thermodynamic properties (and their derivatives).

Symbolic differentiation library for simulation of multibody rigid systems

Abstract This article presents a symbolic kernel dedicated to differentiation which is used in a system for both animation and simulation. Theoretical aspects about complexity of motion equations using a Lagrangian formalism are discussed using research previously carried out in differentiation area. From this study we derive a method that generate motion equations whose cost is linear with the number of degrees of freedom and whose implementation is realized using this kernel. Experimental results are presented using this method and a complex physical mechanism is described in order to prove the ability of our system to deal with such mechanisms.

Finding apparent horizons in numerical relativity.

This paper presents a detailed discussion of the ``Newton's method'' algorithm for finding apparent horizons in 3+1 numerical relativity. We describe a method for computing the Jacobian matrix of the finite differenced $H(h)$ function by symbolically differentiating the finite difference equations, giving the Jacobian elements directly in terms of the finite difference molecule coefficients used in computing $H(h)$. Assuming the finite differencing scheme commutes with linearization, we show how the Jacobian elements may be computed by first linearizing the continuum $H(h)$ equations, then finite differencing the linearized (continuum) equations. We find this symbolic differentiation method of computing the $H(h)$ Jacobian to be {\em much} more efficient than the usual numerical perturbation method, and also much easier to implement than is commonly thought. When solving the discrete $H(h) = 0$ equations, we find that Newton's method generally converges very rapidly. However, if the initial guess for the horizon position contains significant high-spatial-frequency error components, Newton's method has a small (poor) radius of convergence. This is {\em not} an artifact of insufficient resolution in the finite difference grid; rather, it appears to be caused by a strong nonlinearity in the continuum $H(h)$ function for high-spatial-frequency error components in $h$. Robust variants of Newton's method can boost the radius of convergence by O(1) factors, but the underlying nonlinearity remains, and appears to worsen rapidly with increasing initial-guess-error spatial frequency. Using 4th~order finite differencing, we find typical accuracies for computed horizon positions in the $10^{-5}$ range for $\Delta\theta = \frac{\pi/2}{50}$.

Model Selection in Ring-Recovery Models Using Score Tests

A strategy is outlined for selecting models for ring-recovery data using score tests. The approach is particularly valuable in avoiding unnecessary fitting of complicated, multiparameter models to data that do not require models of such complexity. Difficulties of convergence of iterative methods and potential boundary-estimation problems are thereby reduced. Data analyzed in Freeman and Morgan (1992, Biometrics 48, 217-236) are reanalyzed using score tests. These tests are repeated using both numerical and symbolic differentiation and also using both observed and expected information. We recommend using the expected information, and find that numerical differentiation is as good as symbolic differentiation. Motivated by the need to-describe a wide range of models succinctly, we also provide a new general notation for ring-recovery models.

Symbolic Differentiation for Fixed-Structure Controller Synthesis

A symbolic matrix differentiation routine called MatGrad is developed for computing gradient expressions for fixed-structure controller synthesis. These expressions can be used with gradient-based optimization algorithms to compute controllers that meet H2, H∞, μ and Pole placement objectives in both continuous and discrete time. MatGrad is written in Mathematica and utilizes routines from the noncommutative algebra package NCAlgebra developed by Helton and Miller.

Symbolic differentiation of the velocity mapping for a serial kinematic chain

This paper symbolically calculates the derivatives of a serial kinematic chain's velocity mapping. The derivatives are taken with respect to a change in one of the joint angles, as well as with respect to time. The result is a linear mapping, whose explicit form depends on the mathematical representation chosen for the Cartesian twist of the end effector. Three of the most common representations are treated: the “inertial”, “body-fixed” and “hybrid” representations. Numerical calculations illustrate the correctness of the presented results.

A method for improving condition number of chemical processes by sensitivity analysis

Simple and effective method for improving Euclidean Norm Condition Number for chemical processing system is presented. The singular value sensitivities of Freudenberg et. al. (1982) is used to estimate the behavior of singular values of process transfer function matrix when design is changed, then the condition number can be calculated straightforwardly. The method requires explicit dependencies of each transfer function matrix elements on design parameters. These dependencies can be obtained either by symbolic differentiation in the form of explicit function of design parameters, or by numerical perturbation studies for units with large and complicated models. Gerschgorin-type lower bound for minimum singular value is introduced to detect the large discrepancies near singular point due to linear limitation of sensitivities. The case study is performed to show the efficiency of the proposed method.

Computing multivariable Taylor series to arbitrary order

Automatic differentiation, manipulating numerical vectors of coefficients, is the efficient way to compute multivariable Taylor series. This does not require symbolic differentiation or numerical approximation but uses exact formulas applied to numerical arrays. Arrays of Taylor series coefficients of any elementary function can be built-up, as the array for each component (combination or function) is a combination of its argument arrays. The functions TIMES and EXP display the algorithmic ideas that enable all of the other standard functions. We study the interesting recursive formulas for these combinations, the resulting algorithms, and the implementation in APL. To handle all coefficients in n variables up to order m , the arrays are hyper-pyramid data structures, considered conceptually as n -dimensional but implemented as one-dimensional arrays. Unlike previous work, this implementation does not require huge arrays for binomial coefficients and indirect referencing. This APL*PLUS III implementation loops through one nested reference array and takes sub-arrays from another for a practical solution to this problem that can make tremendous demands on time and space.

A general and flexible method of problem definition in structural optimization systems

This paper describes the implementation of a general and flexible method of formulating problems of mathematical programming in structural optimization systems. The method enables the formulation and solution of problems involving scalar, integral, min/max, max/min and possibly non-differentiable user defined functions in any conceivable mix. The mathematical formulation is based on the bound formulation, and the implementation specific details involve a parser capable of interpreting and performing symbolic differentiation of the user defined functions.

ADS expert

ADS Expert is an integrated system that (a) provides suggestions about suitable optimization algorithms for a particular structural optimization problem, (b) it offers explanations and guide lines, in the form of hypertext, that refer to the optimization strategy, optimizer and one dimensional search method, (c) it produces the required gradients for the objective function and the constraints using symbolic differentiation, (d) establishes a direct linking with ADS routines and (e) it monitors the evolution of the optimization process. The system can be used as an educational tool for structural optimization and as an efficient integrated system for small to medium size optimization problems for which the objective function and the constraints are given in the form of explicit functions of the design variables. The system is a Windows application and runs quite satisfactorily on PCs.

Mechanism analysis by matrix reduction II. Kinematics

A matrix-based method is presented for calculating velocities and accelerations in two- and three-dimensional mechanisms. The method involves no symbolic manipulation or differentiation and is comparatively simple to implement as a computer algorithm, in that the same arithmetic process is used regardless of the configuration of the mechanism, and regardless of its complexity. The quantity of computation varies linearly with the number of moving parts in the mechanism.

Frontier Tales: The Narrative Contruction of Cultural Borders in Twentieth-Century California

Maclntyre sought to raise a series of crucial questions in an address before the Eighty-First Annual Meeting of the Eastern Division of the American Philosophical Association. What is the best way to evaluate two competing languages, conceptual schemes, or cultural approaches to being in the world? May conceptual schemes or theoretical languages be so radically different as to be fully incommensurable, that is, incapable of intertranslation or meaningful comparison?2Maclntyre, drawing upon the example of the sixteenthcentury Zuni Indian and Spanish colonial frontier, contended that some languages or conceptual schemes might well be partially untranslatable, but this did not necessarily preclude commensurability. His remarks suggest additional questions, namely, what does constitute a frontier situation? What is involved in the symbolic differentiation of one conceptual scheme, one interpretive group—one culture—from another? We might recognize here the familiar problem of boundary construction, the social process of creating a common group identity that marks it as different from others.3

Automatic differentiation in MATLAB

Automatic differentiation is a chain rule based technique for derivative evaluation introduced by Wengert (1964) (see also Wilkins (1964)) and further developed by Rall (1980, 1981, 1983). With the development of more powerful programming languages automatic differentiation has received increased attention as a viable alternative to symbolic differentiation and finite difference methods. (See Griewank (1989).) Here we present a C language implementation of automatic differentiation that can be called from MATLAB, a highly interactive robust scientific computing environment. Our implementation accepts a multivariate function and a point as input returning a function value and gradient, hence the Jacobian at a point can be constructed. The implementation along with examples is described. The code is available from the authors.

Symbolic Determination of Parameter Sensitivities and Adjoint Differential Equations: A Fortran Source Code Generator for Their Numerical Integration

This paper describes a software for the automated generation of FORTRAN source code sub-routines for the numerical integration of non-linear differential equations describing dynamical systems. The use of directed graphs for representing the systems greatly simplifies the definition of models and allows an efficient use of symbolic differentiation rules for generating the source code for: i) the sensi-tivity equations for the state/output trajectories with respect to model parameters; ii) the differential equations for computing the adjoint variables of optimal control problems. Possible applications of this software include model building, sensitivity analysis, identification, optimal experiment design and numerical solution of optimal control problems.

Parametric Differentiation Revisited

A number of now-ancient texts in advanced calculus employed the method of parametric differentiation to deduce evaluations for variety of complicated definite integrals (e.g., [4,7]). In his autobiography [3], R. P. Feynman mentioned how he frequently used this approach when confronted with difficult integrations associated with mathematical and physical problems. He referred to that method as a different box of tools. Most modern texts either ignore the subject or provide only one or two examples to illustrate theorems on the uniform convergence of improper integrals. (See [8] for more recent treatment of integrals.) However, the method of parametric differentiation has much broader scope than this. It can, in fact, be introduced into the elementary calculus to simplify the treatment of partial fractions with repeated factors, to carry out integrations ordinarily handled by parts, and to treat integrals that use substitutions. When tied in with other mathematical machinery (e.g., complex variables), it can be used to deduce formulas for special functions and to solve problems in differential equations that involve iterated operators. Finally, the method can be conveniently used in conjunction with mathematical software packages that handle symbolic differentiations. In this note, we give three examples from calculus to illustrate the flexibility of this approach at the elementary level. One of the integrals treated is connected with the t-distribution. We also note, without providing complete discussion, how this technique can be used to evaluate the Riemann zeta function at 2p, p = 1, 2,.. ., and how it can be used in differential equations. A more extensive set of applications is available from the author. In the work to follow, we call upon the Leibniz product rule for differentiation as well as theorems that permit interchanging orders of differentiation and integration (see [5, 6]). We also need to make frequent use of the formula

Symbolic computation of flow in a rotating pipe

AbstractA perturbation solution of the fully developed flow through a pipe of circular cross‐section, which rotates uniformly around an axis oriented perpendicularly to its own, is considered. The perturbation parameter is given by R = 2Ωa2/ν in terms of the angular velocity Ω, the pipe radius a and the kinematic viscosity ν of the fluid. The two coupled non‐linear equations for the axial velocity ω and the streamfunction ϕ of the transverse (secondary) flow lead to an infinite system of linear equations. This system allows first the computation of a given order ϕn, n ϕ 1, of the perturbation expansion ϕ = ∑ Rnϕn in terms of ωn‐1, the (n‐1)‐th order of the expansion ω = ∑ Rnωn, and of the lower orders ϕ1,…,ϕn − 1. Then it permits the computation of ωn from ω0,…,ωn − 1 and ϕ1,…,ϕ;n. The computation starts from the Hagen–Poiseuille flow ω0, i.e. the perturbation is around this flow.The computations are performed analytically by computer, with the REDUCE and MAPLE systems. The essential elements for this are the appropriate co‐ordinates: in the complex co‐ordinates chosen the two‐dimensional harmonic (Laplace, Δ) and biharmonic (Δ2) operators are ideally suited for (symbolic) quadratures. Symmetry considerations as well as analysis of the equations for ωn, ϕn and of the boundary conditions lead to general (polynomial) formulae for these functions, with coeffcients to be determined. Their determination, order by order, implies, in complex co‐ordinates, only (symbolic) differentiation and quadratures. The coefficients themselves are polynomials in the Reynolds number c of the (unperturbed) Hagen–Poiseuille flow. They are tabulated in the paper for the orders n ⩽ 6 of the perturbation expansion.

A proposed numerical algorithm for solving nonlinear index problems

We present and demonstrate a numerical algorithm to detect and solve mixed systems of nonlinear ordinary differential and algebraic equations that have an index problem. Symbolic differentiation is used to generate the equations needed; symbolic elimination is not required as in many other algorithms. We show that the so-called initialization and propagation problems are in fact the same problem; they can therefore be handled by a common approach. Numerical as well as structural singularities arising in index problems are accommodated. In principle, the algorithm can be used with any integration scheme, including an explicit one. This algorithm is suitable for solving index problems that previously could not be solved

Towards the automatic numerical solution of partial differential equations

Adaptive mesh enrichment techniques are becoming recognized as a means of automating many of the decisions associated with the computer solution of partial differential equations. Towards this end, we describe an intelligent interface where partial differential equations and their data may be entered in a relatively natural mathematical language. Fortran subroutines for functions and Jacobians that are required by the numerical software are generated automatically with symbolic differentiation used to evaluate Jacobians. We further describe the use of this symbolic interface with an adaptive local mesh refinement finite element method for two-dimensional parabolic partial differential systems. Solutions are calculated using Galerkin's method with a piecewise bilinear polynomial basis in space and backward Euler integration in time. Estimates of the local discretization error, obtained by a p-refinement technique, are used to control a local h-refinement strategy where finer grids are recursively introduced in regions where a prescribed tolerance is exceeded. Fine grids at a given level of refinement may overlap each other and independent solutions are generated on each of them. A version of the Schwarz alternating principle is used to coordinate solutions between overlapping fine grids.

An APL approach to differential calculus yields a powerful tool

An APL workspace is developed for the purpose of calculating numerical values of derivatives. The heart of the method, called automatic differentiation, is manipulation of numerical vectors --- an APL way of thinking. The method is not symbolic manipulation as taught in calculus, nor is it approximation as taught in numerical analysis. Automatic differentiation is implemented as a workspace (called GRADIENT) of 14 simple, one-line, numerical vector functions that perform all of the formal differentiation rules. The ideas are introduced for simple first derivatives of single variable functions. We then show that the GRADIENT workspace enables the calculation of all first-order partial derivatives of any typical function in any number of variables. This workspace is used as a tool in the solution of systems of nonlinear equations by Newton's method. Finally, we discuss how APL concepts led to a new approach to higher-order derivatives. This approach enables the calculation of higher-order derivatives for problems that overwhelm commercial symbolic differentiation programs; moreover, the resulting accuracy cannot be obtained by numerical approximation techniques.