Backward Differentiation Research Articles

Local modeling method for multi-field coupling of electric-magnetic-thermal-structure of pulsed high current linear driving device

The pulsed high current linear driving device operates under extreme conditions, and various forms of metal damages will reduce the service life of the device. At present, the multi-physics coupling mechanism of pulsed high current linear driving device is still unclear, and the multi-parameter diagnosis method in the laboratory environment is limited. Therefore, it is urgent to clarify the evolution process of multiple physical parameters through numerical modeling methods, in order to guide the optimization of the overall performance and improve the service life of the device. In this work, mathematical and physical models of electromagnetic field, temperature field and structural field under dynamic conditions are established. The local solution is carried out by using the characteristics of rail reverse motion and the invariant physical quantities at the distal end of the contact. The constraint equations of the non-equipotential surface of the rail entrance and the armature-rail interface conditions under the technical framework are derived. The constraint equations are applied by the penalty function method. The model also takes into account the practical factors such as the temperature dependence of the material properties, thermal stresses, and the frictional heat of the contact surface. The finite element discrete format of the electromagnetic field and the temperature field is solved in the form of Euler’s backward differentiation, and the structural field is solved by the Newmark method. The reliability of the model is verified by comparing the calculation results with the numerical tools EMAP3D and Comsol under the same configuration and input conditions, as well as related experiments. Through the numerical simulation of the C-type armature, the typical evolution process of the corresponding multi-parameter is obtained. During sliding electrical contact, the velocity skin effect becomes more pronounced with velocity increasing. The current is gradually concentrated on the surface of the rail, and the highest current density is found at the rear edge of the contact surface and at the edge of the outer arm of the armature. Moreover, the magnetic induction intensity at the tail of the contact surface continues to shrink over time. The heat-concentrated region appears at the top edge of the contact surface, and with time going by, it extends along the sliding direction and bottom direction of the armature. In addition, there is peak stress in the front of the rail contact and significant stress at the armature throat. When the local stress at the throat of the armature exceeds the corresponding yield strength, it can cause serious deformation or even fracture of the armature.

Efficient differentiable quadratic programming layers: an ADMM approach

Recent advances in neural-network architecture allow for seamless integration of convex optimization problems as differentiable layers in an end-to-end trainable neural network. Integrating medium and large scale quadratic programs into a deep neural network architecture, however, is challenging as solving quadratic programs exactly by interior-point methods has worst-case cubic complexity in the number of variables. In this paper, we present an alternative network layer architecture based on the alternating direction method of multipliers (ADMM) that is capable of scaling to moderate sized problems with 100–1000 decision variables and thousands of training examples. Backward differentiation is performed by implicit differentiation of a customized fixed-point iteration. Simulated results demonstrate the computational advantage of the ADMM layer, which for medium scale problems is approximately an order of magnitude faster than the state-of-the-art layers. Furthermore, our novel backward-pass routine is computationally efficient in comparison to the standard approach based on unrolled differentiation or implicit differentiation of the KKT optimality conditions. We conclude with examples from portfolio optimization in the integrated prediction and optimization paradigm.

Robust corner detection with fractional calculus for magnetic resonance imaging

In medical image processing and analyzing, corners are one of the most important features, which makes point-based registration and diagnosis studies essential. However, existing corner detectors are impressible to noise. This work proposes an order-steerable and robust image corner detector, while a fractional gradient operator consisting of fractional-order forward and backward differentiation and integration is derived. The fractional gradient operator achieving a 90° phase shift as the traditional first derivative does provides a new idea to calculate the gradient in an image. Qualitative and quantitative comparison experiments with exiting detection methods based on integer-order derivative are performed on simulated and real medical slices. The comparison experiments indicate that the present corner detector in this paper allows very strong response for corner detection and provides a better ability of detecting and locating image corners and robustness against noise.

A finite element data assimilation method for the wave equation

We design a primal-dual stabilized finite element method for the numerical approximation of a data assimilation problem subject to the acoustic wave equation. For the forward problem, piecewise affine, continuous, finite element functions are used for the approximation in space and backward differentiation is used in time. Stabilizing terms are added on the discrete level. The design of these terms is driven by numerical stability and the stability of the continuous problem, with the objective of minimizing the computational error. Error estimates are then derived that are optimal with respect to the approximation properties of the numerical scheme and the stability properties of the continuous problem. The effects of discretizing the (smooth) domain boundary and other perturbations in data are included in the analysis.

Numerical simulation of Swift–Hohenberg equation by the fourth-order compact scheme

A high-order accurate compact scheme for the Swift–Hohenberg equation is presented in this paper. We discretize the Swift–Hohenberg equation by a fourth-order compact finite difference formula in space and a backward differentiation with second-order accurate in time, respectively. A stabilized splitting scheme is presented and a Newton-type iterative method is introduced to deal with the nonlinear term. Therefore, a large time step can be used. The resulting discrete systems are solved by a fast and efficient nonlinear multigrid solver. Adaptive time step method is implemented to reduce the computational cost. Various numerical simulations including a convergence test of the proposed scheme, comparison with second-order scheme, a test of the stability of the proposed scheme, extension of the adaptive time step method, comparison with the phase field crystal equation, a study of the effect of computational domain and boundary condition, and an evolution of Swift–Hohenberg equation in three dimensions, are performed to demonstrate the efficiency of our proposed method.

Stochastic automatic differentiation: automatic differentiation for Monte-Carlo simulations

In this paper we re-formulate the automatic differentiation (and in particular, the backward automatic differentiation, also known as adjoint automatic differentiation, AAD) for random variables. While this is just a formal re-interpretation it allows one to investigate the algorithms in the presence of stochastic operators like expectation, conditional expectation or indicator functions.We then specify the algorithms to efficiently incorporate non-pathwise operators (like conditional expectation operators). Under a comparably mild assumption it is possible to retain the simplicity of the backward automatic differentiation algorithm in the presence of conditional expectation operators. This simplifies important applications like - in mathematical finance - the application of backward automatic differentiation to the valuation of Bermudan options or calculation of xVA's.We give the proof for a generalized version of the result. We then discuss in detail how the framework allows dramatic reduction of the memory requirements and improves the performance of a tapeless implementation of automatic differentiation (while the implementation brings advantages similar to ‘vector AAD’ (sometimes called tape compression) for free, it allows improvements beyond this. We present the implementation aspects and show how concepts from object-functional programing, like immutable objects and lazy evaluation enable additional reductions of the memory requirements.

Fully discrete finite element data assimilation method for the heat equation

We consider a finite element discretization for the reconstruction of the final state of the heat equation, when the initial data is unknown, but additional data is given in a sub domain in the space time. For the discretization in space we consider standard continuous affine finite element approximation, and the time derivative is discretized using a backward differentiation. We regularize the discrete system by adding a penalty on theH2-semi-norm of the initial data, scaled with the mesh-parameter. The analysis of the method uses techniques developed in E. Burman and L. Oksanen [Numer. Math.139(2018) 505–528], combining discrete stability of the numerical method with sharp Carleman estimates for the physical problem, to derive optimal error estimates for the approximate solution. For the natural space time energy norm, away fromt= 0, the convergence is the same as for the classical problem with known initial data, but contrary to the classical case, we do not obtain faster convergence for theL2-norm at the final time.

On compact vector formats in the solution of the chemical master equation with backward differentiation

SummaryA stochastic chemical system with multiple types of molecules interacting through reaction channels can be modeled as a continuous‐time Markov chain with a countably infinite multidimensional state space. Starting from an initial probability distribution, the time evolution of the probability distribution associated with this continuous‐time Markov chain is described by a system of ordinary differential equations, known as the chemical master equation (CME). This paper shows how one can solve the CME using backward differentiation. In doing this, a novel approach to truncate the state space at each time step using a prediction vector is proposed. The infinitesimal generator matrix associated with the truncated state space is represented compactly, and exactly, using a sum of Kronecker products of matrices associated with molecules. This exact representation is already compact and does not require a low‐rank approximation in the hierarchical Tucker decomposition (HTD) format. During transient analysis, compact solution vectors in HTD format are employed with the exact, compact, and truncated generated matrices in Kronecker form, and the linear systems are solved with the Jacobi method using fixed or adaptive rank control strategies on the compact vectors. Results of simulation on benchmark models are compared with those of the proposed solver and another version, which works with compact vectors and highly accurate low‐rank approximations of the truncated generator matrices in quantized tensor train format and solves the linear systems with the density matrix renormalization group method. Results indicate that there is a reason to solve the CME numerically, and adaptive rank control strategies on compact vectors in HTD format improve time and memory requirements significantly.

PV and XVA Greeks for Callable Exotics by Algorithmic Differentiation

We generalize the algorithmic differentiation method proposed by Antonov (2016) from price Greeks to XVA Greeks. This method, named Backward Differentiation (BD), was developed in the context of computing price or PV Greeks for individual callable exotic trades.We start by treating cases where cashflow derivatives are sufficient for computing PV/XVA Greeks, i.e., where the differentiation of conditional expectations (or regression functions) is not necessary. For example, PV Greeks for Bermudan swaptions can be computed without having to perform the complicated step of regression function differentiation. We modify the Backward Differentiation algorithm to calculate Greeks for such instruments: the method is applied during the backward pricing procedure and has almost no overhead with respect to a pure backward pricing (without the Greeks).A general XVA calculation cannot be done using only the cashflow derivatives - some exceptions are listed in this article - instead, the differentiation of future instrument values that are results of the regression may be required. We leverage the algorithmic calculation of future values (Algorithmic Exposures) and describe the Adjoint Differentiation (AD) and the new BD for XVA Greeks. The latter algorithm is much simpler than the former, in particular, it does require the use of the instrument tape, i.e., it does not require the storage of certain payoff derivatives during the pricing procedure as is the case for AD. At the same time, both AD and BD enjoy a similar level of performance.

Algorithmic Differentiation for Callable Exotics

In this article, we study the algorithmic calculation of present values greeks for callable exotic instruments. The speed of greeks evaluations becomes important with recent initial margin rules, including the ISDA standard model SIMM, requiring sensitivity calculations for non-cleared deals (e.g. callable exotic/structured products) for a large set of risk factors and on a daily basis.In general, present values of callable trades are obtained by the backward repetitive application of conditional expectation via least-squares Monte Carlo (regression) and arithmetic operations encoded in so called pricing (payoff) script. By nature, this pricing procedure is known to be computationally intensive leading to very slow greeks, if computed by a simple bump-and-reprice. The fast alternative is the adjoint differentiation (AD) method. However, a direct application of the AD is not straightforward because regressions introduce path interdependency, whereas traditional AD is applied on a path-by-path basis. We describe how to extend the traditional AD method to include the regressions. Usually, the procedure must include the recording of information in a data structure, often referred to as the instrument tape, during the backward pricing. This is followed by a subsequent playback of the tape in which the recorded information is used to compute the greeks. The tape adds a significant complexity to the AD implementation and can introduce memory issues (due to the significant storage demands), coding and debugging difficulties, etc.Next, we propose a new tapeless differentiation method, called backward differentiation (BD), for exotic deals. It is applied during the backward pricing procedure following the payoff script. Importantly, it completely avoids the tape and all of its related complications.We demonstrate the efficiency of the BD on the example of a Bermudan swaption calculated with a one-factor Hull-White model. For 50-100 greeks, the BD calculation time is only 2-6 times slower than a single pricing which leads to up 20 times acceleration with respect to the bump-and-reprice. We also give a numerical convergence analysis demonstrating the high quality of the BD greeks.

Solution of the incompressible Navier–Stokes equations by the method of lines

SummaryThe numerical method of lines (NUMOL) is a numerical technique used to solve efficiently partial differential equations. In this paper, the NUMOL is applied to the solution of the two‐dimensional unsteady Navier–Stokes equations for incompressible laminar flows in Cartesian coordinates. The Navier–Stokes equations are first discretized (in space) on a staggered grid as in the Marker and Cell scheme. The discretized Navier–Stokes equations form an index 2 system of differential algebraic equations, which are afterwards reduced to a system of ordinary differential equations (ODEs), using the discretized form of the continuity equation. The pressure field is computed solving a discrete pressure Poisson equation. Finally, the resulting ODEs are solved using the backward differentiation formulas.The proposed method is illustrated with Dirichlet boundary conditions through applications to the driven cavity flow and to the backward facing step flow. Copyright © 2015 John Wiley & Sons, Ltd.

A backward automatic differentiation framework for reservoir simulation

In numerical reservoir simulations, Newton’s method is a concise, robust and, perhaps the most commonly used method to solve nonlinear partial differential equations (PDEs). However, as reservoir simulators incorporate more and more physical and chemical phenomena, writing codes that compute gradients for reservoir simulation equations can become quite complicated. This paper presents an automatic differentiation (AD) framework that is specially designed for simplifying coding and simultaneously maintaining computational efficiency. First a parse tree for a mathematical expression is built and evaluated with the backward mode AD, and then the derivatives with respect to the expression’s arguments are transformed to derivatives with respect to the PDE’s independent variables. The first stage can be realized either by runtime polymorphism to gain higher flexibility or by compile-time polymorphism to gain faster execution speed; the second stage is realized by linear combinations of sparse vectors, which can be accelerated by recording the target column indices. The AD framework has been implemented in an in-house reservoir simulator. Individual tests on some complex mathematical expressions were carried out to compare the speed of the manual implementation, the runtime polymorphic implementation and the compile-time polymorphic implementation of the differentiation. Then the performance of the three was analyzed in complete simulations. These cases indicate that the proposed approach has good efficiency and is applicable to reservoir simulations.

A comparison of high-order time integrators for thermal convection in rotating spherical shells

A numerical study of several time integration methods for solving the three-dimensional Boussinesq thermal convection equations in rotating spherical shells is presented. Implicit and semi-implicit time integration techniques based on backward differentiation and extrapolation formulae are considered. The use of Krylov techniques allows the implicit treatment of the Coriolis term with low storage requirements. The codes are validated with a known benchmark, and their efficiency is studied. The results show that the use of high-order methods, especially those with time step and order control, increase the efficiency of the time integration, and allows to obtain more accurate solutions.

The additional dynamics of least squares completions for linear differential algebraic equations

Several approaches have been proposed for numerically solving lower dimensional, nonlinear, higher index differential algebraic equations (DAEs) for which more classical numerical methods such as backward differentiation or implicit Runge–Kutta may not be appropriate. One of these approaches is called explicit integration (EI). This approach is based on solving nonlinear DAE derivative arrays using nonlinear singular least squares methods. This results in a computed ODE, called the least squares completion, whose solutions contain those of the original DAE. This ODE is then integrated by a classical numerical method. The additional dynamics of the least squares completion can affect the numerical solution of the DAE. This paper begins the study of determining these extra dynamics.

A simplified implicit Newmark integration scheme for finite rotations

Implicit Newmark schemes for integration of finite rotations in structural and continuum mechanics typically are more complicated than those used for translational motion. Using Euler’s representation of the rotation tensor Λ in terms of a rotation vector ϕ , numerical integration connecting the values { ϕ n , Λ n } and { ϕ n + 1 , Λ n + 1 } at the beginning and end of the time step is expressed in terms of an incremental rotation vector θ and the associated incremental rotation tensor Λ ( θ ) . Here, it is shown using backward differentiation that neglecting third order terms in θ , the approximation for the angular velocity ω in terms of θ and its time derivative has the same form as that between velocity and displacement. Consequently, the simplified Newmark scheme in terms of { θ , ω , ω ̇ } has the same form as that for updating translations. Details of the Newmark scheme and an analytical expression for the tangent stiffness tensor for the associated Newton–Raphson iteration procedure have been presented for rigid body dynamics. The resulting integration scheme has been tested on a nontrivial problem of three-dimensional motion of a rigid body using a constant time step. The results justify the use of the simplified Newmark scheme for finite rotations.

A trust-region method applied to parameter identification of a simple prey–predator model

In this paper, the calibration of the non linear Lotka–Volterra model is used to compare the robustness and efficiency (CPU time) of different optimisation algorithms. Five versions of a quasi-Newton trust-region algorithm are developed and compared with a widely used quasi-Newton method. The trust-region algorithms is more robust and three of them are numerically cheaper than the more usual line search approach. Computation of the first derivatives of the objective function is cheaper with the backward differentiation (or adjoint model) technique than with the forward method as soon as the number of parameter is greater than a few ones. In the optimisation problem, the additional information about the Jacobian matrix made available by the forward method reduces the number of iterations but does not compensate for the increased numerical costs. A quasi-Newton trust-region algorithm with backward differentiation and BFGS update after both successful and unsuccessful iterations represents a robust and efficient algorithm that can be used to calibrate very demanding dynamic models.

Likelihood-Based Analysis of Linear State-Space Models Using the Cholesky Decomposition

Data associated with the linear state-space model can be assembled as a matrix whose Cholesky decomposition leads directly to a likelihood evaluation. It is possible to build several matrices for which this is true. Although the chosen matrix or assemblage can be very large, rows and columns can usually be rearranged so that sparse matrix factorization is feasible and provides an alternative to the Kalman filter. Moreover, technology for calculating derivatives of the log-likelihood using backward differentiation is available, and hence it is possible to maximize the likelihood using the Newton–Raphson approach. Emphasis is given to the estimation of dispersion parameters by both maximum likelihood and restricted maximum likelihood, and an illustration is provided for an ARMA(1,1) model.

Staggered Time Integrators for Wave Equations

We consider variations of the Adams--Bashforth, backward differentiation, and Runge--Kutta families of time integrators to solve systems of linear wave equations on uniform, time-staggered grids. These methods are found to have smaller local truncation errors and to allow larger stable time steps than traditional nonstaggered versions of equivalent orders. We investigate the accuracy and stability of these methods analytically, experimentally, and through the use of a novel root portrait technique.

Application of modern time integrators to Hart's inelastic model

Abstract In this paper we consider time integration of unified inelastic models. We apply two recently proposed integration schemes. One fully implicit integrator is based on backward differentiation. The other one is a combined explicit/semi-implicit Runge–Kutta algorithm of Rosenbrock-type. As an application we consider Hart's inelastic constitutive model in its uniaxial formulation. We simulate constant strain rate tension tests and relaxation tests and draw conclusions.

Role of kinetics in laser processing

A computational model has been developed in order to simulate any laser-induced surface modification process with or without the kinetics condition. Front-fixing method with Landau transformation has been used to represent the geometric domain. Spectral collocation method has been utilized for spatial discretization in this model, while backwards differentiation has been used for temporal discretization. The computational model has been applied to the problem of laser-induced melting of an iron sheet with a medium power laser. In order to understand the importance of kinetics of phase transformation, the problem has been solved with and without the kinetics condition at the interface, and the results have been compared. It is observed from the test case that if kinetics condition is not used, the interface location is over-predicted and the free surface temperature is under-predicted. Both of these factors may have important consequences in several material processing techniques such as laser machining, laser annealing and pulsed laser ablation.