Automatic Differentiation Technique Research Articles

Applications of automatic differentiation to the time‐dependent Schrödinger equation

AbstractA new approach based upon the Taylor series method is proposed for propagating solutions of the time‐dependent Schrödinger equation. Replacing the spatial derivative of the wave function with finite difference formulas, we derive a recursive formula for the evaluation of Taylor coefficients. The automatic differentiation technique is used to recursively calculate the required Taylor coefficients. We also develop an implicit scheme for the recursive evaluation of these coefficients. We then advance the solution in time using a Taylor series expansion. Excellent computational results are obtained when this method is applied to a one‐dimensional reflectionless potential and a two‐dimensional barrier transmission problem. © 2010 Wiley Periodicals, Inc. Int J Quantum Chem, 2010

Trajectory approach to quantum wave packet dynamics: The correlated derivative propagation method

The correlated derivative propagation method is proposed for the time evolution of quantum wave packets. Without truncation of higher derivatives of hydrodynamic fields, the exact equations of motion derived in the derivative propagation method are solved through the Taylor series method by propagating correlated quantum trajectories. By mapping the physical grid to a uniform computational grid, the necessary higher derivatives of these fields are computed by the finite difference method, and the Jacobian terms of the transformation are determined by the moving least squares method. The Taylor coefficients for hydrodynamic fields and their derivatives are evaluated exactly using the automatic differentiation technique. Excellent results are obtained for barrier transmission problems.

Sensitivity analysis of dynamic biological systems with time-delays

BackgroundMathematical modeling has been applied to the study and analysis of complex biological systems for a long time. Some processes in biological systems, such as the gene expression and feedback control in signal transduction networks, involve a time delay. These systems are represented as delay differential equation (DDE) models. Numerical sensitivity analysis of a DDE model by the direct method requires the solutions of model and sensitivity equations with time-delays. The major effort is the computation of Jacobian matrix when computing the solution of sensitivity equations. The computation of partial derivatives of complex equations either by the analytic method or by symbolic manipulation is time consuming, inconvenient, and prone to introduce human errors. To address this problem, an automatic approach to obtain the derivatives of complex functions efficiently and accurately is necessary.ResultsWe have proposed an efficient algorithm with an adaptive step size control to compute the solution and dynamic sensitivities of biological systems described by ordinal differential equations (ODEs). The adaptive direct-decoupled algorithm is extended to solve the solution and dynamic sensitivities of time-delay systems describing by DDEs. To save the human effort and avoid the human errors in the computation of partial derivatives, an automatic differentiation technique is embedded in the extended algorithm to evaluate the Jacobian matrix. The extended algorithm is implemented and applied to two realistic models with time-delays: the cardiovascular control system and the TNF-α signal transduction network. The results show that the extended algorithm is a good tool for dynamic sensitivity analysis on DDE models with less user intervention.ConclusionsBy comparing with direct-coupled methods in theory, the extended algorithm is efficient, accurate, and easy to use for end users without programming background to do dynamic sensitivity analysis on complex biological systems with time-delays.

Efficient reliability estimate of passive thermal hydraulic safety system with automatic differentiation

An approach for efficient estimation of passive safety system functional reliability has been developed and applied to a simplified model of the passive residual heat transport system typical of sodium cooled fast reactors to demonstrate the reduction in computational time. The method is based on generating linear approximations to the best estimate computer code, using the technique of automatic reverse differentiation. This technique enables determination of linear approximation to the code in a few runs independent of the number of input variables for each response variable. The likely error due to linear approximation is reduced by augmented sampling through best estimate code in the neighborhood of the linear failure surface but in a sub domain where linear approximation error is relatively more. The efficiency of this new approach is compared with importance sampling MCS which uses the linear approximation near the failure region and with Direct Monte-Carlo Simulation. In the importance sampling MCS, variants employing random sampling with Box–Muller algorithm and Markov Chain algorithm are inter-compared. The significance of the results with respect to system reliability is also discussed.

An Efficient Implementation of Automatic Differentiation in Interior Point Optimal Power Flow

This paper presents an improved implementation of automatic differentiation (AD) technique in rectangular interior point optimal power flow (OPF). Distinguished from the existing implementation of AD, the proposed implementation adds a subroutine to identify all constant first-order and second-order derivates by AD and form a list of constant derivates before the processing of iterations. At every iteration of interior point OPF algorithm, only the changing derivates are updated by AD tool. An excellent AD software-ADC-is used as a basic AD tool to finish the proposed implementation. A user-defined model interface is provided with AD technique to enhance performance and flexibility. Numerical studies on several large-scale power systems indicate that the proposed implementation of AD can compete with hand code in execution speed without loss of maintainability and flexibility of AD codes. This paper demonstrates that AD technique has an application potential in online operating environments of power systems instead of hand-coded derivates, and greatly relieves the burdens of software developers.

Automatic Differentiation based QMOM for Population Balance Equations

The quadrature method of moments (QMOM) has emerged as a promising tool for the solution of population balance equations in the past few years. The QMOM requires solving differential algebraic equations (DAEs) consisting of ordinary differential equations related to the evolution of moments as well as nonlinear algebraic equations resulting from the quadrature approximation of moments. In this paper, the use of automatic differentiation (AD) technique is proposed for solution of DAEs arising in QMOM. In the proposed method, the variables of interest are approximated using high-order Taylor series. Using AD, the high-order Taylor coefficients can be recursively calculated to obtain high-fidelity solution of the DAE system. Benchmark examples drawn from literature are used to demonstrate the superior accuracy and computational advantage of the AD-QMOM over existing state-of-the-art techniques, such as DAE-QMOM.

Automation of primal and sensitivity analysis of transient coupled problems

The paper describes a hybrid symbolic–numeric approach to automation of primal and sensitivity analysis of computational models formulated and solved by finite element method. The necessary apparatus for the automation of steady-state, steady-state coupled, transient and transient coupled problems is introduced as combination of a symbolic system, an automatic differentiation (AD) technique and an automatic code generation. For this purpose the paper extends the classical formulation of AD by additional operators necessary for a high abstract description of primal and sensitivity analysis of the typical computational models. An appropriate abstract description for the fully implicit primal and sensitivity analysis of hyperelastic and elasto-plastic problems and a symbolic input for the generation of necessary user subroutines for the two-dimensional, hyperelastic finite element are presented at the end.

Différentiation automatique et formes de Taylor en analyse statique de programmes numériques

Recent work on static analysis of numerical programs has shown that abstract interpretation is well-suited to the validation of the precision of floating-point computations. However, the non relational, interval based techniques used in this context yields some imprecise results due to the well-known wrapping effect. These over-approximations can be reduced by means of relational numerical domains which introduce some constraints between the terms of a computation. In this article, we use automatic differentiation techniques in order to define a new static analysis which relates the floating-point numbers to their error terms. This relation improves the precision of the analysis.

Control of biped walking robot for human living environment

AbstractThis paper introduces biped robot adaptation to human living environment from viewpoints of battery operation time extension and environmental recognition. These issues are important when robots actually work at home. First, in order to extend battery operation time, we propose energy‐saving bipedal locomotion gait. The problem is formulated as an optimal control problem, which is conventionally hard to solve when a target system is complicated. In this paper, partial derivatives appeared in optimal control problem are implicitly represented by using automatic differentiation technique. This approach enables complicated optimal control problem solvable. In combination with receding horizon control, its computation cost is also reduced. Second, we introduce the biped walk tracking based on the camera image mounted on the walking robot, and the visual servoing by the posture change for the purpose of the target image tracking in the camera frame. We propose a new control law to track the rotated target object using the characteristic of the walking, which considered the interference between translational motion and rotational motion. The decoupling is realized by simulations and experiments. As a result, the walking robot tracked the translated and rotated target object without a practical issue. Copyright © 2009 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Automatic Assessment of Uncertainties in the Case of Urban Flood Modeling

Urban flood modeling has long been attracting attention from hydraulic modelers. Due to the complicated floodplain topography, inaccurate boundary conditions, etc, uncertainties will arise from urban flood simulations and should be addressed properly. The present study deals with a novel contribution to the field of urban flood modeling with the aim to quantify uncertainties by examining the response of a finite volume two-dimensional hydraulic model, RUBAR20, used for simulating urban flooding at a crossroad. Two methods have been launched to assess uncertainties and the confident range of the solutions. The first method is the well-known Divided Differences (DD) approximation. The second method is a recently developed technique, based on the computation of derivatives using Automatic Differentiation techniques (AD). For the second method, Tapenade software has been used to calculate derivatives of the water depth with respect to the boundary conditions and roughness coefficients. Benchmark tests were carried out against well-known flows using a high quality and high spatial resolution data set collected from laboratory open channels. Both methods provided good solutions for a reasonable set of parameters. Moreover, the AD adjoint model is found to be useful in the case of large number of input parameters.

Determination of functional gradient in an optimal control problem related to metal solidification

The gradient of the cost functional in the discrete optimal control problem of metal solidification in casting is exactly evaluated. The mathematical model describing the solidification process is based on a three-dimensional two-phase initial-boundary value problem of the Stefan type. Formulas determining exact gradient determination are derived using the fast automatic differentiation technique.

Multivariate Visual Explanation for High Dimensional Datasets.

Understanding multivariate relationships is an important task in multivariate data analysis. Unfortunately, existing multivariate visualization systems lose effectiveness when analyzing relationships among variables that span more than a few dimensions. We present a novel multivariate visual explanation approach that helps users interactively discover multivariate relationships among a large number of dimensions by integrating automatic numerical differentiation techniques and multidimensional visualization techniques. The result is an efficient workflow for multivariate analysis model construction, interactive dimension reduction, and multivariate knowledge discovery leveraging both automatic multivariate analysis and interactive multivariate data visual exploration. Case studies and a formal user study with a real dataset illustrate the effectiveness of this approach.

Development of the Adjoint Model of a Canopy Radiative Transfer Model for Sensitivity Study and Inversion of Leaf Area Index

Many canopy reflectance models have been developed in the last decades and used for estimating land surface biogeophysical variables, such as leaf area index (LAI), from satellite observations through optimization procedures. In most studies, the derivative information of the canopy reflectance model has not been used effectively, which limits this approach for regional and global applications. The final solutions are often converged to the local minima. To address these issues, the adjoint model of a canopy radiative transfer model is developed in this study through the automatic differentiation technique. The developed adjoint model is used for sensitivity study, and a combination of the adjoint model with the trust region global optimization method is performed to retrieve LAI from the Enhanced Thematic Mapper Plus (ETM+). This study demonstrates that this method can be reliably used for inverting LAI efficiently and is suitable for global applications.

Nonlinear system identification for predictive control using continuous time recurrent neural networks and automatic differentiation

In this paper, a continuous time recurrent neural network (CTRNN) is developed to be used in nonlinear model predictive control (NMPC) context. The neural network represented in a general nonlinear state-space form is used to predict the future dynamic behavior of the nonlinear process in real time. An efficient training algorithm for the proposed network is developed using automatic differentiation (AD) techniques. By automatically generating Taylor coefficients, the algorithm not only solves the differentiation equations of the network but also produces the sensitivity for the training problem. The same approach is also used to solve the online optimization problem in the predictive controller. The proposed neural network and the nonlinear predictive controller were tested on an evaporation case study. A good model fitting for the nonlinear plant is obtained using the new method. A comparison with other approaches shows that the new algorithm can considerably reduce network training time and improve solution accuracy. The CTRNN trained is used as an internal model in a predictive controller and results in good performance under different operating conditions.

A new method based on remote sensing observations and data assimilation for estimation of evapotranspiration over field crops

Much research has focused on the estimation of evapotranspiration (ET) over field crops for agricultural applications such as irrigation scheduling. In this research, a variational data assimilation scheme is developed based on the weak constraint concept. It assimilates surface skin temperature into a simple land surface model for estimation of ET. An automatic differentiation technique is used to derive the adjoint codes to evaluate the gradient of the cost function. After construction of this assimilation system, application case studies are examined to test its performance over field crops. The results show that ET can be retrieved with highly satisfactory accuracy.

A Weak-Constraint-Based Data Assimilation Scheme for Estimating Surface Turbulent Fluxes

Much attention has been focused on the assimilation of satellite data and products into land surface processes. In this letter, a variational data assimilation scheme is developed based on the weak-constraint concept. It assimilates surface skin temperature into a simple land surface model for the estimation of turbulent fluxes. An automatic differentiation technique is used to derive the adjoint codes to evaluate the gradient of the cost function. After the construction of this assimilation system, numerical experiments are conducted to test its performance with different model errors, and the comparison is also made with the strong constraint scheme. The results show that the land surface turbulent fluxes can be retrieved with highly satisfactory accuracy.

Differential recurrent neural network based predictive control

In this paper an efficient algorithm to train general differential recurrent neural network (DRNN) is developed. The trained network can be directly used in the nonlinear model predictive control (NMPC) context. The neural network is represented in a general nonlinear state-space form and used to predict the future dynamic behavior of the nonlinear process in real time. In the new training algorithms, the ODEs of the model and the dynamic sensitivity are solved simultaneously using Taylor series expansion and automatic differentiation (AD) techniques. The same approach is also used to solve the online optimization problem in the predictive controller. The efficiency and effectiveness of the DRNN training algorithm and the NMPC approach are demonstrated through a two-CSTR case study. A good model fitting for the nonlinear plant at different sampling rates is obtained using the new method. A comparison with other approaches shows that the new algorithm can considerably reduce network training time and improve solution accuracy. The DRNN based NMPC approach results in good control performance under different operating conditions.

Efficient symbolic differentiation for graphics applications

Functions with densely interconnected expression graphs, which arise in computer graphics applications such as dynamics, space-time optimization, and PRT, can be difficult to efficiently differentiate using existing symbolic or automatic differentiation techniques. Our new algorithm, D*, computes efficient symbolic derivatives for these functions by symbolically executing the expression graph at compile time to eliminate common subexpressions and by exploiting the special nature of the graph that represents the derivative of a function. This graph has a sum of products form; the new algorithm computes a factorization of this derivative graph along with an efficient grouping of product terms into subexpressions. For the problems in our test suite D* generates symbolic derivatives which are up to 4.6 x 103 times faster than those computed by the symbolic math program Mathematica and up to 2.2x105 times faster than the non-symbolic automatic differentiation program CppAD. In some cases the D* derivatives rival the best manually derived solutions.

Efficient symbolic differentiation for graphics applications

Functions with densely interconnected expression graphs, which arise in computer graphics applications such as dynamics, space-time optimization, and PRT, can be difficult to efficiently differentiate using existing symbolic or automatic differentiation techniques. Our new algorithm, D* , computes efficient symbolic derivatives for these functions by symbolically executing the expression graph at compile time to eliminate common subexpressions and by exploiting the special nature of the graph that represents the derivative of a function. This graph has a sum of products form; the new algorithm computes a factorization of this derivative graph along with an efficient grouping of product terms into subexpressions. For the problems in our test suite D* generates symbolic derivatives which are up to 4.6 x 10 3 times faster than those computed by the symbolic math program Mathematica and up to 2.2x10 5 times faster than the non-symbolic automatic differentiation program CppAD. In some cases the D* derivatives rival the best manually derived solutions.

Retrieving surface parameters for climate models from Moderate Resolution Imaging Spectroradiometer (MODIS)‐Multiangle Imaging Spectroradiometer (MISR) albedo products

We present a computer‐efficient software package enabling us to assimilate operational remote‐sensing flux products into a state‐of‐the‐art two‐stream radiation transfer scheme suitable for climate models. This package implements the adjoint and Hessian codes, generated using automatic differentiation techniques, of a cost function balancing (1) the deviation from the a priori knowledge on the model parameter values and (2) the misfit between the observed remote‐sensing fluxes and the two‐stream model simulations. The individual weights of these contributions are specified notably via covariance matrices of the uncertainties in the a priori knowledge on the model parameters and the measurements. The proposed procedure delivers a Gaussian approximation of the PDFs of the retrieved model parameter values. The a posteriori covariance matrix is further exploited to evaluate, in turn, the posterior probability density functions of the radiant fluxes simulated by the two‐stream model, including those that are not measured, for example, the fraction of radiation absorbed in the ground. Applications are conducted using Moderate Resolution Imaging Spectroradiometer (MODIS) and Multiangle Imaging Spectroradiometer (MISR) broadband surface albedo products. It turns out that the differences between these two albedo sets may translate into discernible signatures on some retrieved model parameters. Meanwhile, adding the Joint Research Centre (JRC)‐Fraction of Absorbed Photosynthetically Active Radiation (FAPAR) Sea‐viewing Wide Field‐of‐view Sensor (SeaWiFS) products into the measurements yields a significant reduction of uncertainties. Results from these applications indicate that the products retrieved from the two‐stream inversion procedure (1) exhibit much less variability than those generated by the operational algorithms for the LAI and FAPAR, and (2) are in good agreement with the available ground‐based estimates.