Automatic Differentiation Using Dual Numbers - Use Case

Abstract

Automatic Differentiation (AD), also called algorithmic differentiation, is a method to obtain derivatives of univariate as well as multivariate functions without the need of using numerical or symbolic differentiation. The accuracy of AD is limited by the word length of the computing platform used for implementation; it is otherwise exact. Recent applications of AD are found in Machine Learning, particularly in Deep Neural Networks (DNNs), Kinematics, Computational Fluid Dynamics, optimization problems, and solutions of differential equations. AD is carried out either in forward or backward methods when implemented with real numbers. However, AD using dual numbers is of current interest. Nonlinear relationships between sensor data of a thermal power plant are modeled using DNNs and it is possible to estimate the sensitivity measures by obtaining the gradients of output w.r.t. input feature vector of DNNs. The sensitivities can be estimated using DNNs which obviates the need for complex first-principle thermodynamic modeling. The sensitivities can be obtained using high-dimensional dual numbers to represent the DNN model parameters. Though dual numbers do not form a perfect ring (field) as per the definition of Group Number Theory, they are effective in simultaneous evaluation of functions and their Jacobians and Hessians. This paper discusses the use of AD with dual numbers to obtain the sensor sensitivity of a typical 500 MW using DNN.