Comparing high-order multivariate AD methods

Abstract

ABSTRACTTo compute every high-order multivariate derivative value, interpolation methods will be shown to be less accurate than a direct forward multivariate Taylor series method, becoming significant for degrees higher than 10. As order increases, interpolation methods rely on increasingly ill-conditioned matrices where simply rounding exact rational values produced corresponding error in some resulting derivative values. Both interpolation and direct methods use forward AD (algorithmic differentiation); the direct method propagates multivariate series coefficients of the original function, while interpolation methods propagate univariate series of the function in fixed directions and reconstruct the multivariate values. Such interpolation methods, differing in direction choices and reconstruction methods, have been shown to be theoretically more efficient than the direct method for high order. Four alternatives were implemented in MATLAB (interpreted and using random access arrays) on the same laptop. In our implementations, the direct method was competitive and often faster in run time, in addition to maintaining good accuracy. Since AD tools in compiled languages are much faster, more comparison is needed. Direct method efficiency depends on indexing subsets within the large non-rectangular data structure for multivariate series coefficients. We explain key implementation details of our direct method that uses a global reference array.