On stable piecewise linearization and generalized algorithmic differentiation

Abstract

It is shown how functions that are defined by evaluation programs involving the absolute value function abs() (besides smooth elementals) can be approximated locally by piecewise-linear models in the style of algorithmic or automatic differentiation (AD). The model can be generated by a minor modification of standard AD tools and it is Lipschitz continuous with respect to the base point at which it is developed. The discrepancy between the original function, which is piecewise differentiable, and the piecewise linear model is of second order in the distance to the base point. Consequently, successive piecewise linearization yields bundle type methods for unconstrained minimization and Newton-type equation solvers. As a third fundamental numerical task we consider the integration of ordinary differential equations, for which we examine generalizations of the midpoint and the trapezoidal rule for the case of Lipschitz continuous right hand sides (RHSs). As a by-product of piecewise linearization, we show how to compute at any base point some generalized Jacobians of the original function, namely those that are conically active as defined by Khan and Barton. This subset of the Clarke Jacobian is never empty, independent of the particular function representation in terms of elementals, and also invariant with respect to linear transformations on domain and range. However, like all generalized derivatives the conically active Jacobians reduce almost everywhere to the singleton formed by the proper Jacobian, which may approximate the original function only in a minuscule neighbourhood. Since the piecewise linearization always reflects kinks in the vicinity, we illustrate how it can be used to approximate generalized Jacobians at nearby points along a user specified preferred direction.