Second Derivative Estimates Research Articles

The selection of interpolation points in the numerical approximation of second and third order derivatives

The first or higher derivatives of a function may be estimated numerically by applying Neville's polynomial extrapolation process to a sequence of approximations to the derivative, each consisting of a suitable linear combination of function values. The sequences of evaluation points which minimise the magnification of rounding errors relative to the truncation error for first, second and third order derivatives are determined, and it is shown that by defining the sequences of evaluation points by certain geometric progressions the amount of computation may be reduced without greatly increasing the rounding error magnification.

On the convergence of a class of derivative-free minimization algorithms

A convergence analysis is presented for a general class of derivative-free algorithms for minimizing a functionf(x) for which the analytic form of the gradient and the Hessian is impractical to obtain. The class of algorithms accepts finite-difference approximation to the gradient, with stepsizes chosen in such a way that the length of the stepsize must meet two conditions involving the previous stepsize and the distance from the last estimate of the solution to the current estimate. The algorithms also maintain an approximation to the second-derivative matrix and require that the change inx made at each iteration be subject to a bound that is also revised automatically. The convergence theorems have the features that the starting pointx 1 need not be close to the true solution andf(x) need not be convex. Furthermore, despite the fact that the second-derivative approximation may not converge to the true Hessian at the solution, the rate of convergence is still Q-superlinear. The theorry is also shown to be applicable to a modification of Powell's dog-leg algorithm.

Higher-Order Numerical Solutions Using Cubic Splines

A cubic spline collocation procedure was developed for the numerical solution of partial differential equations. This spline procedure is reformulated so that the accuracy of the second-derivative approximation is improved and parallels that previously obtained for lower derivative terms. The final result is a numerical procedure having overall third-order accuracy of a nonuniform mesh. Solutions using both spline procedures, as well as three-point finite difference methods, are presented for several model problems.

Variable-Metric Algorithm Employing Linear and Quadratic Penalties

A variable-metric algorithm is described which makes use of both linear and quadratic penalty terms for handling nonlinear constraints, and employs both projection and penalty features. Quadratic penalty coefficients are adjusted in a process that attempts to maintain a positive-definite matrix of second partial derivatives of the function, including penalty terms, without generating the large positive eigenvalues that traditionally attend the use of quadratic penalties, which cause zigzagging and slowed convergence. The schemes contemplated make use of inferred second-order properties, not only in terms of the variable metric of DFP (or its relatives), but by estimation of second directional derivatives, by fitting cubics to various functions along directions of search. Some experiments are described with a simple constrained-minimum problem contrived to offer difficulties with methods that use only linear penalties, hence taxing the quadratic-penalty-adjustment procedure.

Note on sequential estimation of second partial derivatives

This note presents a simple formula for estimating the matrix of the second partial derivatives of a function.

Sequential estimation of second partial derivatives

Sequential estimation of second partial derivatives

Simultaneous estimation of first and second derivatives of a cost function

A correlation method for estimating the second derivative, as well as the first derivative, of a cost function which is quadratic in an input parameter is described. The method, using a 3-level msequence perturbation signal, enables the cost-function minimum to be reached in a single step in a noisefree system. The effects of noise, and the extension of the method to multiparameter cost functions, are discussed.