Statistical approach for highest precision numerical differentiation

Abstract

In the case of the numerical differentiation (ND), the numerical evidence shows that the round-off errors have a large random-like component when considered as a function of the size of the discretization parameter. If a derivative is evaluated on a computer many times with different but reasonable discretization parameters, the round-off errors have tendency to average out and one gets results where the related uncertainty is largely suppressed. Applying this approach to a situation where the round-off error dominates over the discretization error (i.e. the discretization parameter is chosen small), one can effectively increase the precision of the ND by several orders of magnitude in the absolute error. For a general differentiable (e.g. non-analytic) black-box function differentiated in the context of a fixed machine epsilon, the presented method is presumably the most precise way of performing the ND nowadays known. The method is of practical use and has a potential to be generalized to other numerical procedures.