Sparsity in function and derivative approximation via the empirical feature space

Abstract

Several practical applications require estimation of the values of a function and its derivative at specific sample locations.This is a challenging task particularly when the explicit forms of the function and its derivative are not known. There have been a few methods proposed in the literature to learn an approximant that simultaneously uses values of a function as well as values of its derivatives or partial derivatives. These methods typically use Support Vector Regression (SVR) and solve a Quadratic Programming Problem (QPP) for the task, which results in a learning model that can estimate the function and derivative values. In this paper, we propose an alternative novel approach that focuses on introducing sparsity in such a learning model, that is based on minimizing the model complexity in the Empirical Feature Space (EFS). Sparsity in such a model is useful when it needs to be evaluated a large number of times as it entails lower computational cost compared to a dense model. The proposed approach, called the EFSRD (EFS Regression for Function and Derivative approximation), involves solving a Linear Programming Problem (LPP). On a number of benchmark examples, EFSRD learns models that offer comparable or better performance, while learning models that are nearly a fourth the size of those obtained by existing approaches.