A novel multi-innovation gradient support vector machine regression method

Abstract

For the regression problem of support vector machine, the solution processes of the most existing methods use offline datasets, which cannot be realized online. For this problem, this paper presents a new online approach to identify these unknown parameters contained in the support vector machine. A new cost function is constructed by substituting the error term into the standard cost function, which is different from the standard support vector machine, and the gradient descent approach is then used to minimize the newly created loss function, thus proposing a stochastic gradient support vector machine algorithm to estimate the unknown parameters based on the recursive identification methods. Furthermore, to advance the property of the stochastic gradient support vector machine algorithm, a moving data window is used to widen the scalar information into a fixed-length innovation vector, thereby increasing the amount of information used in the parameter estimation based on the multi-innovation identification theory. In addition, the forgetting factor is brought into the proposed algorithms, and the corresponding forgetting factor recursive algorithms are derived. These methods are recursive identification methods, which may be implemented online and are more efficient in terms of computing. Finally, utilizing the MatLab platform, the validity and usefulness of the explored methodologies are proven using several numerical simulation examples.