Accelerating projections to kernel-induced spaces by feature approximation

Abstract

A method for speeding-up data projections onto kernel-induced feature spaces (derived using e.g. kernel Principal Component Analysis - kPCA) is presented in the paper. The proposed idea is to simplify the derived features, implicitly defined by all training samples and dominant eigenvectors of problem-specific generalized eigenproblems, by appropriate approximations. Instead of employing the whole training set, we propose to use a small pool of its appropriately selected representatives and we formulate a rule for deriving the corresponding weight vectors that replace the considered dominant eigenvectors. The representatives are determined via clustering of training data, whereas weighting coefficients are chosen to minimize original feature approximation errors. The concept has been experimentally verified for kernel-PCA using both artificial and real datasets. It has been shown that the presented approach provides reduction in feature-extraction complexity, which implies a proportional increase in data projection speed, by one-to-two orders of magnitude, without sacrificing data analysis accuracy. Therefore, the proposed approach is well-suited for kernel-based, intelligent data analysis applications that are to be executed on resource-limited systems, such as embedded or IoT devices, or for systems where processing time is critical.