Online kernel dictionary learning on a budget

Abstract

Online kernel-based dictionary learning (DL) algorithms are considered, which perform DL on training data lifted to a high-dimensional feature space via a nonlinear mapping. Compared to batch versions, online algorithms require low computational complexity, essential for processing the Big Data, based on the stochastic gradient descent method. However, as with any kernel-based learning algorithms, the number of parameters needed to represent the desired dictionary is equal to the number of training samples, which incurs prohibitive memory requirement and computational complexity for large-scale datasets. In this work, appropriate sparsification and pruning strategies are combined with online kernel DL to mitigate this issue. Numerical tests verify the efficacy of the proposed strategies.