Adaptive Classification Using Incremental Linearized Kernel Embedding

Abstract

This paper considers the problem of adaptive classification for performing pattern discrimination in varying conditions when new data arrives. A new efficient method is presented to incrementally update features in a Nyström-approximated linearized kernel embedding (LKE). Our method leverages a fast eigendecomposition algorithm for symmetric Arrowhead matrices. The proposed method can also be applied to kernel principal component analysis (KPCA) or similar problems. A mechanism is proposed which allows the incremental linearized kernel embedding to be used for updating of dictionaries in a sparse representation-based classification algorithm. The method is based on transporting the dictionaries into the embedding expanded with new data points and avoids the need to learn new dictionary matrices every time new data becomes available. The effectiveness of the developed methods is illustrated on two handwritten digit image data sets namely MNIST and USPS. Classification performance before and after sequential embedding updates is evaluated and compared. Comparisons are also made between our incremental LKE algorithm and the conventional approach to updating the empirical kernel map in terms of their computational requirements and numerical stability.