Toward Budgeted Online Kernel Ridge Regression on Streaming Data

Abstract

“Concept drift” makes learning from streaming data fundamentally different from traditional batch learning. Focusing on the regression task on streaming data, this paper presents an efficient online learning algorithm, i.e., budgeted online kernel ridge regression (BOKRR). It is a budget version kernel ridge regression algorithm coupled with minimum contribution criterion to maintain the budget of the active set. BOKRR employs low-rank correction technology and the Sherman–Morrison–Woodbury formula to update the dynamic KRR model with the computational complexity of only $\mathcal {O}(B^{2})$ with $B$ learning samples (Budget size of an active set). Limited storage burden and efficient computational ability make the proposed BOKRR algorithm an ideal candidate to process streaming data. The experimental results on benchmark and real-world datasets further demonstrate the validity and efficiency of the proposed algorithms.