Online regularized pairwise learning with least squares loss

Abstract

In this paper, we study online algorithm for pairwise problems generated from the Tikhonov regularization scheme associated with the least squares loss function and a reproducing kernel Hilbert space (RKHS). This work establishes the convergence for the last iterate of the online pairwise algorithm with the polynomially decaying step sizes and varying regularization parameters. We show that the obtained error rate in [Formula: see text]-norm can be nearly optimal in the minimax sense under some mild conditions. Our analysis is achieved by a sharp estimate for the norms of the learning sequence and the characterization of RKHS using its associated integral operators and probability inequalities for random variables with values in a Hilbert space.