Learning theory estimates for coefficient-based regularized regression

Abstract

We consider a coefficient-based regularized regression in a data dependent hypothesis space. For a given set of samples, functions in this hypothesis space are defined to be linear combinations of basis functions generated by a kernel function and sample data. We do not need the kernel to be symmetric or positive semi-definite, which provides flexibility and adaptivity for the learning algorithm. Another advantage of this algorithm is that, it is computationally effective without any optimization processes. In this paper, we apply concentration techniques with ℓ2-empirical covering numbers to present an elaborate capacity dependent analysis for the algorithm, which yields shaper estimates in both confidence estimation and convergence rate. When the kernel is C∞, under a very mild regularity condition on the regression function, the rate can be arbitrarily close to m−1.