Learning Rates for -Regularized Kernel Classifiers

Hongzhi Tong,Di-Rong Chen,Fenghong Yang

doi:10.1155/2013/496282

Abstract

We consider a family of classification algorithms generated from a regularization kernel scheme associated with -regularizer and convex loss function. Our main purpose is to provide an explicit convergence rate for the excess misclassification error of the produced classifiers. The error decomposition includes approximation error, hypothesis error, and sample error. We apply some novel techniques to estimate the hypothesis error and sample error. Learning rates are eventually derived under some assumptions on the kernel, the input space, the marginal distribution, and the approximation error.

Highlights

Let X be a compact subset of Rn, Y = {−1, 1}
ΡX is the marginal distribution on X and ρ (⋅ | x) is the conditional probability measure at x induced by ρ
[26] studies classification problem with hinge loss Vh and l1 complexity regularization in a finite-dimensional hypothesis space spanned by a set of base functions. While it does not assume a kernel setting nor is it assumed that the expansion must be in terms of the sample points, so the problem of data-dependent hypothesis space is not present there

Summary

Introduction

Regularized learning schemes are implemented by minimizing a penalized version of empirical error over a set of functions, called a hypothesis space. In this paper we will consider a different regularization scheme in RKHS for classification; in our setting, the regularizer is l1-norm of the coefficients in the kernel expansions over the sample points. [26] studies classification problem with hinge loss Vh and l1 complexity regularization in a finite-dimensional hypothesis space spanned by a set of base functions. While it does not assume a kernel setting nor is it assumed that the expansion must be in terms of the sample points, so the problem of data-dependent hypothesis space is not present there. In this paper we will present an elaborate error analysis for algorithm (10), and we use a modified error decomposition technique that was firstly introduced in [28], by dealing with the approximation error, the hypothesis error, and the sample error, and we derive an explicit learning rate for classification scheme (10) under some assumptions

Preliminaries

Estimating the Hypothesis Error

Estimating the Sample Error

Deriving Learning Rates