Learning performance of Tikhonov regularization algorithm with geometrically beta-mixing observations

Abstract

Estimating the generalization performance of learning algorithms is one of the main purposes of machine learning theoretical research. The previous results describing the generalization ability of Tikhonov regularization algorithm are almost all based on independent and identically distributed (i.i.d.) samples. In this paper we go far beyond this classical framework by establishing the bound on the generalization ability of Tikhonov regularization algorithm with geometrically beta-mixing observations. We first establish two refined probability inequalities for geometrically beta-mixing sequences, and then we obtain the generalization bounds of Tikhonov regularization algorithm with geometrically beta-mixing observations and show that Tikhonov regularization algorithm with geometrically beta-mixing observations is consistent. These obtained bounds on the learning performance of Tikhonov regularization algorithm with geometrically beta-mixing observations are proved to be suitable to geometrically ergodic Markov chain samples and hidden Markov models.