Multi-task learning via linear functional strategy

Abstract

In machine learning, the multi-task learning is a natural approach that exploits the relations among different tasks to improve the performance. We develop a theoretical analysis of multi-penalty least-square regularization scheme on the reproducing kernel Hilbert space in vector-valued setting. The results hold for general framework of vector-valued functions; therefore they can be applied to multi-task learning problems. We study an approach for multi-penalty regularization scheme which is based on the idea of linear combination of different regularized solutions. We estimate the coefficients of the linear combination by means of the so-called linear functional strategy. We discuss a theoretical justification of the linear functional strategy which particularly provides the optimal convergence rates for multi-penalty regularization scheme. Finally, we provide an empirical analysis of the multi-view manifold regularization scheme based on the linear functional strategy for the challenging multi-class image classification and species recognition with attributes.