Multi-Task Learning With Localized Generalization Error Model

Abstract

In cases, the same or similar network architecture is used to deal with related but different tasks, where tasks come from different statistical distributions in the sample input space and share some common features. Multi-Task Learning (MTL) combines multiple related tasks for training at the same time, so as to learn some shared feature representation among multiple tasks. However, it is difficult to improve each task when statistical distributions of these related tasks are greatly different. This is caused by the difficulty of extracting an effective generalization of feature representation from multiple tasks. Moreover, it also slows down the convergence rate of MTL. Therefore, we propose a MTL method based on the Localized Generalization Error Model (L-GEM). The L-GEM improves the generalization capability of the trained model by minimizing the upper bound of generalization error of it with respect to unseen samples similar to training samples. It also helps to narrow the gap between different tasks due to different statistical distributions in MTL. Experimental results show that the L-GEM speeds up the training process while significantly improves the final convergence results.