Informative relationship multi-task learning: Exploring pairwise contribution across tasks’ sharing knowledge

Abstract

Multi-task learning is a machine learning paradigm, that aims to leverage useful domain information to help improve the generalization performance of all tasks. Learning the relationships of tasks helps to identify the latent tasks’ associations and access a better performance. However, most of the existing methods hardly pay attention to the determination of knowledge interaction among tasks and instead concentrate solely on certain aspects of task affinity. This compulsory similarity among all tasks leads to deficiencies in both task diversity and model robustness. To address this issue, we emphasize the task relationships within mutual information interaction. We propose a regularized framework from an informative perspective to quantify pairwise contributions during the knowledge-sharing stage, meanwhile utilizing an exclusive Lasso to identify the exclusive characteristics of tasks. An efficient optimization algorithm is developed to solve the proposed objective function. Detailed theoretical analyses and extensive experiments on both synthetic and real-world datasets are provided to demonstrate the effectiveness of our proposed method.