Transferability prediction among classification and regression tasks using optimal transport

Abstract

Transfer learning is a method for improving generalization performance by training a model for a different task first and then additionally training the pre-learned weights for the target task. However, transferability—the ease with which source task can be effectively transferred to which target task—is often unknown. Existing works proposed methods of measuring the transferability between classification tasks using images and discrete labels, but it cannot be applied to regression tasks. In this work, we investigate transferability among classification and regression tasks, and propose a method for predicting transferability by extending the optimal transport theory. Our transferability prediction model also can be applied to subjective tasks (e.g., aesthetics and memorability), which are usually regression tasks. We show that the appropriate source (pre-training) tasks can be predicted for the chosen target task without conducting actual pre-training and transferring trials. Experimental results demonstrated high prediction accuracy (correlation coefficient of ρ=0.791\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\rho =0.791$$\\end{document}) and a speed improvement of approximately 300 times compared with the above-mentioned greedy approach.