Semi-Supervised Domain Adaptation by Covariance Matching.

Abstract

Transferring knowledge from a source domain to a target domain by domain adaptation has been an interesting and challenging problem in many machine learning applications. The key problem is how to match the data distributions of the two heterogeneous domains in a proper way such that they can be treated indifferently for learning. We propose a covariance matching approach DACoM for semi-supervised domain adaptation. The DACoM embeds the original samples into a common latent space linearly such that the covariance mismatch of the two mapped distributions is minimized, and the local geometric structure and discriminative information are preserved simultaneously. The KKT conditions of DACoM model are given as a nonlinear eigenvalue equation. We show that the KKT conditions could at least ensure local optimality. An efficient eigen-updating algorithm is then given for solving the nonlinear eigenvalue problem, whose convergence is guaranteed conditionally. To deal with the case when homogeneous information could only be matched nonlinearly, a kernel version of DACoM is further considered. We also analyze the generalization bound for our domain adaptation approaches. Numerical experiments on simulation datasets and real-world applications are given to comprehensively demonstrate the effectiveness and efficiency of the proposed approach. The experiments show that our method outperforms other existing methods for both homogeneous and heterogeneous domain adaptation.