Modeling Sequential Domain Shift through Estimation of Optimal Sub-spaces for Categorization

Abstract

Domain adaptation (DA) is the process in which labeled training samples available from one domain is used to improve the performance of statistical tasks performed on test samples drawn from a different domain. The domain from which the training samples are obtained is termed as the source domain, and the counterpart consisting of the test samples is termed as the target domain. Few unlabeled training samples are also taken from the target domain in order to approximate its distribution. In this paper, we propose a new method of unsupervised DA, where a set of domain invariant sub-spaces are estimated using the geometrical and statistical properties of the source and target domains. This is a modification of the work done by Gopalan et al. [2], where the geodesic path from the principal components of the source to that of the target is considered in the Grassmann manifold, and the intermediary points are sampled to represent the incremental change in the geometric properties of the data in source and target domains. Instead of the geodesic path, we consider an alternate path of shortest length between the principal components of source and target, with the property that the intermediary sample points on the path form domain invariant sub-spaces using the concept of Maximum Mean Discrepancy (MMD) [3]. Thus we model the change in the geometric properties of data in both the domains sequentially, in a manner such that the distributions of projected data from both the domains always remain similar along the path. The entire formulation is done in the kernel space which makes it more robust to non-linear transformations. Let X and Y be the source and target domains having nX and nY number of instances respectively. If Φ(.) is a universal kernel function, then in kernel space the source and target domains are Φ(X) ∈ RnX×d and Φ(Y ) ∈ RnX×d respectively. Let KXX and KYY be the kernel gram matrices of Φ(X) and Φ(Y ) respectively. Let D = [X ;Y ] denote the combined source and target domain data, and the corresponding data in kernel space is given as Φ(D). The kernel gram matrix formed using D is given by