An adapted surrogate kernel for classification under covariate shift

Abstract

A general solution to covariate shift is to minimize a distributional discrepancy between training and test data. To solve this problem, kernel-based learning methods minimize the discrepancy in a kernel-induced feature space by transforming training data to be similar to test data and learn the transformed data in the feature space. However, when they are applied to classification tasks, transformed training data points from different classes can be mixed up in the kernel-induced space since they ignore the class labels in training data. Therefore, we propose an adapted surrogate kernel that is able to manage the large class discrepancy. The proposed method incorporates the class discrepancy into the surrogate kernel which tries to minimize the discrepancy in a kernel-induced feature space. To do this, we interpret the surrogate kernel with the Nyström method which allows prior knowledge to be incorporated into kernel approximation. By using class discrepancy derived from training data as prior knowledge, the adapted surrogate kernel does not keep only the role of surrogate kernel, but also the large class discrepancy among classes. The experimental results on several classification tasks including text classification and WiFi localization prove that the proposed kernel results in significant improvement in classification performance under covariate shift.