Regularized Deep Transfer Learning: When CNN Meets kNN

Abstract

In this brief, we propose a regularized deep transfer learning architecture composed of a softmax classifier and a k-nearest neighbor (kNN) classifier. We aim to generalize the softmax classifier possibly well under the regularization effect of the kNN classifier. That is, given a training sample, the kNN classifier assigns a soft label vector according to its distance to the center of each class. The working mechanism of the kNN classifier is attributed to both the source and target domains. On the one hand, it gradually becomes stronger by backpropagating the cross-entropy classification loss on the source images. On the other hand, for target data, we enforce to minimize the discrepancy between the label vectors produced by the kNN classifier and the softmax classifier. Using the kNN classifier, we are able to reduce the intra-class variations on the source domain and meanwhile pull close the source and target feature distributions, which can better bound the expected error of target domain. In experiment, we demonstrate that our method compares favorably with the state-of-the-arts on benchmarks.