Deep sketch feature for cross-domain image retrieval

Abstract

Deep learning has been proven to be very effective for various image recognition tasks, e.g., image classification, semantic segmentation, image retrieval, shape classification, etc. However, existing works on deep learning for image recognition mainly focus on either natural image data or binary shape data. In this paper, we show that deep convolutional neural networks (DCNN) is also suitable for cross-domain image recognition, i.e., using sketch as query to retrieve natural images in a large dataset. To solve this kind of cross-domain problem, we propose to train CNN jointly using image data and sketch data in a novel way. The learned deep feature is effective for cross-domain image retrieval – using simple Euclidean distance on the learned feature can significantly outperform the previous state-of-the-arts. In addition, we find that pre-training and a feasible data-argumentation for DCNN can largely surpass human-level performance in the standard sketch classification benchmark.