Deep Learning to Hash with Application to Cross-View Nearest Neighbor Search

Abstract

Learning hash functions for approximate nearest neighbor search of high-dimensional data has received a surge of interests in recent years. Most existing methods are often concerned with learning hash functions for nearest neighbor search on high-dimensional data from a single source. In many real-world applications, data can be collected from diverse sources or represented using different feature descriptors. This raises an open challenge, i.e., the Cross-View Nearest Neighbor Search (CVNNS), where the representation of a query instance can be different from that of target instances to be retrieved in database. The key challenge of cross-view search is to learn an effective shared representation which can effectively connect the query instance and the target instances to be retrieved. In this paper, we present a new cross-view nearest neighbor search scheme by applying the emerging deep learning to hash techniques. In particular, we investigate two different architectures of deep Restricted Boltzmann Machines (RBMs) for learning to hash toward cross-view nearest neighbor search, and conduct extensive experiments to examine their empirical performance on diverse settings of cross-view image retrieval tasks. The encouraging results show that our technique outperforms the state-of-the-art approaches.