Deep Neural Network Boosted Large Scale Image Recognition Using User Click Data

Abstract

Recognition by traditional visual features will result in a semantic gap. To bridge this gap, the user click data is employed recently. With this kind of data, images can be represented by the click feature vectors indicating their clicked queries. However, the large-scale and noisy queries from search engines bring low efficiency and accuracy. To enhance its performance, we propose to merge the queries with equivalent semantics and recognize images by the low-dimensional click features with merged queries. Besides, to further enhance its discriminative ability, we combine the click feature with deep CNN feature for image representation. The experimental results for image recognition on Clickture-Dog dataset show that, compared to a common visual feature, the user click feature is more powerful to characterize the image contents and can beat the state-of-the-art CNN feature. Also, the click feature computed on the merged queries will perform much better than CNN feature.