A New Mobile Visual Search System Based on the Human Visual System

Abstract

We describe a novel mobile visual search system based on the saliencymechanism and sparse coding principle of the human visual system (HVS). In the featureextraction step, we first divide an image into different regions using thesaliency extraction algorithm. Then scale-invariant feature transform (SIFT)descriptors in all regions are extracted while regional identities arepreserved based on their various saliency levels. According to the sparsecoding principle in the HVS, we adopt a local neighbor preserving Hash functionto establish the binary sparse expression of the SIFT features. In the searchingstep, the nearest neighbors matched to the hashing codes are processed accordingto different saliency levels. Matching scores of images in the database arederived from the matching of hashing codes. Subsequently, the matching scoresof all levels are weighed by degrees of saliency to obtain the initial set of results. In order to further ensure matching accuracy, we propose an optimized retrieval scheme based on global texture information. We conduct extensive experiments on an actual mobile platform in large-scale datasets by using Corel-1000. The resultsshow that the proposed method outperforms the state-of-the-art algorithms on accuracyrate, and no significant increase in the running time of the feature extractionand retrieval can be observed.