Large-scale visual search based on voting in reduced pose space with application to mobile search and video collections

Abstract

We present a novel visual search system that deals with scalability, is fast enough for commercial applications, and addresses limitations present in current visual search engines. Most scalable visual search approaches rely on local features, the Bag of Visual Words representation, and a ranking mechanism based on some vector space model [1, 2]. However, since in those methods the initial rankings do not take into account any spatial information, they are not well suited to identify multiple small objects “buried” within complex scenes. To alleviate this limitation we propose to perform the initial ranking using clustering of matches in a limited pose space. We also describe its smooth integration with Soft Assignment of Visual Words and RANSAC-inspired spatial consistency verification. We demonstrate that our system addresses the problem and show the use of the method in several commercially attractive applications.