Nowadays, evolution of mobile devices make demand for searching information increasing expressively. Many applications have been developed for recognition tasks. In this paper, we present a new and efficient visual search system for finding similar images on the large database. We first propose a compact, discriminative image representation called Locality Preserving Vector which can explicitly exploit neighborhood structure of data and attains high retrieval accuracy in the low-dimensional space. We then integrate topic modeling into visual search system for extracting topic related and image-specific information. These information enables images which likely contain the same objects to be ranked with higher similarity. The experiments show that our approach provides competitive accuracy with very low memory cost.