Sparse Circulant Binary Embedding: An Asymptotic Analysis

Abstract

Binary embedding refers to methods for embedding points in $\mathbb {R}^{d}$ into vertices in the Hamming cube of dimension ${\mathcal{O}}(d)$ , such that the normalized Hamming distance between the codes preserves a prespecified distance between vectors in the original space. A common approach to binary embedding is to use random projection , followed by one-bit quantization to produce binary codes. Of particular interest, in this letter, is sparse circulant binary embedding (SCBE), where a sparse random circulant matrix with random sampling at a rate of $1-s$ for $s\in (0, 1)$ is used for random projection. The SCBE has the space complexity ${\mathcal{O}}((1-s)d)$ , while unstructured random projection has the space complexity ${\mathcal{O}}(d^2)$ . We present an asymptotic analysis of SCBE, when $d$ approaches $\infty$ , showing that the performance of SCBE is comparable to that of binary embedding with unstructured random projection whilethe former has the space complexity ${\mathcal{O}}((1-s)d)$ and the time complexity ${\mathcal{O}}(d\log d)$ but the latter has both space and time complexities ${\mathcal{O}}(d^2)$ .