Hamming Metric Multi-Granularity Locality-Sensitive Bloom Filter

Abstract

A Bloom filter is a type of space-efficient data structure that supports membership tests in numerous network applications. Recently, emerging applications require an approximate membership test (AMT) rather than conventional (exact-matching) membership test. Some AMT problems can be effectively solved by using a locality-sensitive hashing (LSH) based Bloom filter. However, existing work cannot handle changing Hamming distances. In this paper, we present a new Hamming metric locality-sensitive Bloom filter (HLBF) to tackle the challenge. Each object of the data set is hashed by bit sampling LSH functions and encoded into a standard Bloom filter in the HLBF structure. To support AMTs with different given Hamming distances, we propose a multi-granularity test algorithm (called the M-HLBF) based on the HLBF and virtual objects which are created from the given test object. Theoretical analyses show that false positive rates and false negative rates can be controlled within low levels. To further accelerate the processing of an AMT, we also illustrate a hardware implementation. Extensive experimental results demonstrate that our method is quite promising in achieving high efficiency and flexibility for processing AMTs with different granularities/distances.