D-Ary Cuckoo Filter: A Space Efficient Data Structure for Set Membership Lookup

Abstract

Many networking and distributed systems use Bloom filters and their variants for high speed set membership tests. Such probabilistic techniques provide very good space efficiency at the cost of a small fraction of false positive answers. The original Bloom filters do not permit deletion of items from the set and most attempts to extend Bloom filters to support deletions suffer from either space or time performance degradation. Recently, inspired by Cuckoo Hashing, Fan et. al. proposed a data structure called Cuckoo filter that achieves even better space performance than Bloom filters while supporting dynamic insertion and deletion of items. By allowing that each element has more than two candidate buckets, d-ary Cuckoo Hashing is capable of providing much higher space utilization. Motivated by this study, we generalize Cuckoo filter to d-ary Cuckoo filter for further reduction in space cost. The main difficulty here is that increasing the number of candidate buckets is not as easy as it appears because only fingerprints are available for the calculation of candidate locations. To solve this problem, we introduce the base-d digitwise xor operations as the foundation for computing the d candidate buckets of each element in a cyclic fashion. Theoretical analysis and experiment study show that d-ary Cuckoo filters can save up to one bit for each element at the cost of increased lookup and insertion performance.