Abstract
With the breakthrough of edge intelligence, we are witnessing a booming increase in distributed applications on edge nodes. These distributed applications need to apply a novel data representation algorithm to support data-information exchanging and data-information based decision among different edge nodes. As the most efficient data compact representation algorithm, Counting Bloom Filter (CBF) is an extension of Bloom filter, which enables updating data representation as well as inserting data into a representation. To facilitate distributed applications on edge nodes, edge nodes need to exchange and summarize the information of the data collected from different edge nodes. Impossible to merge with other CBFs, the existing CBF and its variants thus cannot be used for representing and exchanging data information among edge nodes. To handle this problem, we design a novel mergeable CBF, mergeCBF. Based on an insight about the counting processing of a CBF, we unfold the counter array of the conventional CBF to a group of bit arrays, and in order to support merging multiple filters, map each inputted item to the cells in this group of cuckoo-scheduled bit arrays instead of the counters in CBF. Experiments on real-world datasets demonstrate that mergeCBF can support conventional operations and merging operations in an efficient way without degrading the quality of the representation results.
Highlights
According to a survey of Cisco, there will be 850 ZB of data generated by mobile users and IoT devices by 2021 [1]
To provide a mergeable counting bloom filter for common high-speed random access memory (RAM), we study the counting processing of a Counting bloom filter (CBF), and obtain an insight that the reason why CBF cannot support merging operations is that CBF does not record any other information of the inputted data items but its hit count on some specific counters in CBF, and CBF cannot identify the repeated data items inserted in the different candidate filters to be merged
Rottenstreich et al A general method [16] based on variable increments, called Variable Increment Counting Bloom Filter (VI-CBF), can always achieve a lower false positive rate and a lower overflow probability bound than CBF in practical systems
Summary
According to a survey of Cisco, there will be 850 ZB of data generated by mobile users and IoT devices by 2021 [1]. According to the configuration method of the basic Bloom filter, including the number of cells (m), the number of hash functions (k), etc [9] In this way, CBF can achieve a compact representation of the inserted data items without involving a large false positive rate (ε). After k cells are set according to k hash results, all ‘‘1’’ bits in different bit arrays of our filter are used as the record of this inserted item. Since the inserting sequence of items has no impact on their compact representation results in the bit arrays of mergeCBF, the ‘‘1’’ bits representing the same items in different mergeCBFs can be switched to the cells at the same positions. This make our mergeCBF can support the merging operation effectively
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
