MCFsyn: A Multi-Party Set Reconciliation Protocol With the Marked Cuckoo Filter

Lailong Luo,Richard T.B Ma,Yawei Zhao,Deke Guo,Ori Rottenstreich,Xueshan Luo

doi:10.1109/tpds.2021.3074440

Abstract

Multi-party set reconciliation is a key component in distributed and networking systems. It naturally contains two dimensions, i.e., set representation and reconciliation protocol. However, existing sketch data structures are insufficient to satisfy the new needs brought by the multi-party scenario simultaneously, including space-efficiency, mergeability, and completeness. The current reconciliation protocols, on the other hand, fail to achieve the global optimization of communication cost. To this end, in this article, we propose the marked cuckoo filter (MCF), a data structure for representing set members. Grounded on MCF, we implement the MCFsyn protocol to reconcile multiple sets. MCFsyn aggregates and distributes sets information represented by MCFs along with an underlying minimum spanning tree among the participants. The participants then identify the different elements by traversing the overall MCF which contains the information of all elements in the union set. For the identified missing elements, MCFsyn helps the participants to choose the optimal senders to fetch with the minimum communication cost. Comprehensive evaluations indicate that MCFsyn significantly outperforms existing alternatives in terms of both reconciliation accuracy and communication cost.

Highlights

AS USERS migrate their computation and data to the cloud, cloud-based services such as Dropbox, Google Drive, and OneDrive have emerged to enable users to access data from various devices and allow team collaborations over the same data
The metrics include the number of false positives, the number of false negatives, and the communication cost of reconciliation
We present the MCFsyn protocol for multiparty set reconciliation in distributed scenarios

Summary

Introduction

AS USERS migrate their computation and data to the cloud, cloud-based services such as Dropbox, Google Drive, and OneDrive have emerged to enable users to access data from various devices and allow team collaborations over the same data. The target is to identify and thereafter exchange the different elements among these sets, so that after reconciling S10 1⁄4 S20 1⁄4 Á Á Á 1⁄4 Sn0 1⁄4 S 1⁄4 [i Si. One previous approach for two-party set reconciliation uses characteristic polynomials [11], [12] coupled with coding theories such as Reed-Solomon codes, BHC codes, etc. One previous approach for two-party set reconciliation uses characteristic polynomials [11], [12] coupled with coding theories such as Reed-Solomon codes, BHC codes, etc This kind of methods treat each element as an integer value.

Results

Discussion

Conclusion