GenSync: A New Framework for Benchmarking and Optimizing Reconciliation of Data

Abstract

In the set reconciliation problem, remote parties seek to reconcile similar sets of data according to an efficiency objective, such as minimizing communication or computation. Though investigated for many individual distributed applications, this problem still lacks a holistic treatment, and this is the aim of this work. Specifically, we design and analyze GenSync, a unified set reconciliation framework that incorporates several state-of-the-art set reconciliation protocols with an integrated testbed. We compare and analyze the various protocols and offer general guidelines for selecting a good protocol for a given application. Through extensive experiments, we demonstrate that the optimal choice of protocol is highly sensitive to several parameters, including network properties (e.g., bandwidth and latency) and computing power. Notably, none of our framework’s protocols are universally dominant under diverse conditions, and a poor protocol choice may lead to a 5x hit in performance. To demonstrate our framework, we measure the effects of protocol choice in reconciling memory pools of adjacent Bitcoin nodes.