Designing Overlay Networks for Handling Exhaust Data in a Distributed Topic-based Pub/Sub Architecture

Abstract

To provide event-driven services in IoT, scalable methods of topic-based pub/sub messaging are indispensable. Methods using structured overlay networks are promising candidates. However, existing methods have the problem of wasting network resources, because they lack adaptivity to “exhaust data, ” which have low or no value most of the time. The problem contains two aspects. One is that each publisher node continues to forward data to a relay node even if there are no subscribers. The other is that excessively large multicast trees are constructed for low value data, which will be received by only a small number of subscribers. In this paper, we formulate the desirable design of overlay networks by defining a property called “strong relay-free” as an expansion of relay-free property. The property involves publishers and subscribers composing connected subgraphs to enable detecting the absence of subscribers and autonomously adjusting the tree size. We also propose a practical method satisfying the property by using Skip Graph, and evaluate it through simulation experiments. We confirmed that the proposed method can suspend publishing adaptively, and shorten the path length on multicast trees by more than 75% under an experimental condition with 100,000 nodes.