Towards efficient event aggregation in a decentralized publish-subscribe system

Abstract

Recently, decentralized publish-subscribe (pub-sub) systems have gained popularity as a scalable asynchronous messaging paradigm over wide-area networks. Most existing pub-sub systems, however, have been designed with the implicit assumption that published data is clean and accurate. As the pub-sub paradigm is incorporated in real-world applications with human participants, this assumption becomes increasingly invalid due to the inherent noise in the event stream. The noise can take many forms, including redundant, incomplete, inaccurate, and even malicious event messages.This paper explores the distributed computing issues involved in handling event streams with redundant and incomplete messages. Given a distributed broker overlay-based pub-sub system, we present our initial ideas for (1) aggregating event information scattered across multiple messages generated by different publishers and (2) eliminating redundant event messages. Key to our approach is the concept of an event-gatherer---a designated broker in the routing graph that acts as a proxy sink for all messages of a particular event---located at the graph center of the corresponding routing tree. This paper proposes a novel decentralized algorithm to find this graph center. Early results show that the proposed scheme typically reduces the message load by over 60% with less than 25% time overhead to subscribers.