Scalable Overlay Network Research Articles

Low-diameter topic-based pub/sub overlay network construction with minimum-maximum node degree.

In the construction of effective and scalable overlay networks, publish/subscribe (pub/sub) network designers prefer to keep the diameter and maximum node degree of the network low. However, existing algorithms are not capable of simultaneously decreasing the maximum node degree and the network diameter. To address this issue in an overlay network with various topics, we present herein a heuristic algorithm, called the constant-diameter minimum–maximum degree (CD-MAX), which decreases the maximum node degree and maintains the diameter of the overlay network at two as the highest. The proposed algorithm based on the greedy merge algorithm selects the node with the minimum number of neighbors. The output of the CD-MAX algorithm is enhanced by applying a refinement stage through the CD-MAX-Ref algorithm, which further improves the maximum node degrees. The numerical results of the algorithm simulation indicate that the CD-MAX and CD-MAX-Ref algorithms improve the maximum node-degree by up to 64% and run up to four times faster than similar algorithms.

Distributed mitigation of content pollution in peer‐to‐peer video streaming networks

Video streaming has become increasingly popular in the Internet. Frequently, video transmissions are based on peer-to-peer networks, in which peers running on end-user hosts transmit data among themselves. An important security vulnerability of this strategy is that content can be easily altered by malicious users. Thus, it becomes essential to diagnose and fight content pollution in these systems. In this work, the authors present a novel strategy that relies on comparison-based diagnosis to mitigate content pollution in live video streaming peer-to-peer networks. This strategy is fully distributed and effectively combats the dissemination of content pollution. In the strategy, peers independently identify and avoid polluters. The solution works on top of the scalable overlay network Fireflies. Experimental results are presented showing the effectiveness and the low overhead of the solution. In particular, the strategy was able to significantly reduce content pollution propagation in diverse network configurations.

Polynomial-Time Data Reduction for the Subset Interconnection Design Problem

The NP-hard Subset Interconnection Design problem, also known as Minimum Topic-Connected Overlay, is motivated by numerous applications including the design of scalable overlay networks and vacuum systems. It has as input a finite set $V$ and a collection of subsets $V_1, V_2, \ldots, V_m \subseteq V$, and asks for a minimum-cardinality edge set $E$ such that for the graph $G=(V,E)$ all induced subgraphs $G[V_1], G[V_2], \ldots, G[V_m]$ are connected. We study Subset Interconnection Design in the context of polynomial-time data reduction rules that preserve the possibility of constructing optimal solutions. Our contribution is threefold: First, we show the incorrectness of earlier polynomial-time data reduction rules. Second, we show linear-time solvability in case of a constant number $m$ of subsets, implying fixed-parameter tractability for the parameter $m$. Third, we provide a fixed-parameter tractability result for small subset sizes and tree-like output graphs. To achieve our results, we elaborate on polynomial-time data reduction rules which also may be of practical use in solving Subset Interconnection Design.

SKIP +

Peer-to-peer systems rely on a scalable overlay network that enables efficient routing between its members. Hypercubic topologies facilitate such operations while each node only needs to connect to a small number of other nodes. In contrast to static communication networks, peer-to-peer networks allow nodes to adapt their neighbor set over time in order to react to join and leave events and failures. This article shows how to maintain such networks in a robust manner. Concretely, we present a distributed and self-stabilizing algorithm that constructs a (slightly extended) skip graph, SKIP + , in polylogarithmic time from any given initial state in which the overlay network is still weakly connected. This is an exponential improvement compared to previously known self-stabilizing algorithms for overlay networks. In addition, our algorithm handles individual joins and leaves locally and efficiently.

On the approximability and hardness of minimum topic connected overlay and its special instances

In the context of designing a scalable overlay network to support decentralized topic-based pub/sub communication, the Minimum Topic-Connected Overlay problem (Min-TCO in short) has been investigated: given a set of t topics and a collection of n users together with the lists of topics they are interested in, the aim is to connect these users to a network by a minimum number of edges such that every graph induced by users interested in a common topic is connected. It is known that Min-TCO is NP-hard and approximable within O(logt) in polynomial time.In this paper, we further investigate the problem and some of its special instances. We give various hardness results for instances where the number of topics in which a user is interested in is bounded by a constant, and also for the instances where the number of users interested in a common topic is a constant. For the latter case, we present a first constant approximation algorithm. We also present some polynomial-time algorithms for very restricted instances of Min-TCO.

Improving Computation Power by Reducing Query Response Time in Peer-to-Peer Environment

Problem statement: The Peer-to-Peer (P2P) was an emerging model and was being widely adopted in today’s internet computing.P2P traffic contributes the largest portion of the internet traffic which was unnecessary and led to delay. In existing Scalable Bipartite Overlay (SBO) network, Out of “N” peers, one group of peers were probing the messages (or) queries whereas the other group of peers were computing. This resulted in high query response time .Since the system was bipartite all peers were not capable of computing and probing, resulting in increased computation time. Approach: To overcome the above issue we propose a system RQR (Reduction on query response time) which is decentralized and unstructured where each and every peer can perform both probing and computing with no restrictions. Results: Optimal path finding algorithm is designed to find all possible path existing in the P2P network along with the optimal path. Conclusion: Our comparison with earlier approaches show that less than 60% of reduction on query response time which increases the system performance.

A new and effective hierarchical overlay structure for Peer-to-Peer networks

The tremendous growth of public interest in Peer-to-Peer (P2P) networks in recent years has initiated a lot of research work on how to design efficient overlay structures for P2P systems. Scalable overlay networks such as Chord, CAN, Pastry, and Tapestry provide no control over where data is stored and the location of the peers and resources is determined by the hash values of their identifiers and keys respectively. As a result, these overlays cannot support range queries and other proximity-aware complex queries directly. In this paper, we present a hierarchical P2P overlay network called SkipCluster, which is capable of supporting both exact-match and multi-dimensional range queries efficiently without consumption of extra memory space. SkipCluster is derived from skip graphs and SkipNet, but it has a two-tier hierarchical architecture.In both tiers, peers are connected in sequence according to the order of their peer IDs, and related resources are stored near each other without hashing of their resource keys. We design a novel data structure called Triple Linked List (TLL) to store each super-peer’s pointers in the higher tier, which can be used to find the longest prefix and speed up query routing of inter-cluster. In the lower tier, each intra-cluster peer’s routing table contains pointers with exponentially incremental distance. Experimental results show that SkipCluster can speed up both exact-match and range queries in different network sizes.

RTSP-based Mobile Peer-to-Peer Streaming System

Peer-to-peer is emerging as a potentially disruptive technology for content distribution in the mobile Internet. In addition to the already well-known peer-to-peer file sharing, real-time peer-to-peer streaming is gaining popularity. This paper presents an effective real-time peer-to-peer streaming system for the mobile environment. The basis for the system is a scalable overlay network which groups peer into clusters according to their proximity using RTT values between peers as a criteria for the cluster selection. The actual media delivery in the system is implemented using the partial RTP stream concept: the original RTP sessions related to a media delivery are split into a number of so-called partial streams according to a predefined set of parameters in such a way that it allows low-complexity reassembly of the original media session in real-time at the receiving end. Partial streams also help in utilizing the upload capacity with finer granularity than just per one original stream. This is beneficial in mobile environments where bandwidth can be scarce.

Algebra-Based Scalable Overlay Network Monitoring: Algorithms, Evaluation, and Applications

Overlay network monitoring enables distributed Internet applications to detect and recover from path outages and periods of degraded performance within seconds. For an overlay network with end hosts, existing systems either require measurements, and thus lack scalability, or can only estimate the latency but not congestion or failures. Our earlier extended abstract [Y. Chen, D. Bindel, and R. H. Katz, ldquoTomography-based overlay network monitoring,rdquo Proceedings of the ACM SIGCOMM Internet Measurement Conference (IMC), 2003] briefly proposes an algebraic approach that selectively monitors linearly independent paths that can fully describe all the paths. The loss rates and latency of these paths can be used to estimate the loss rates and latency of all other paths. Our scheme only assumes knowledge of the underlying IP topology, with links dynamically varying between lossy and normal. In this paper, we improve, implement, and extensively evaluate such a monitoring system. We further make the following contributions: i) scalability analysis indicating that for reasonably large n (e.g., 100), the growth of k is bounded as O(n log n), ii) efficient adaptation algorithms for topology changes, such as the addition or removal of end hosts and routing changes, iii) measurement load balancing schemes, iv) topology measurement error handling, and v) design and implementation of an adaptive streaming media system as a representative application. Both simulation and Internet experiments demonstrate we obtain highly accurate path loss rate estimation while adapting to topology changes within seconds and handling topology errors.

An algebraic approach to practical and scalable overlay network monitoring

Overlay network monitoring enables distributed Internet applications to detect and recover from path outages and periods of degraded performance within seconds. For an overlay network with n end hosts, existing systems either require O ( n 2 ) measurements, and thus lack scalability, or can only estimate the latency but not congestion or failures. Our earlier extended abstract [1] briefly proposes an algebraic approach that selectively monitors k linearly independent paths that can fully describe all the O ( n 2 ) paths. The loss rates and latency of these k paths can be used to estimate the loss rates and latency of all other paths. Our scheme only assumes knowledge of the underlying IP topology, with links dynamically varying between lossy and normal.In this paper, we improve, implement and extensively evaluate such a monitoring system. We further make the following contributions: i) scalability analysis indicating that for reasonably large n ( e.g. , 100), the growth of k is bounded as O ( n log n ), ii) efficient adaptation algorithms for topology changes, such as the addition or removal of end hosts and routing changes, iii) measurement load balancing schemes, and iv) topology measurement error handling. Both simulation and Internet experiments demonstrate we obtain highly accurate path loss rate estimation while adapting to topology changes within seconds and handling topology errors.

Large-scale personalized video streaming with program insertion proxies

Increasingly intelligent Internet overlay networks promise to deliver streaming media and value-added media services in ways that cannot be achieved easily with conventional broadcast networks. Such an overlay would allow an individual viewer or groups of viewers to receive unique programming content (e.g., commercial advertisements, entertainment) that matches their previously specified preferences. Toward this end, we introduce a scalable overlay network architecture and signaling mechanism that permits dynamic program insertions in live high-quality video streams transmitted over IP networks. We describe the implementation of an application proxy that dynamically inserts prerecorded video programs into NTSC Dl quality motion-JPEG streams with no visible artifacts. We argue that on-demand program switching is merely a first step toward a new generation of media services; increases in computing power ultimately permits network-based proxies to manipulate and augment multimedia content as it flows through the network.

Deterministic SkipNet

We present a deterministic scalable overlay network. In contrast, most previous overlay networks use randomness or hashing (pseudo-randomness) to achieve a uniform distribution of data and routing traffic.