Overlay Tree Research Articles

Design and implementation of an advanced MQTT broker for distributed pub/sub scenarios

MQTT is one of the most popular communication protocols for Internet of Things applications. Based on a publish/subscribe pattern, it relies on a single broker to exchange messages among clients according to topics of interest. However, such a centralized approach does not scale well and is prone to single point of failure risks, calling for solutions where multiple brokers cooperate together in a distributed fashion. In this paper, we present a complete solution for a distributed MQTT broker systems. We target several functional primitives which are key in such a scenario: broker discovery and failure recovery, overlay tree network creation and message routing. Moreover, we also focus on the case where multiple topics are present in the system. In such a scenario, a single tree-based overlay network connecting the different brokers may not be the most efficient solution. To cope with this issue, we propose a topic-based routing scheme for MQTT distributed brokers. The proposed solution creates multiple overlay networks in the distributed system, each one linking together only the brokers whose connected clients have interest in the same topics. We implement the complete system as an extension of the popular HiveMQ MQTT broker and perform several experiments to test its performance in scenarios characterized by a different publishers/subscribers configurations as well as number of topics existing in the system.

A Novel Modeling Method of the Crowdsourcing Design Process for Complex Products-Based an Object-Oriented Petri Net

Crowdsourcing design for complex products has open objectives and requires collaboration among different fields. It is difficult to use a single approach to describe the process in which the number and interaction of tasks and steps of tasks are changed dynamically depending on the execution state. Motivated by modeling the collaborative business process of crowdsourcing design in complex products, a crowdsourcing hierarchical process modeling approach based on an object-oriented Petri net is proposed. From the perspective of organization and task, the crowdsourcing object interaction and design process are modeled separately. First, an organization layer modeling approach based on expert review is proposed to describe the information transfer relationship among crowdsourcing objects from the organization perspective. Then, the task layer modeling approach based on the component decomposition idea is proposed. From the perspective of the task, the task is decomposed by using a design structure matrix (DSM) and a product structure tree to model the interaction of the design process. The object-oriented Petri net-is used to realize the representation of the model. Finally, the method is verified by taking the model of the crowdsourcing design of an air cooler as an example. The model subnet is analyzed by means of an overlay tree and incidence matrix, and the reliability and robustness of the model are proven. This method provides a reference for the process modeling of the crowdsourcing design of complex products.

An Efficient Real-Time Data Dissemination Multicast Protocol for Big Data in Wireless Sensor Networks

As data source for Big Data, wireless sensor networks play great role in data collection and dissemination. Specially, real-time communication remains one of the crucial research challenges, because of the high complexity and severe networking requirements on restrained node in wireless sensor networks. Moreover, current schemes are assumed to take general traffic model for real-time delivery so they lack adaptability. To solve this problem, few routing protocols have been designed to accommodate new real-time model, (m,k)-firm, which is regarded as the most applicable scheme for event as well as query based applications in wireless sensor networks. However, since current schemes for (m,k)-firm stream are proposed to support unicast communications only, they cannot be applied to multicast communications where many group members are willing to receive data packets from the sink node. To overcome this problem, we propose a new multicast scheme for (m,k)-firm stream to deliver data packet to group members. To construct a multicast tree, different types of overlay tree are constructed according to distance based priority (DBP) value. Simulation results prove that the proposed scheme can meet (m,k)-firm requirement and have longer network lifetime than existing schemes.

Error Resilient Multipath Video Delivery on Wireless Overlay Networks

Real time applications delivering multimedia data over wireless networks still pose many challenges due to high throughput and stringent delay requirements. Overlay networks with multipath transmission is the promising solution to address the above problems. But in wireless networks the maintenance of overlay networks induce additional overheads affecting the bulky and delay sensitive delivery of multimedia data. To minimize the overheads, this work introduces the Error Compensated Data Distribution Model (ECDD) that aids in reducing end to end delays and overheads arising from packet retransmissions. The ECDD adopts mTreebone algorithm to identify the unstable wireless nodes and construct overlay tree. The overlay tree is further split to support multipath transmissions. A sub packetization mechanism is adopted for multipath video data delivery in the ECDD. A forward error correction mechanism and sub-packet retransmission techniques adopted in ECDD enables to reduce the overhead and end to end delay. The simulation results presented in this paper prove that the ECDD model proposed achieves lower end to end delay and outperforms the existing models in place. Retransmission requests are minimized by about 52.27% and bit errors are reduced by about 23.93% than Sub-Packet based Multipath Load Distribution.

VMCast: A VM-Assisted Stability Enhancing Solution for Tree-Based Overlay Multicast

Tree-based overlay multicast is an effective group communication method for media streaming applications. However, a group member’s departure causes all of its descendants to be disconnected from the multicast tree for some time, which results in poor performance. The above problem is difficult to be addressed because overlay multicast tree is intrinsically instable. In this paper, we proposed a novel stability enhancing solution, VMCast, for tree-based overlay multicast. This solution uses two types of on-demand cloud virtual machines (VMs), i.e., multicast VMs (MVMs) and compensation VMs (CVMs). MVMs are used to disseminate the multicast data, whereas CVMs are used to offer streaming compensation. The used VMs in the same cloud datacenter constitute a VM cluster. Each VM cluster is responsible for a service domain (VMSD), and each group member belongs to a specific VMSD. The data source delivers the multicast data to MVMs through a reliable path, and MVMs further disseminate the data to group members along domain overlay multicast trees. The above approach structurally improves the stability of the overlay multicast tree. We further utilized CVM-based streaming compensation to enhance the stability of the data distribution in the VMSDs. VMCast can be used as an extension to existing tree-based overlay multicast solutions, to provide better services for media streaming applications. We applied VMCast to two application instances (i.e., HMTP and HCcast). The results show that it can obviously enhance the stability of the data distribution.

On Dynamic Video Source Decision in VANETs: An On-Demand Clustering Approach

In vehicular ad hoc networks (VANETs), video communication makes a significant contribution to quality of experience (QoE) for people on the road. However, the selection of the video source is an impediment to video delivery due to the high mobility and dynamic topology of VANETs. An improper provider not only leads to frequent interruptions of communications, but the transmission of the invalid video fragments would also result in the waster of precious bandwidth. To address the issue, a novel video source decision scheme, named Cluster and Dynamic Overlay based video delivery over VANETs (CDOV), is proposed in this paper. By the on-demand clustering approach, nodes with the same video requirement/supply and moving features are clustered. Further, in a cluster, an overlay tree is constructed dynamically based on the relation between supply and demand, in which all requesters can find their greedy optimal source easily. In addition, the intracluster communication and head-RSU communication are designed for video streaming over this network structure. Using extensive simulations, the effectiveness of the proposed scheme is demonstrated. Compared with two existing works, the proposed solution is capable of obtaining lower startup latency and higher delivery ratio.

Optimal backup parent pools for resilient multicast trees on peer-to-peer networks

Due to churn as well as node and link failures in peer-to-peer (P2P) networks, providing resilient multicast is a challenging issue, particularly for overlay trees. In this work, we first study analytical properties of the backup parent pool that aims at improving the resilience of overlay multicast trees in P2P video streaming. We then present a novel greedy degree-constrained multicast tree construction algorithm that addresses the tradeoffs between maximizing resiliency, maximizing bandwidth utilization, and minimizing delay. The choice of essential design parameters is studied together with seamlessness of the streaming under a variety of fault scenarios. Simulation results indicate that the overhead introduced by the mechanism is negligible if there is sufficient bandwidth in the system. Both analytic and simulation results indicate that the proposed approach improves resiliency, bandwidth utilization, and delay. Comparisons with an algorithm from the literature are carried out and it is observed that the proposed method is superior under a wide range of conditions.

Multi-Paths Multicast Overlay Tree in Hybrid Mobile Ad-Hoc Network (HYMANET)

Mobile ad-hoc network(MANET) is a self-organized network over infraless structured radio links. And hybrid mobile ad-hoc network(HYMANET) is a network which is combined with infra(AP) and MANET. In these networks, one of the most critical problems is the deficient network resource for receiving multimedia streams. The other problem is that overlay connection tree shuts down frequently following mobile node(MN)s’ movement. For solving these problems, this paper generates multi-paths overlay multicast connection tree using multiple parent-child relationships according to the number of hops towards AP(access point) in hybrid ad-hoc network. The proposed method can be used as service for live-video such as news, sports broadcasting image and movies in HYMANET. And this paper can minimize redundant multimedia streams that could appear when supporting multimedia service using multicast.

Mesh-Pull Backup Parent Pools for Video-On-Demand Multicast Trees

Resilient multicast is a challenging issue for overlay trees particularly because of high churn. In this work, we propose a mechanism that allows scalable video multicast. While the regular operation involves treepush of the video, any node that looses its parent on the tree solicits video from a predetermined backup set of nodes in a mesh-pull fashion. The main idea is to allocate less bandwidth for backup to improve bandwidth utilization while maintaining the best possible video quality. The choice of essential design parameters are studied together with seamlessness of the streaming under variety of fault scenarios. Simulation results indicate the optimality of the proposed approach as far as resiliency, bandwidth utilization, delay and video quality are concerned.

An Oblivious Spanning Tree for Single-Sink Buy-at-Bulk in Low Doubling-Dimension Graphs

We consider the problem of constructing a single spanning tree for the single-sink buy-at-bulk network design problem for doubling-dimension graphs. We compute a spanning tree to route a set of demands along a graph G to or from a designated sink node. The demands could be aggregated at (or symmetrically distributed to) intermediate edges where the fusion cost is specified by a nonnegative concave function f. We describe a novel approach for developing an oblivious spanning tree in the sense that it is independent of the number and location of data sources (or demands) and cost function at the edges. We present a deterministic, polynomial-time algorithm for constructing a spanning tree in low doubling-dimension graphs that guarantees a log3 D-approximation over the optimal cost, where D is the diameter of the graph G. With a constant fusion-cost function, our spanning tree gives an O(log3 D)-approximation for every Steiner tree that includes the sink. We also provide a Ω(log n) lower bound for any oblivious tree in low doubling-dimension graphs. To our knowledge, this is the first paper to propose a single spanning tree solution to the single-sink buy-at-bulk network design problem (as opposed to multiple overlay trees).

Optimization of Tree-like Core Overlay in Hybrid-structured Application-layer Multicast

The tree topology in multicast systems has high transmission efficiency, low latency, but poor resilience to node failures. In our work, some nodes are selected as backbone nodes to construct a tree-like core overlay. Backbone nodes are reliable enough and have strong upload capacity as well, which is helpful to overcome the shortcomings of tree topology. The core overlay is organized into a spanning tree while the whole overlay is of mesh-like topology. This paper focuses on improving the performance of the application-layer multicast overlay by optimizing the core overlay which is periodically adjusted with the proposed optimization algorithm. Our approach is to construct the overlay tree based on the out-degree weighted reliability where the reliability of a node is weighted by its upload bandwidth (out-degree). There is no illegal solution during the evolution which ensures the evolution efficiency. Simulation results show that the proposed approach greatly enhances the reliability of the tree-like core overlay systems and achieves shorter delay simultaneously. Its reliability performance is better than the reliability-first algorithm and its delay is very close to that of the degree-first algorithm. The complexity of the proposed algorithm is acceptable for application. Therefore the proposed approach is efficient for the topology optimization of a real multicast overlay.

A Robust Streaming Media System

Abstract Presently, application layer multicast protocols (ALM) are proposed as substitute for IP multicast and have made extraordinary achievements. Integrated with Multi-data-stream mode such as Multiple Description Coding (MDC), ALM becomes more scalable and robust in high-dynamic Internet environment compared with single data stream. Although MDC can provide a flexible data transmission style, the synchronization of different descriptions encoded from one video source is proved to be difficult due to different delay on diverse transmission paths. In this paper, an ALM system called HMDC is proposed to improve accepted video quality of streaming media, hosts can join the separate overlay trees in different layers simultaneously, then the maximum synchronized descriptions of the same layer are worked out to acquire the best video quality. Simulations implemented on Internet-like topology indicate that HMDC achieves better video quality, lower link stress, higher robustness and comparable latency compared with traditional ALM protocols.

A Secure and Stable Multicast Overlay Network with Load Balancing for Scalable IPTV Services

The emerging multimedia Internet application IPTV over P2P network preserves significant advantages in scalability. IPTV media content delivered in P2P networks over public Internet still preserves the issues of privacy and intellectual property rights. In this paper, we use SIP protocol to construct a secure application-layer multicast overlay network for IPTV, called SIPTVMON. SIPTVMON can secure all the IPTV media delivery paths against eavesdroppers via elliptic-curve Diffie-Hellman (ECDH) key exchange on SIP signaling and AES encryption. Its load-balancing overlay tree is also optimized from peer heterogeneity and churn of peer joining and leaving to minimize both service degradation and latency. The performance results from large-scale simulations and experiments on different optimization criteria demonstrate SIPTVMON's cost effectiveness in quality of privacy protection, stability from user churn, and good perceptual quality of objective PSNR values for scalable IPTV services over Internet.

On predictable large-scale data delivery in prefix-based virtualized content networks

IPTV, software replication, and other large scale content distribution services raise the need for fast and efficient content delivery mechanisms in underlay as well as overlay networks. Multicast, the natural approach on the network layer, has not been deployed globally, and solutions are pushed to the application layer. For a flexible, sustainable deployment the distribution mechanisms in use should scale up to many thousand group members and provide predictable performance to dynamically adjust to actual performance requirements. In this paper, we present a rigorous analytical model complemented by extensive simulations for content delivery on prefix-based overlay trees. We examine BIDIR-SAM, a generic multicast distribution scheme guided by prefixes that allow for late next-hop binding. Our evaluation quantitatively substantiates all major performance aspects. We derive the distribution functions of hop counts, packet replication loads, as well as all relevant cost measures, which scale logarithmically with network and receiver sizes. Prefix-based content delivery exhibits a churn resistance similar to the underlying key-based routing layer and a multicast efficiency scaling factor close to native group communication protocols. These results make the approach especially suitable for large and very large content groups.

A new Application-Level Multicast technique for stable, robust and efficient overlay tree construction

Even though Application Level Multicast (ALM) protocols are proposed as an efficient alternative to the classical IP multicast model, they still suffer from the same rearrangement overhead problem. Indeed, when a member joins or leaves the multicast session, a part of the overlay tree should be rearranged. This rearrangement overhead can significantly disturb the fluidity of the application data stream received by other members. In addition, the efficiently of the constructed overlay tree depends entirely on the honesty and the cooperation of all participating members. However such behavior cannot be guaranteed and some selfish and non-cooperative nodes may take profit from the honesty of other members in the overlay. In this paper we present a new technique, called Membership Duration Aware ALM (MDA technique), that takes advantage of the announced membership duration of each new joining member in order to construct a more stable and robust overlay trees. Our technique can be used together with any existing centralized or distributed ALM protocol to reduce the overall rearrangement overhead. Simulation results confirm the efficiency of our technique when applied to both centralized and distributed ALM protocol. Then, we analyze in depth the impact of cheating nodes on the performances and the stability of overlay trees constructed by our MDA-ALM protocol.

The Multiple Implied Overlay Tree P2P Progressive Transmission Method of Mixed Subdivision Mesh

This paper presents a progressive transmission method of mixed subdivision mesh based on multiple implied overlay tree P2P network. Peer-to-Peer networks are widely used in media transmission because of its excellent network scalability. Scalable media service, especially mixed subdivision mesh is proposed to adapt to different terminal processing abilities. For stratified characteristics of progressive mesh, we use multiple implied tree cover P2P technology to transmit 3D mesh of mixed subdivision model. It effectively realizes to reduce MAX, average system load link and lower the average Relative delay penalty. Finally a complete set of solution is given, which is proved to be an effective method by simulation experiment.

An overlay multicast protocol for live streaming and delay-guaranteed interactive media

In many collaborative multimedia applications, there is often a requirement for simultaneously supporting live streaming and shareable interaction. A major challenge in designing such an application by overlay multicast is how to simultaneously provide scalable live streaming and delay-guaranteed interactive media. Live streaming by overlay multicast incurs additional application-layer latency, which conflicts with the delay-sensitive property of interactive media. To handle this dilemma, in this paper, we propose a layered degree-constrained overlay multicast protocol, which organizes the overlay multicast tree as a layered degree-constrained core tree and an extended tree. The core tree maintains available resources in its top layers for subsequent two-way interaction, whereas the extended tree expands the core tree for one-way live streaming. Our simulation and experimental results show that the proposed overlay multicast protocol can simultaneously provide delay-guaranteed interactive media as well as scalable live streaming.

Overlay tree construction to distribute layered streaming by application layer multicast

With the development of broadband technologies, more and more multimedia contents such as video or bulk file are being distributed over the Internet. However, an important issue to be resolved is how to construct an efficient overlay to provide users with the acceptable throughput and delay, while receiving multimedia contents. Therefore, this paper designs a tree construction algorithm by distributing the layered steaming over the ALM in order to improve both throughput and user delay. Firstly, to improve the throughput, based on theory analysis, we define a layered degree, out/in-degree and the corresponding constraints to manage the layered streaming and nodes. Secondly, a novel method, called Bi-cast, is proposed to reduce user delay during data-transmission. Thirdly, by using the defined degrees and the Bi-cast, we present a tree construction algorithm. Both the simulation and the implementation are carried out, and the results show that our proposal can obtain better performance than other conventional methods.

An Interest Management Architecture by ALM and Region Partition for Large-Scale Distributed Virtual Environment

Large-Scale Distributed Virtual Environments (LSDVEs) must deal with the challenge of supporting a large number of users interacting on the Internet while keeping the communication among the parties synchronous and highly reactive. Interest Management is widely used to filter unnecessary messages in such systems. We present an interest management architecture that supports a large number of users on the Internet by dividing the virtual environment into multiple adjacent hexagonal regions. In the architecture, messaging among entities is based on a multi-server communication infrastructure by the Application Layer Multicasting (ALM). And each region is mapped to an ALM tree with a master node constructing the overlay tree and managing nodes that lie in its region. The interest area of each entity is composed of two parts: the inner hexagon and outer hexagon. By the two parts, inter-region as well as intra-region interaction is supported to maintain a continuous view without increasing too much traffic. In addition, the buffer borderlines near the boundaries are introduced to reduce the number of connections and disconnections that occur when a node frequently moves at the boundary of the regions without affecting the messaging of the entities and modifying the partition structure.

Overlay Real-Time Video Multicast System

In this paper, we present an effective overlay real-time video multicast system over the Internet. The proposed system effectively integrates overlay multicast technology and video compression technology. Overlay multicast tree and target bit rate are determined to satisfy the given average delay constraint, and H.263+ rate control is implemented to enhance the human visual perceptual quality over the multicast tree. Finally, experimental results are provided to show the performance of the proposed overlay video multicast system over the Internet.