Broadcast Latency Research Articles

The ExaNeSt Prototype: Evaluation of Efficient HPC Communication Hardware in an ARM-based Multi-FPGA Rack

We present and evaluate the ExaNeSt Prototype, which compactly packages 128 Xilinx ZU9EG MPSoCs, 2 TBytes of DRAM, and 8 TBytes of SSD into a liquid-cooled rack, using a custom interconnection hardware based on 10 Gbps links. We developed this testbed in 2016-2019 in order to leverage the flexibility of FPGAs for experimenting with efficient hardware support for HPC communication among tens of thousands of processors and accelerators in the quest towards Exascale systems and beyond. In the years since then, we carefully studied this system, and we present our key design choices and insights resulting from our measurement and analysis. We developed this testbed, from architecture to the PCBs and the runtime software, within the ExaNeSt project. It is fully operational in configurations with up to 8x4x4 MPSoC nodes. It achieves high density through tight board design, while also leveraging state-of-the-art liquid cooling technology. In this paper, we present a thorough architectural analysis, along with important aspects of our infrastructure development. Our custom interconnect includes a low-cost low-latency network interface, offering user-level, zero-copy RDMA, which we coupled with the ARMv8 processors in the MPSoCs. We further developed the corresponding runtimes that allow us to test real MPI applications on the large-scale testbed. We evaluated our platform through MPI microbenchmarks, mini, and full MPI applications. Single hop, one way latency is \(1.3\) \(\mu\) s; approximately \(0.47\) \(\mu\) s out of these are attributed to network interface and the user-space library that exposes its functionality to the runtime. Latency over longer paths increases as expected, reaching \(2.55\) \(\mu\) s for a five-hop path. Bandwidth tests show that, for single hop, link utilization reaches \(82\%\) of the theoretical capacity. Microbenchmarks based on MPI collectives reveal that broadcast latency scales as expected when the number of participating ranks increases. We also implemented a custom MPI_Allreduce accelerator in the network interface, which reduces the latency of such collectives by up to \(88\%\) . We assess performance scaling through weak and strong scaling tests for HPCG, LAMMPS, and the miniFE mini application; for all these tests, parallelization efficiency is at least \(69\%\) , or better.

Structure-Adaptive and Power-Aware Broadcast Scheduling for Multihop Wireless-Powered IoT Networks

Wireless Power Transfer technology, which can charge IoT devices over the air, has become a promising technology for IoT networks. In wireless-powered IoT networks, broadcasting is a fundamental networking service for disseminating messages to the whole network. To seek a fast and collision-free broadcast schedule, the problem of Minimum Latency Broadcast Scheduling (MLBS) has been well studied when nodes are energy-abundant. However, in wireless-powered networks, a node can only receive or transmit packets after it has harvested enough energy. In such networks, it is of great importance to exploit the divergent harvested energy to reduce the broadcast latency. Unfortunately, existing works always assume a predetermined tree and a fixed transmission power for broadcast scheduling, which greatly limits their performance. Thus, in this article, we investigate the first work for the MLBS problem in wireless-powered networks without relying on predetermined trees. First, the problem is formulated and proved to be NP-hard. Then, two structure-adaptive scheduling algorithms are proposed with a theoretical bound, which can intertwine the construction of broadcast tree with the computation of an energy-aware schedule simultaneously. Furthermore, a power-aware scheduling method is also proposed to take the structure of the broadcast tree, the adjustment of nodes’ transmission powers, and the interference during transmissions into account simultaneously. Additionally, the algorithm for the MLBS problem under the physical interference model is also studied. Finally, the theoretical analysis and simulation results verify that the proposed algorithms have high performance in terms of latency.

Distributed Stable Multisource Global Broadcast for SINR-Based Wireless Multihop Networks

Multi-source global broadcast is a fundamental problem in multi-hop wireless networks. The Static Multi-source Global Broadcast problem (SMGB), which considers static packet injection at all source nodes, has been extensively studied in recent years. However, packets are more likely to be continuously injected over time in realistic multi-hop wireless networks. In this paper, we focus on studying the Dynamic Multi-source Global Broadcast problem (DMGB), in which packets are continuously injected to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k\geq 2$ </tex-math></inline-formula> ) source nodes in the network according to a widely-used dynamic packet injection model and the objective is to disseminate each injected packet across the whole network quickly. We solve this DMGB problem under the Signal-to-Interference-plus-Noise-Ratio (SINR) interference model. Specifically, we first present a distributed randomized algorithm for solving the SMGB problem. We then iterate this SMGB algorithm repeatedly to construct a distributed DMGB algorithm. We prove the proposed DMGB algorithm is stable, i.e., the expected number of packets in each node’s message queue is bounded at any time and further the expected global broadcast latency for each injected packet is bounded. Simulation results validate the effectiveness of the proposed DMGB algorithm.

Broadcast Scheduling Protocols in Multi-Hop Mobile Ad hoc Networks

When packets are sending in multi-hop mobile unintended networks numerous problems occur like flooding, rebroadcast, broadcast latency, power conservation and collision. If multiple transmission of packets simultaneously in MANETs that using the slot assignments approach, when additional channels are transmitting at the same time as the first slot allocations, interference may occur at the nodes. Because of the multi-hops data transfer, the network performance is hampered by the constrained bandwidth and therefore the self-initiated topological alterations. Therefore, a broadcast algorithm is important within the mobile ad hoc network for collision control and reliable communication. This paper proposes two new broadcasting protocols: modify SRBS and DSB algorithms. The planned algorithms outperform context of efficiency, reliability, traffic overload and reachability in highly mobile networks is an enhanced performance within the different environments. Evaluation of simulation results with other well-known exiting protocols as DFCN and PEGSP algorithms shows that the proposed protocol performance is best within the wireless network and channel bandwidths are well utilized within the network.

Structure-Free Broadcast Scheduling for Duty-Cycled Multihop Wireless Sensor Networks

Broadcasting is an essential operation in wireless networks for disseminating the message from the source node to all other nodes. Unfortunately, the problem of Minimum Latency Broadcast Scheduling (MLBS) in duty-cycled wireless sensor networks is not well studied. In existing works, the construction of broadcast tree and the scheduling of transmissions are conducted separately, where a tree-based structure is used as the input of the scheduling algorithm. Relying on a pre-determined tree may result in a much large latency even using the optimal scheduling method. Thus, the MLBS problem in duty-cycled WSNs without the above limitation is investigated in this paper. First, to avoid relying on a pre-determined structure, a two-step scheduling algorithm is proposed to construct the broadcast tree and compute a collision-free schedule simultaneously. To the best of our knowledge, this is the first work that can integrate these two kinds of operations together. Second, a novel transmission mode, i.e., concurrent broadcasting, is first introduced for wireless networks and several techniques are designed to further improve the broadcast latency. Third, the multiple messages broadcasting and all-to-all broadcasting algorithms, which can generate a series of broadcast schedules independently without a pre-determined tree, are also proposed by taking care of the collisions in both the current and the previous broadcast schedules. Finally, the theoretical analysis and experimental results demonstrate the efficiency of the proposed algorithms in terms of latency.

Analysis and Optimization of Multihop Broadcast Communication in the Internet of Vehicles Based on C-V2X Mode 4

Cellular Vehicle-to-Everything (C-V2X) Mode 4, developed by 3GPP, supports vehicle-to-vehicle communication without a cellular base station via LTE sidelink. The messages transmitted in the Internet of Vehicles (IoV) include periodic single-hop broadcast Beacon messages and multi-hop broadcast Emergency messages triggered by events. In this paper, we analyze the impact of multi-hop broadcast of Emergency messages in C-V2X Mode 4 on the performance of Beacon messages and propose collision probability models for Beacon messages and Emergency messages. The non-periodicity nature and lower broadcast latency requirement of the multi-hop Emergency messages may cause resource allocation conflicts in the resource selection process of C-V2X Mode 4, thus result in forwarding collision of Emergency messages and collision of subsequent Beacon messages. Therefore, we propose several multi-hop broadcast schemes for Emergency messages to reduce the collision probability by assigning independent resource grants for Emergency messages and adjusting the number of forwarding nodes. The performance of Beacon messages and Emergency messages of several different forwarding schemes proposed are simulated. The simulation results show that the proposed broadcasting schemes significantly improve the performance of Beacon messages and Emergency messages.

Distribution of multi-hop latency for probabilistic broadcasting protocols in grid-based Wireless Sensor Networks

This paper provides a theoretical model for the computation of the probability density function of multi-hop broadcast latency when using probabilistic broadcasting schemes in Wireless Sensor Networks (WSN). The probability density functions of multi-hop latency are derived by considering the per-hop random delays employed in protocols and the geometry of the position of nodes. The assumptions of the model allow computation of probability density functions that approximate simulation results in a variety of network scenarios and per-hop random delay configurations. The accuracy of the theoretical predictions increases in scenarios with moderate redundancy of broadcast packets.

Energy efficient broadcast protocols for asynchronous duty-cycled wireless sensor networks

In asynchronous duty-cycled wireless sensor networks (WSNs), each node has its own sleep schedule that is independent of the schedules of neighboring nodes. Since each node follows its own independent sleep schedule, the probability that a node’s neighbors will wake up concurrently is quite low. For this reason, each single hop broadcast must be completed through multiple unicast transmissions, and the sender node must remain awake until all of its neighboring nodes have received the broadcast packet. In this paper, we present two energy efficient broadcast protocols, namely Broadcast Progress-based Efficient Multihop Broadcast (BPEMB) protocol and Tree-based Efficient Multihop Broadcast (TEMB) protocol, for asynchronous duty-cycled WSNs. In both of the proposed protocols, the decision of whether to transmit the broadcast packet is based on the link quality information and without any forwarder’s guidance list. The redundant transmissions and collisions are avoided effectively with cooperation among the neighbors, so that the node with the best link quality to the receiver node takes responsibility to forward the packet to the receiver node. With this approach, nodes that are not involved in transmitting the broadcast packet can go to sleep and save their energy. In the BPEMB protocol, the forwarding decision of transmitting the broadcast packet is also influenced by the broadcast progress information, which is piggybacked in the broadcast packet. Through extensive simulations, we demonstrate that the proposed protocols can significantly improve the broadcast efficiency through reducing the energy consumption and minimizing the broadcast latency. The evaluation results also show that both the BPEMB and TEMB protocols can substantially reduce the collisions and redundant transmissions in asynchronous duty-cycled WSNs.

Broadcast Scheduling in Battery-Free Wireless Sensor Networks

Battery-Free Wireless Sensor Networks (BF-WSNs) are newly emerging Wireless Sensor Networks (WSNs) to break through the energy limitations of traditional WSNs. In BF-WSNs, the broadcast scheduling problem is more challenging than that in traditional WSNs. This article investigates the broadcast scheduling problem in BF-WSNs with the purpose of minimizing broadcast latency. The Minimum-Latency Broadcast Scheduling problem in BF-WSNs (MLBS-BF) is formally defined and its NP-hardness is proved. Three approximation algorithms for solving the MLBS-BF problem are proposed. The broadcast latency of the broadcast schedules produced by the proposed algorithms is analyzed. The correctness and approximation ratio of the proposed algorithms are also proved. Finally, extensive simulations are conducted to evaluate the performances of the proposed algorithms. The simulation results show that the proposed algorithms have high performance.

Latency Reduction in Probabilistic Broadcast Protocols for Ad Hoc Networks

This letter introduces a strategy to reduce broadcast latency in multi-hop ad-hoc networks; this overall reduction is the result of the accumulation of a series of per-node delay reductions. Current broadcasting protocols employ uniformly distributed random delays on a per-node basis, while the proposed strategy consists of using a truncated-exponential distribution to determine these delays. Such an approach can reduce latency significantly while maintaining broadcasting reachability, yet it entails minimal additional overhead in relation to existing protocols.

Optimized large-message broadcast for deep learning workloads: MPI, MPI+NCCL, or NCCL2?

Traditionally, MPI runtimes have been designed for clusters with a large number of nodes. However, with the advent of MPI+CUDA applications and GPU clusters with a relatively smaller number of nodes, efficient communication schemes need to be designed for such systems. This coupled with new application workloads brought forward by Deep Learning (DL) frameworks like Caffe and Microsoft Cognitive Toolkit (CNTK) pose additional design constraints due to very large message communication of GPU buffers during the training phase. In this context, special-purpose libraries like NVIDIA NCCL have emerged to deal with DL workloads. In this paper, we address these new challenges for MPI runtimes and propose two new designs to deal with them: (1) A pipelined chain (PC) design for MPI_Bcast that provides efficient intra- and inter-node communication of GPU buffers, and (2) A Topology-Aware pipelined chain (TA-PC) design for systems with multiple GPUs to fully exploit all the available PCIe links available within a multi-GPU node.To highlight the benefits of our designs, we present an in-depth performance landscape for the proposed MPI_Bcast (MPI) designs, our earlier NCCL-based MPI_Bcast (MPI+NCCL1) design, and ncclBroadcast (NCCL2) design. The proposed designs offer up to 14 × and 16.6 × better performance than MPI+NCCL1 based solutions for intra- and inter-node broadcast latency, respectively. With the recent introduction of NCCL2 (inter-node capable) library, we have enhanced our performance results by adding comparisons for the proposed MPI_Bcast designs as well as ncclBroadcast (NCCL2) design. We report up to 10 × better performance for small and medium message sizes and comparable performance for large message sizes. We also observed that the TA-PC design is up to 50% better than the PC design for MPI_Bcast to 64 GPUs. Furthermore, we provide application level performance comparison using a CUDA-Aware version of CNTK called CA-CNTK. The proposed MPI_Bcast designs provide up to 7% improvement over MPI+NCCL based solutions for data parallel training of the VGG network on 128 GPUs. We present our performance evaluation on three GPU clusters with diverse characteristics: (1) KESCH; a dense multi-GPU system with 8 K80 GPU cards per node, (2) RI2; with a single K80 GPU card per node, and (3) Owens; with a single P100 GPU per node.

Characterizing User Behaviors in Mobile Personal Livecast

Mobile personal livecast (MPL) services are emerging and have received great attention recently. In MPL, numerous and geo-distributed ordinary people broadcast their video contents to worldwide viewers. Different from conventional social networking services like Twitter and Facebook, which have a tolerance for interaction delay, the interactions (e.g., chat messages) in a personal livecast must be in real-time with low feedback latency. These unique characteristics inspire us to: (1) investigate how the relationships (e.g., social links and geo-locations) between viewers and broadcasters influence the user behaviors, which has yet to be explored in depth; and (2) explore insights to benefit the improvement of system performance. In this article, we carry out extensive measurements of a representative MPL system, with a large-scale dataset containing 11M users. In the current costly and limited cloud-based MPL system, which is faced with scalability problem, we find: (1) the long content uploading distances between broadcasters and cloud ingesting servers result in an impaired system QoS, including a high broadcast latency and a frequently buffering events; and (2) most of the broadcasters in MPL are geographically locally popular (the majority of the views come from the same region of the broadcaster), which consume vast computation and bandwidth resources of the clouds and Content Delivery Networks. Fortunately, the emergence of edge computing, which provides cloud-computing capabilities at the edge of the mobile network, naturally sheds new light on the MPL system; i.e., localized ingesting, transcoding, and delivering locally popular live content is possible. Based on these critical observations, we propose an edge-assisted MPL system that collaboratively utilizes the core-cloud and abundant edge computing resources to improve the system efficiency and scalability. In our framework, we consider a dynamic broadcaster assignment to minimize the broadcast latency while keeping the resource lease cost low. We formulate the broadcaster scheduling as a stable matching with migration problem to solve it effectively. Compared with the current pure cloud-based system, our edge-assisted delivery approach reduces the broadcast latency by about 35%.

Dandelion++

Recent work has demonstrated significant anonymity vulnerabilities in Bitcoin's networking stack. In particular, the current mechanism for broadcasting Bitcoin transactions allows third-party observers to link transactions to the IP addresses that originated them. This lays the groundwork for low-cost, large-scale deanonymization attacks. In this work, we present \Algopp, a first-principles defense against large-scale deanonymization attacks with near-optimal information-theoretic guarantees. Dandelion++ builds upon a recent proposal called Dandelion that exhibited similar goals. However, in this paper, we highlight some simplifying assumptions made in Dandelion, and show how they can lead to serious deanonymization attacks when violated. In contrast, Dandelion++ defends against stronger adversaries that are allowed to disobey protocol. Dandelion++ is lightweight, scalable, and completely interoperable with the existing Bitcoin network. We evaluate it through experiments on Bitcoin's mainnet (i.e., the live Bitcoin network) to demonstrate its interoperability and low broadcast latency overhead.

Dandelion++

Recent work has demonstrated significant anonymity vulnerabilities in Bitcoin's networking stack. In particular, the current mechanism for broadcasting Bitcoin transactions allows third-party observers to link transactions to the IP addresses that originated them. This lays the groundwork for low-cost, large-scale deanonymization attacks. In this work, we present Dandelion++, a first-principles defense against large-scale deanonymization attacks with near-optimal information-theoretic guarantees. Dandelion++ builds upon a recent proposal called Dandelion that exhibited similar goals. However, in this paper, we highlight some simplifying assumptions made in Dandelion, and show how they can lead to serious deanonymization attacks when violated. In contrast, Dandelion++ defends against stronger adversaries that are allowed to disobey protocol. Dandleion++ is lightweight, scalable, and completely interoperable with the existing Bitcoin network.We evaluate it through experiments on Bitcoin's mainnet (i.e., the live Bitcoin network) to demonstrate its interoperability and low broadcast latency overhead.

Critical‐Path Aware Scheduling for Latency Efficient Broadcast in Duty‐Cycled Wireless Sensor Networks

Minimum latency scheduling has arisen as one of the most crucial problems for broadcasting in duty‐cycled Wireless Sensor Networks (WSNs). Typical solutions for the broadcast scheduling iteratively search for nodes able to transmit a message simultaneously. Other nodes are prevented from transmissions to ensure that there is no collision in a network. Such collision‐preventions result in extra delays for a broadcast and may increase overall latency if the delays occur along critical paths of the network. To facilitate the broadcast latency minimization, we propose a novel approach, critical‐path aware scheduling (CAS), which schedules transmissions with a preference of nodes in critical paths of a duty‐cycled WSN. This paper presents two schemes employing CAS which produce collision‐free and collision‐tolerant broadcast schedules, respectively. The collision‐free CAS scheme guarantees an approximation ratio of (Δ − 1)T in terms of latency, where Δ denotes the maximum node degree in a network. By allowing collision at noncritical nodes, the collision‐tolerant CAS scheme reduces up to 10.2 percent broadcast latency compared with the collision‐free ones while requiring additional transmissions for the noncritical nodes experiencing collisions. Simulation results show that broadcast latencies of the two proposed schemes are significantly shorter than those of the existing methods.

Live Broadcast With Community Interactions: Bottlenecks and Optimizations

Recent years have witnessed the rapid growth of new live broadcast services, represented by Twitch.tv and YouTube live events, where videos are crowdsourced from amateur users (e.g., game players), rather than from commercial and professional TV broadcaster or content providers. The viewers also actively contribute to the content through embedded open-chat channels. Such community interactions among viewers, or even between broadcasters and viewers, make content generation highly diversified and engaging, particularly for the young generation. In this context, cross-viewer synchronization is highly desirable; otherwise the viewers with shorter broadcast latency may act as spoilers, significantly affecting the user experience of other viewers. In this paper, we show that the end-to-end delay has a dramatically amplified impact on the broadcast latency for individual viewers. We suggest smart rate adaptation to achieve cross-viewer synchronization, and develop distributed algorithms based on dual decomposition. We further extend our solution to the cloud environment, and present the concept of ShadowCast, which moves broadcasters to the cloud to provide high-quality streams beyond broadcasters’ network bandwidth constraint. Its practicability and effectiveness is demonstrated by our implementation and test bed experiments.

Collision-tolerant broadcast scheduling in duty-cycled wireless sensor networks

The minimum-latency broadcast problem in duty-cycled wireless sensor networks has received significant attention over the last few years. A common approach for the problem is to assign collision-free transmitting times to forwarding nodes for disseminating a message from one source node to all other nodes according to their given duty-cycle schedules and transmission ranges. However, preventing collision for all transmissions may increase latency in the broadcast schedules. This paper proposes a novel strategy of Collision-Tolerant Scheduling (CTS) that offers an opportunity to reduce broadcast latency by allowing collisions at non-critical nodes to speed up the broadcast process for critical ones. The completion of broadcast scheduling, i.e. all nodes receive a broadcast message, is ensured by additionally transmitting the message to non-critical nodes experiencing collision. We employ the scheduling strategy in two proposed broadcast schemes: Degree-based CTS (DCTS) and MIS-based CTS (MCTS), which select forwarding nodes based on the node degree and maximal independent set information, respectively. The results of both theoretical analysis and simulation reveal the remarkable advantages of CTS in minimizing broadcast latency in duty-cycled WSNs. DCTS and MCTS guarantee approximation ratios of (Δ−1)T and 12T in terms of broadcast latency, where Δ and T denote the maximum node degree and the number of time slots in a working period, respectively. The two schemes reduce to at least 94 percent of the broadcast latency compared with existing schemes, while slightly increasing the number of transmissions due to the additional transmissions. Thanks to the latency reduction, the proposed schemes require 93 percent less energy than existing ones.

Low-Latency Multi-Flow Cooperative Broadcast in Fading Wireless Networks

Though a cooperative broadcast scheme has been proposed for fading environments, it has two defects: First, it only handles a packet flow from a single source node in the network, but does not consider the scenario of multiple packet flows simultaneously broadcasted from different source nodes. Second, it only allows a single relay node to forward a packet in each time slot, though multiple relay nodes forwarding in a time slot can significantly reduce broadcast latency. In this paper, we aim achieve low-latency multi-flow broadcast in wireless multi-hop networks with fading channels. To describe the interference among the transmission in different flows, we incorporate the Rayleigh fading model to the signal to noise ratio (SNR) model. Then, we introduce a cooperative diversity scheme which allows multiple relays forwarding in a time slot to reduce broadcast latency. We then formulate an interesting problem: In a fading environment, what is the optimal relay allocation schedule to minimize the broadcast latency? We propose a warm up heuristic algorithm for single-flow cooperative broadcast, based on which, we further propose a heuristic algorithm for multi-flow cooperative broadcast. Simulation results demonstrate that the two algorithms achieve lower broadcast latency than a previous method.

Y-Hamiltonian Layers Broadcast Algorithm

Anew approach to broadcast in wormhole routed three-dimensional networks is proposed. One of the most important process in communication and parallel computer is broadcast approach.. The approach of this case of Broadcasting is to send the message from one source to all destinations in the network which corresponds to one-to-all communication. Wormhole routing is a fundamental routing mechanism in modern parallel computers which is characterized with low communication latency. We show how to apply this approach to 3-D meshes. Wormhole routing is divided the packets into set of FLITS (flow control digits). The first Flit of the packet (Header Flit) is containing the destination address and all subsets flits will follow the routing way of the header Flit. In this paper, we consider an efficient algorithm for broadcasting on an all-port wormhole-routed 3D mesh with arbitrary size. We introduce an efficient algorithm, Y-Hamiltonian Layers Broadcast(Y-HLB). In this paper the behaviors of this algorithm were compared to the previous results, our paradigm reduces broadcast latency and is simpler. In this paper our simulation results show the average of our proposed algorithm over the other algorithms that presented.

X-Hamiltonian Surface Broadcast Algorithm

Broadcast is one of the most important approach in distributed memory parallel computers that is used to find a routing approach from one source to all nodes in the mesh. Broadcasting is a data communication task in which corresponds to one-to-all communication. Routing schema is the approach used to determine the road that is used to send a message from a source node to destination nodes. In this paper, we propose an efficient algorithm for broadcasting on an all-port wormhole-routed 3D mesh with arbitrary size. Wormhole routing is a fundamental routing mechanism in modern parallel computers which is characterized with low communication latency. We show how to apply this approach to 3-D meshes. In wormhole, routing large network packets are broken into small pieces called FLITs (flow control digits). The destination address is kept in the first flit which is called the header flit and sets up the routing behavior for all subsequent flits associated with the packet. In this paper, we introduce an efficient algorithm, X-Hamiltonian Surface Broadcast (X-HSB) which uses broadcast communication facility with deadlock-free wormhole routing in general three dimensional networks. In this paper, the behaviors of this algorithm are compared to the previous results using simulation; our paradigm reduces broadcast latency and is simpler. The results presented in this paper indicate the advantage of our proposed algorithm.