Wormhole-routed Networks Research Articles

Computationally Efficient Worst-Case Analysis of Flow-Controlled Networks With Network Calculus

Networks with hop-by-hop flow control occur in several contexts, from data centers to systems architectures (e.g., wormhole-routing networks on chip). A worst-case end-to-end delay in such networks can be computed using Network Calculus (NC), an algebraic theory where traffic and service guarantees are represented as curves in a Cartesian plane. NC uses transformation operations, e.g., the min-plus convolution, to model how the traffic profile changes with the traversal of network nodes. NC allows one to model flow-controlled systems, hence one can compute the end-to-end <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">service curve</i> describing the minimum service guaranteed to a flow traversing a tandem of flow-controlled nodes. However, while the algebraic expression of such an end-to-end service curve is quite compact, its computation is often intractable from an algorithmic standpoint: data structures tend to grow quickly to unfeasibly large sizes, making operations intractable, even with as few as three hops. In this paper, we propose computational and algebraic techniques to mitigate the above problem. We show that existing techniques (such as reduction to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">compact domains</i> ) cannot be used in this case, and propose an arsenal of solutions, which include methods to mitigate the data representation space explosion as well as computationally efficient algorithms for the min-plus convolution operation. We show that our solutions allow a significant speedup, enable analysis of previously unfeasible case studies, and - since they do not rely on any approximation - still provide exact results.

A Decision Procedure for Deadlock-Free Routing in Wormhole Networks

Deadlock freedom is a key challenge in the design of communication networks. Wormhole switching is a popular switching technique, which is also prone to deadlocks. Deadlock analysis of routing functions is a manual and complex task. We propose an algorithm that automatically proves routing functions deadlock-free or outputs a minimal counter-example explaining the source of the deadlock. Our algorithm is the first to automatically check a necessary and sufficient condition for deadlock-free routing. We illustrate its efficiency in a complex adaptive routing function for torus topologies. Results are encouraging. Deciding deadlock freedom is co-NP-Complete for wormhole networks. Nevertheless, our tool proves a 13 × 13 torus deadlock-free within seconds. Finding minimal deadlocks is more difficult. Our tool needs four minutes to find a minimal deadlock in a 11 × 11 torus while it needs nine hours for a 12 × 12 network.

Link-Disjoint Broadcasting Algorithm in Wormhole-Routed 3D Petersen-Torus Networks

In a network, one-to-all broadcasting is the process of disseminating messages from a source node to all the nodes existing in the network through successive data transmissions between pairs of nodes. Broadcasting is the most primary communication process in a network. A 3D Petersen-Torus (3D PT) network has been proposed recently. the three-dimensionally magnified Petersen-Torus topology, 3D PT, is suitable for establishing a wireless sensor network (WSN) in 3D rectangular or cylindrical structures such as buildings. We propose a link-disjoint broadcasting algorithm for half-duplex 3D PT networks with wormhole routing and prove that the broadcasting algorithm is link-disjoint and the broadcasting step is [Formula: see text] in 3D PT[Formula: see text].

An accurate performance model for network-on-chip and multicomputer interconnection networks

In this paper, we present a mathematical background for a new approach for performances modeling of interconnection networks, based on analyzing the packet blocking and waiting time spent in each channel passing through all possible paths in the channel dependency graph. We have proposed a new, simple and very accurate analytical model for deterministic routing in wormhole networks, which is general in terms of the network topology and traffic distribution. An accurate calculation of the variance of the service time has been developed, which overcomes the rough approximation used, as a rule, in the existing models. The model supports two-dimensional mesh topologies, widely used in network-on-chip architectures, and multidimensional topologies, popular in multicomputer architectures. It is applicable even for irregular topologies and arbitrary application-specific traffic. Results obtained through simulation show that the model achieves a high degree of accuracy.

A parallel implementation of Strassen’s matrix multiplication algorithm for wormhole-routed all-port 2D torus networks

A new parallel implementation of Strassen's matrix multiplication algorithm is proposed for massively parallel supercomputers with 2D, all-port torus interconnection networks. The proposed algorith...

On Necessary and Sufficient Conditions for Deadlock-Free Routing in Wormhole Networks

Wormhole switching is a popular switching technique in interconnection networks. This technique is also prone to deadlocks. Adaptive routing algorithms provide alternative paths that can be used to escape congested areas and prevent some deadlocks to occur. If not designed carefully, these new paths may as well introduce deadlocks. A successful solution to deadlock prevention is to constrain the routing function such that it does not introduce any deadlock. Many necessary and sufficient conditions for deadlock-free routing have been proposed. The definition and the proof of these conditions are complex and error-prone. These conditions are often counterintuitive and difficult to understand. Moreover, they are not static, as they all require the analysis of configurations, i.e., the network state. The contribution of this paper is twofold. We present the first static necessary and sufficient condition for deadlock-free routing in wormhole networks. Our condition is much simpler and requires less assumptions than all previous ones. It is formally proven correct using an automated proof assistant. In particular, our condition applies to incoherent routing functions which was considered an open problem. Second, we prove the deadlock decision problem co-NP-complete for wormhole networks.

A Comment on “A Necessary and Sufficient Condition for Deadlock-Free Adaptive Routing in Wormhole Networks”

The purpose of this comment is to show that Duato's condition for deadlock freedom is only sufficient and not necessary. We propose a fix to keep the condition necessary. The issue is subtle but essential: in a wormhole network worms necessarily do not intersect.

Design and Analysis of On-Chip Networks for Large-Scale Cache Systems

Switched networks have been adopted in on-chip communication for their scalability and efficient resource sharing. However, using a general network for a specific domain may result in unnecessary high cost and low performance when the interconnects are not optimized for the domain. Designing an optimal network for the specific domain is challenging because in-depth knowledge of interconnects and the application domain is required. Recently proposed Nonuniform Cache Architectures (NUCAs) use wormhole-routed 2D mesh networks in L2 caches. We observe that in NUCAs, network resources are underutilized with the considerable area cost (41 percent of cache) and the network delay is significantly large (63 percent of cache access time). Motivated by our observations, we investigate both router architecture and network topology for communication behaviors in large-scale cache systems. We present Fast-LRU replacement, where cache replacement overlaps with data request delivery. Next, we propose a deadlock-free XYX routing algorithm in a mesh network and present a new halo network topology to reduce the required links. Finally, we introduce a single-cycle multicast router that needs small modification of the unicast router design. Simulation results show that our design improves the average IPC by 38 percent over the mesh design with Multicast Promotion replacement and uses 12 percent of the interconnection area of the mesh network.

Communication modeling of multicast in all-port wormhole-routed NoCs

Multicast is one of the most frequently used collective communication operations in multi-core SoC platforms. Bus as the traditional interconnect architecture for SoC development has been highly efficient in delivering multicast messages. Since the bus is non-scalable, it can not address the bandwidth requirements of the large SoCs. The networks on-chip (NoCs) emerged as a scalable alternative to address the increasing communication demands of such systems. However, due to its hop-to-hop communication, the NoCs may not be able to deliver multicast operations as efficiently as buses do. Adopting multi-port routers has been an approach to improve the performance of the multicast operations in interconnection networks. This paper presents a novel analytical model to compute communication latency of the multicast operation in wormhole-routed interconnection networks employing asynchronous multi-port routers scheme. The model is applied to the Quarc NoC and its validity is verified by comparing the model predictions against the results obtained from a discrete-event simulator developed using OMNET++.

Multicast communication in wormhole-routed 2D torus networks with hamiltonian cycle model

In this paper, we propose an efficient multipath multicast routing algorithm in wormhole-routed 2D torus networks. We first introduce a hamiltonian cycle model for exploiting the feature of torus networks. Based on this model, we find a hamiltonian cycle in torus networks. Then, an efficient multipath multicast routing algorithm with hamiltonian cycle model (mulitpath-HCM) is presented. The proposed multipath multicast routing algorithm utilizes communication channels more uniformly in order to reduce the path length of the routing messages, making the multicasting more efficient. Simulation results show that the multicast latency of the proposed multipath-HCM routing algorithm is superior to that of fixed and dual-path routing algorithms.

Effects of Adaptive Wormhole Routing in Event Builder Networks

The data acquisition system of the CMS experiment at the Large Hadron Collider features a two-stage event builder, which combines data from about 500 sources into full events at an aggregate throughput of 100 GB/s. To meet the requirements, several architectures and interconnect technologies have been quantitatively evaluated. Myrinet will be used for the communication from the underground frontend devices to the surface event building system. Gigabit Ethernet is deployed in the surface event building system. Nearly full bi-section throughput can be obtained using a custom software driver for Myrinet based on barrel shifter traffic shaping. This paper discusses the use of Myrinet dual-port network interface cards supporting channel bonding to achieve virtual 5 GBit/s links with adaptive routing to alleviate the throughput limitations associated with wormhole routing. Adaptive routing is not expected to be suitable for high-throughput event builder applications in high-energy physics. To corroborate this claim, results from the CMS event builder preseries installation at CERN are presented and the problems of wormhole routing networks are discussed.

An accurate performance model of fully adaptive routing in wormhole-switched two-dimensional mesh multicomputers

Numerous analytical performance models have been proposed for deterministic wormhole-routed mesh networks while only a single model, to our best knowledge, has been suggested for fully adaptive wormhole routing in mesh interconnection networks. This paper proposes a new and accurate analytical performance model for fully adaptive wormhole-routed mesh networks. Simulation results show that unlike the previously proposed model which is only accurate in light traffic loads, this model can be used for the performance analysis of almost all traffic loads.

Multicast communication in OTIS-hypercube multi-computer systems

The ability to perform multicast communication efficiently in parallel computing systems is increasingly becoming an essential feature of these systems. Therefore, any emerging parallel architecture must be able to provide effective multicasting performance. OTIS-based multi-computer systems are parallel systems that efficiently exploit optical connections between distant nodes, while using electronic connections for physically close processors. No previous work has attempted to determine an optimal multicast communication scheme for OTIS networks. This paper presents two approaches to implement efficient multicast communication in wormhole-routed OTIS networks, for the two different possible unicast routing schemes of these networks, in the absence of hardware multicast support. A simulation-based evaluation of the different multicast techniques is presented. The performance of the two proposed multicasting schemes are also compared under different working conditions.

An Efficient Tree-Based Multicasting Algorithm on Wormhole-Routed Star Graph Interconnection Networks Embedded with Hamiltonian Path

Multicasting is an important issue for numerous applications in parallel and distributed computing. In multicasting, the same message is delivered from a source node to an arbitrary number of destination nodes. The star graph interconnection network has been recognized as an attractive alternative to the popular hypercube network. In this paper, we propose an efficient and deadlock-free tree-based multi-cast routing scheme for wormhole-routed star graph networks with hamiltonian path. In our proposed routing scheme, the router is with the input-buffer-based asynchronous replication mechanism that requires extra hardware cost. Meanwhile, the router simultaneously sends incoming flits on more than one outgoing channel. We perform simulation experiments with the network latency and the network traffic. Experimental results show that the proposed scheme reduces multicast latency more efficiently than other schemes.

A performance model of compressionless routing in k-ary n-cube networks

Several analytical models of fully adaptive routing in wormhole-routed networks have recently been reported in the literature. All these models, however, have been discussed for routing algorithms with deadlock avoidance. Recent studies have revealed that deadlocks are quite rare in the network, especially when enough routing freedom is provided. Thus, the hardware resources, e.g. virtual channels, dedicated for deadlock avoidance are not utilised most of the time. This consideration has motivated researchers to introduce fully adaptive routing algorithms with deadlock-recovery. This paper proposes a new analytical model to predict message latency in k-ary n-cubes with compressionless routing, a fully adaptive algorithm that uses deadlock-recovery. The proposed model uses results from queueing systems with impatient customers to capture the effects of the timeout mechanism used in this routing algorithm to deal with message deadlock. The validity of the model is demonstrated by comparing results predicted by the analytical model against those obtained through simulation experiments.

Multipath-Based Multicasting Strategies for Wormhole-Routed Star Graph Interconnection Networks

The star graph interconnection network has been recognized as an attractive alternative to the popular hypercube network. In this paper, we present a multipath-based multicast routing model for wormholerouted star graph networks, propose two efficient multipath routing schemes, and contrast the performance of the proposed schemes with the performance of the scheme presented in our previous work. Both of the two proposed schemes have been proven to be deadlock-free. The first scheme, simple multipath routing, uses multiple independent paths for concurrent multicasting. The second scheme, two-phase multipath routing, includes two phases: source-to-relay and relay-to-destination. For each phase, multicasting is carried out using simple multipath routing. Experimental results show that, for short and medium messages with small message startup latencies, the proposed schemes reduce multicast latency more efficiently than other schemes.

A Broadcast Algorithm for All-port Wormhole-routed Mesh-hypercube Network

A Broadcast Algorithm for All-port Wormhole-routed Mesh-hypercube Network

Modeling Latency in Deterministic Wormhole-Routed Hypercubes under Hot-Spot Traffic

Analytical models of deterministic routing in common wormhole-routed networks, such as the hypercube, have been widely reported in the literature. However, all these models have been discussed for the uniform traffic pattern. The performance of deterministic routing under other important non-uniform communication patterns, such as hot-spots, has often been analyzed through simulation. The main advantage of the analytical approach over simulation is that the analytical models can be used to obtain performance results for large systems that are infeasible by simulation due to the excessive computation demands on conventional computers. This paper presents the first analytical model of deterministic routing in the hypercube in the presence of hot-spot traffic. Simulation results confirm that the proposed model predicts message latency with a reasonable degree of accuracy under different traffic conditions.

Pseudo-cycle-based multicast routing in wormhole-routed networks

This paper addresses the problem of fault-tolerant multicast routing in wormhole-routed multicomputers. A new pseudo-cycle-based routing method is presented for constructing deadlock-free multicast routing algorithms. With at most two virtual channels this technique can be applied to any connected networks with arbitrary topologies. Simulation results show that this technique results in negligible performance degradation even in the presence of a large number of faulty nodes.

Analysis of true fully adaptive routing with software-based deadlock recovery

Several analytical models of fully adaptive routing (AR) in wormhole-routed networks have recently been reported in the literature. All these models, however, have been discussed for routing algorithms with deadlock avoidance. Recent studies have revealed that deadlocks are quite rare in the network, especially when enough routing freedom is provided. Thus the hardware resources, e.g. virtual channels, dedicated for deadlock avoidance are not utilised most of the time. This consideration has motivated researchers to introduce fully adaptive routing algorithms with deadlock recovery. This paper describes a new analytical model of a true fully AR algorithm with software-based deadlock recovery, proposed in Martinez et al. (Software-based deadlock recovery techniques for true fully AR in wormhole networks, Proc. Int. Conf. Parallel Processing (ICPP’97), 1997, p. 182) for k-ary n-cubes. The proposed model uses the results from queueing systems with impatient customers to capture the effects of the timeout mechanism used in this routing algorithm for deadlock detection. Results obtained through simulation experiments confirm that the model predicts message latency with a good degree of accuracy under different working conditions.