Balancing virtual channel utilization for deadlock-free routing in torus networks

Abstract

Torus networks have been widely used by modern commercial supercomputers due to low node degree and linear scalability cost. Meanwhile, the ring in each dimension of a torus creates cyclic channel dependencies, which pose challenges to the design of deadlock-free routing and virtual channel allocation schemes. Existing virtual channel allocation schemes such as dateline employ two virtual channels to avoid deadlock in a ring. While the unbalanced utilization of virtual channels caused by these deadlock avoidance schemes brings lots of performance pathologies. This paper focuses on improving deadlock-free routing algorithms in torus by balancing virtual channel utilization. First, we propose BVCU, a novel virtual channel allocation scheme that can balance the utilization of the two virtual channels used to prevent deadlock. The technique is deadlock free and can achieve almost perfect balanced utilization. Second, we propose a balanced dimension-order routing (BDOR) to provide deadlock-free deterministic routing for $$n$$n-D torus by combining BVCU with dimension-order routing. Also, a balanced fully adaptive routing (BFAR) is proposed. These routing algorithms yield higher performance than existing solutions as they achieve a proper balanced utilization of virtual channels. Finally, simulation results show that the proposed methods achieve up to 16 % throughput improvement.