Optimal Communication Algorithms for Heterogeneous Computing over ATM Networks

Abstract

We present efficient algorithms for designing optimal collective communication primitives for cluster-based heterogeneous computing across Asynchronous Transfer Mode (ATM) networks. The virtual path (VP) concept is known to be a powerful transport mechanism for ATM networks. In many parallel processing applications, the potential communication pattern is known in advance. We then seek to find an optimal set of virtual paths to realize the communication pattern, such that the communication overhead of the parallel processing application is minimized, and at the same time, the extent to which the other data traffic in the network is affected by the data traffic generated by the parallel processing application is also minimized.We consider the following four collective communication problems that occur frequently in parallel computation:one-to-all broadcast,all-to-all broadcast,one-to-all scatter, andall-to-all scatter. We formulate these problems as combinatorial optimization problems of finding a system of virtual path routes in networks, with the objectives of minimizing the path length, the link congestion, and the node copy load. We consider two cases of data sources, namely homogeneous and heterogeneous. For the case ofhomogeneous sources, we presentpolynomial-timealgorithms for the problems of one-to-all broadcast, all-to-all broadcast, and one-to-all scatter, by converting these problems to the maximum-flow problems. For the case ofheterogeneous sources, we show that the problems of all-to-all broadcast, one-to-all scatter, and all-to-all scatter are allNP-complete. Finally, we present novelrandomizedalgorithms for these problems, and show that with high probability these algorithms yield good solutions in the sense that the objective functions yield values close to the optimums. The randomized algorithms can also be used to design near-optimummulticastcommunication primitives. These randomized algorithms make use of linear programming and are easy to implement, which makes them a potential candidate for practical implementations, even for very large networks.