N latency 2N I/O‐bandwidth 2D‐array matrix multiplication algorithm

Abstract

The emergence of the systolic paradigm in 1978 inspired the first 2D‐array parallelization of the sequential matrix multiplication algorithm. Since then, and due to its attractive and appealing features, systolic approach has been gaining great momentum to the point where all 2D‐array parallelization attempts were exclusively systolic. As good result, latency has been successively reduced a number of times (5N, 3N, 2N, 3N/2), where N is the matrix size. But as latency was getting lower, further irregularities were introduced into the array, making the implementation severely compromised either at VLSI level or at system level. The best illustrative case of such irregularities are the two designs proposed by Tsay and Chang in 1995 and considered as the fastest designs (3N/2) that have been developed so far. The purpose of this paper is twofold: we first demonstrate that N+√N/2 is the minimal latency that can be achieved using the systolic approach. Afterwards, we introduce a full‐parallel 2D‐array algorithm with N latency and 2N I/O‐bandwidth. This novel algorithm is not only the fastest algorithm, but is also the most regular one too. A 3D parallel version with O(log N) latency is also presented.