Parallel vector reduction algorithms and architectures

Abstract

Vector operations are important in many computer applications. They often represent the main part of operations of the entire problem and consume a great amount of computing time. So, it is natural to apply parallel computation to vector operations in order to increase the speed of solving a problem. Among vector operations, vector reduction is a known and common type of operation (e.g., vector summation, inner product evaluation). In this paper vector reduction techniques for parallel pipelined processing are discussed. The computation and communication properties and constraints of both single and multiple vector reductions in a multipipeline environment are considered. From this a simple, yet efficient “partitioned linear pipeline array” (PLPA) architecture is proposed and the performance of a number of scheduling algorithms related to this architecture is determined. The performance comparison between the proposed approach and the well-known tree-structured reduction processor is given. From the results of performance analysis, it is shown that the PLPA approach has approximately the same performance as a pipelined binary reduction tree. However, the PLPA approach is much simpler and easier to implement, and is also more flexible than a tree-structured reduction processor. Finally, as an example, the matrix multiplication operation on a PLPA is considered. It is shown that with the PLPA architecture a very good performance can be obtained.