Performance enhancement methods for parallel programs with multilevel memory

Abstract

Modern multiprocessor systems designed for large-scale problems mostly have hierarchical memory subsystems divided into several levels of different speed and capacity. Hierarchical memory organization is dictated, on the one hand, by the large volumes of input data, which must be stored in a large mass storage, and on the other hand, by the need to achieve high speeds, which is possible only when the current data are stored in fast memory is very expensive, economic considerations suggest a hierarchical organization of memory, with lower levels characterized by large capacity and low speed and upper levels conversely characterized by high speed and small capacity. This has led to a whole direction of research associated with NUMA (Non-Uniform Memory Accesses) architectures, where the main focus is on organization of optimal interaction between memory levels with the object of maximizing overall productivity of the parallel system. This problem is becoming progressively more acute, as according to some data processor performance increases at a rate of 50% - 100% annually, while memory speeds increase much more slowly, at rate of 10% annually. Among various approaches to the solution of this problem, we can mention OS-level software methods and programming systems, and also the methodmore » of formal program transformations at the source program level. The formal program transformation method has clear advantages compared to the other methods: it is inexpensive and produces the highest effect because of being targeted to specific classes of parallel programs. The application of the program transformation approach to the design of efficient parallel macropipelined programs with multilevel memory organization is the subject of our paper.« less