Optimisation of Communication Complexity in Parallel Computing

Abstract

Parallel principles have been for a long time the most effective way how to increase performance in parallel computing (parallel computers, parallel algorithms). Dominant parallel computers are based on network of workstations (NOW) and on high integrated network of NOW modules (Grid). Effective using of such parallel computers assumes minimisation of at least substantial overheads representing with overhead function h(s, p). To the most important overhead latency belongs communication latency representing by inter process communication (IPC) of decomposed parallel processes. In this sense the paper is devoted to the modelling of communication complexity in unified parallel and distributed computing. This overhead function, representing by communication overhead, is then an integral part of the complex parallel execution time. Based on this we are able better to optimise PA during the developing process of PA and to come to final effective (optimised) PA. The article analysis such parts in PA developing process which are critical to communication complexity and that first the critical role of problem parallelisation (decomposition strategy) and second at own performance PA optimisation (shared memory, distributed memory, hybrid). Based on analysed optimised examples of PA the paper illustrates these critical parts from the point of user. In similar way the article point to concept of complex isoefficiency function to asymptotic performance predictions of given complex problems on given parallel computer in order to illustrate critical role of decomposition strategy and needed optimisation of communication complexity to developed effective PA.