Abstract
In high-performance computing on distributed-memory systems, communication often represents a significant part of the overall execution time. The relative cost of communication will certainly continue to rise as compute-density growth follows the current technology and industry trends. Design of lower-communication alternatives to fundamental computational algorithms has become an important field of research. For distributed 1-D FFT, communication cost has hitherto remained high as all industry-standard implementations perform three all-to-all internode data exchanges (also called global transposes). These communication steps indeed dominate execution time. In this paper, we present a mathematical framework from which many single-all-to-all and easy-to-implement 1-D FFT algorithms can be derived. For large-scale problems, our implementation can be twice as fast as leading FFT libraries on state-of-the-art computer clusters. Moreover, our framework allows tradeoff between accuracy and performance, further boosting performance if reduced accuracy is acceptable.
Highlights
IntroductionThe works in [25,27], for example, do not count the communication cost incurred when each processor accesses the entire input data or reorders out-of-order results back into natural order
There are many FFT algorithms, they all factor the Discrete Fourier Transform (DFT) matrix algebraically into sparse factors, thereby reducing an O(N 2) arithmetic cost to that of O(N log N ). This arithmetic cost reduction has been instrumental in many advances in computing since the Cooley–Tukey paper appeared
While arithmetic cost reduction has long been a paradigm in computer science, a new paradigm is emerging as we enter a new era where raw arithmetic speed reaches a teraflop on a single die and parallelism in the form of multicore and multinode is prevalent
Summary
The works in [25,27], for example, do not count the communication cost incurred when each processor accesses the entire input data or reorders out-of-order results back into natural order They would require O(N 3/2) computation cost as opposed to O(N log N ). Task computes P sets of length-M FFTs: As depicted above, this decomposition fundamentally requires three all-to-all steps if data order is to be preserved. Recursive applications of this (or variants thereof) decomposition leads to an O(N log N ) arithmetic complexity, but cannot undo the triple-all-to-all requirement. While internode communication is required during convolution, that amount is negligible as each node merely needs an insignificant amount of data from its nextdoor neighbor
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
