A practical external sort for shared disk MPPs

Abstract

An external sort has been implemented and analyzed for a shared disk MPP computer system. In this implementation, we have considered many real world constraints. Decision support functionality in database systems, for instance, often requires that external sorting be done in place on disk, support variable length records, and be restartable from any point of interruption with no loss of data. These three constraints, along with the more standard requirements of speed and stability, affect the choice and implementation of the external sorting algorithm. The implementation of the sample sort algorithm described here meets these requirements. Although written using high level file processing directives, the implementation sorts a 10 GB file in 1.5 h on a 64 processor Connection Machine CM-5 with a DataVault disk system.