Fast generation of long sorted runs for sorting a large file

Abstract

On sequential machines, most internal sorting algorithms can sort no more than m items using a memory of size m. However, sorting with a heap can produce sorted sequences, called runs, of length about twice the heap size. A second advantage of sorting with a heap is that data I/O and the heap restructuring can be performed concurrently to reduce the sorting time. The third advantage is that it can even sort an arbitrarily large file in one pass if the file satisfies a certain ordering condition. The authors present an algorithm running on a linear array to obtain the above advantages. Specifically, the algorithm can produce runs of length about 2(m+2)p on a linear array of p PEs each with a heap of size m, and can overlap I/O with internal operations. In addition, it can completely sort an arbitrarily large file in one pass providing no item in the file has (m+2)p larger items before it. The algorithm can be modified to perform (mp+2p)-way merges. The authors have also estimated the optimal heap size in each PE for the minimum sorting time. >