Parallel Sorting Algorithms Research Articles

Contemplate sorting with columnsort

The efficiency of Internet search engines has made it trivial for students to find implementations of standard algorithms. This fact has led computer science educators to be more creative with their assignments to encourage students to create their own implementations. Unfortunately, excessive creativity can obscure learning objectives, particularly for less insightful students. We demonstrate that recasting the parallel sorting algorithm Columnsort for a uniprocessor environment provides the foundation for a variety of sorting assignments that can engage students while not obscuring educational objectives.

High-speed parallel external sorting of data with arbitrary distribution

Many parallel sorting algorithms of external disk data have been reported such as NOW-sort, SPsort, hill sort and so on. They all reduce the execution time compared with some known sequential sort; however, they differ in terms of the speed, throughput or cost-effectiveness. Mostly, they deal with uniformly distributed data in their value range. If we divide and redistribute data to processors by fixed and equal division of the key range, all processors will have about equal numbers of keys to sort and store. But if irregularly distributed data are given, the performance will suffer severely as the partitioning would no longer produce balanced loads among processors. Few research results have been reported for parallel external sort of data with arbitrary distribution. In this paper, we develop two distribution-insensitive scalable parallel external sorting algorithms that use sampling technique and histogram counts to achieve even distribution of keys, which eventually contribute to achieve good performance. Experimental results on a cluster of 16 Linux workstations show up to threefold enhancement of the performance compared with NOW-sort for sorting 16 GB integer keys.

Sensitivity Analysis of Database Operations: A Case Study of Parallel Partition/Sorting Algorithm

Many data sets follow certain distribution patterns, such as uniform distribution, normal distribution, and so on. Some algorithms for database operations make use of this distribution knowledge, and they usually perform much better than other algorithms, particularly in parallel computers. However, the distribution characteristics of certain data sets can change from time to time in real-life situations and affect the performance of these algorithms. It is thus important to know the behaviour of these algorithms under imperfect situations. This article presents the sensitivity analysis of a statistical algorithm for database operations when data sets do not follow the assumed distribution pattern. A parallel sorting algorithm designed by the author is used as a case study. This algorithm is for parallel computers with multiple instruction streams, multiple data streams architecture with shared memory.

Parallel Self-Index Integer Sorting

We consider the problem of sorting n integers when the elements are drawn from the restricted domain [1…n]. A new deterministic parallel algorithm for sorting n integers is obtained. Its running time is O(lognlog(n/logn)) using n/logn processors on EREW (exclusive read exclusive write) PRAM (parallel random access machine). Also, our algorithm was modified to become optimal when we use \sqrt{n} processors. This algorithm belongs to class EP (Efficient, Polynomial fast).

On a scheme for parallel sorting on heterogeneous clusters

We discuss parallel sorting algorithms and their implementations suitable for cluster architectures in order to optimize cluster resources. We focus on the time spent in computation and the load balancing properties when processors are running at different speeds, i.e. correlated by a multiplicative constant factor (our weak definition of heterogeneous platform). One scheme is under study: parallel sorting by sampling (either regular sampling technique introduced by Shi and Schaeffer [J. Parallel Distrib. Comput. 14 (4) (1992) 361] or the over-partitioning scheme introduced by Li and Sevcik [Parallel sorting by over-partitioning, in: Proceedings of the Sixth Annual Symposium on Parallel Algorithms and Architectures, ACM Press, New York, June 1994]). What is important in the paper is mainly the load balance factor and not necessary the execution time. It is clear that improved load balance leads to improved execution time. The results presented in the paper demonstrate that load balancing for the case of computers with heterogeneous processing capacity is more challenging than for the homogeneous case. The survey, through the sorting case study, allow us to identify some algorithmic issues and software challenges to master heterogeneous cluster platforms in order to better utilize them: data decomposition techniques, scheduling and load balancing methods.

A Framework for Simple Sorting Algorithms on Parallel Disk Systems

In this paper we present a simple parallel sorting algorithm and illustrate its application in general sorting, disk sorting, and hypercube sorting. The algorithm (called the (l,m) -mergesort (LMM)) is an extension of the bitonic and odd—even mergesorts. Literature on parallel sorting is abundant. Many of the algorithms proposed, though being theoretically important, may not perform satisfactorily in practice owing to large constants in their time bounds. The algorithm presented in this paper has the potential of being practical. We present an application to the parallel disk sorting problem. The algorithm is asymptotically optimal (assuming that N is a polynomial in M , where N is the number of records to be sorted and M is the internal memory size). The underlying constant is very small. This algorithm performs better than the disk-striped mergesort (DSM) algorithm when the number of disks is large. Our implementation is as simple as that of DSM (requiring no fancy data structures or prefetch techniques.) As a second application, we prove that we can get a sparse enumeration sort on the hypercube that is simpler than that of the classical algorithm of Nassimi and Sahni [16]. We also show that Leighton's columnsort algorithm is a special case of LMM.

A 2-D parallel convex hull algorithm with optimal communication phases

We investigate the problem of finding the 2-D convex hull of a set of points on a coarse-grained parallel computer. Goodrich has devised a parallel sorting algorithm for n items on P processors which achieves an optimal number of communication phases for all ranges of P⩽ n. Ferreira et al. have recently introduced a deterministic convex hull algorithm with a constant number of communication phases for n and P satisfying n⩾ P 1+ ϵ . Here we present a new parallel 2-D convex hull algorithm with an optimal bound on number of communication phases for all values of P⩽ n while maintaining optimal local computation time.

Parallel Sorting Algorithm Using Multiway Merge and Its Implementation on a Multi-Mesh Network

In this paper, we present a parallel sorting algorithm using the technique of multi-way merge. This algorithm, when implemented on a t dimensional mesh having nt nodes (t>2), sorts nt elements in O((t2−3t+2)n) time, thus offering a better order of time complexity than the [((t2−t)nlogn)/2+O(nt)]-time algorithm of P. F. Corbett and I. D. Scherson (1992, IEEE Trans. Parallel Distrib. Systems3, 626–632). Further, the proposed algorithm can also be implemented on a Multi-Mesh network (1999, D. Das, M. De, and B. P. Sinha, IEEE Trans. Comput.48, 536–551) to sort N elements in 54N1/4+o(N1/4) steps, which shows an improvement over 58N1/4+o(N1/4) steps needed by the algorithm in (1997, M. De, D. Das, M. Ghosh, and P. B. Sinha, IEEE Trans. Comput.46, 1132–1137).

An Efficient Parallel VLSI Sorting Architecture

We present a new parallel sorting algorithm that uses a fixed-size sorter iteratively to sort inputs of arbitrary size. A parallel sorting architecture based on this algorithm is proposed. This architecture consists of three components, linear arrays that support constant-time operations, a multilevel sorting network, and a termination detection tree, all operating concurrently in systolic processing fashion. The structure of this sorting architecture is simple and regular, highly suitable for VLSI realization. Theoretical analysis and experimental data indicate that the performance of this architecture is likely to be excellent in practice.

Parallel Sorting with Limited Bandwidth

We study the problem of sorting on a parallel computer with limited communication bandwidth. By using the PRAM(m) model, where p processors communicate through a globally shared memory which can service m requests per unit time, we focus on the trade-off between the amount of local computation and the amount of interprocessor communication required for parallel sorting algorithms. Our main result is a lower bound of $\Omega(\frac{n \log m}{m \log n})$ on the time required to sort n numbers on the exclusive-read and queued-read variants of the PRAM(m). We also show that Leighton's Columnsort can be used to give an asymptotically matching upper bound in the case where m grows as a fractional power of n. The bounds are of a surprising form in that they have little dependence on the parameter p. This implies that attempting to distribute the workload across more processors while holding the problem size and the size of the shared memory fixed will not improve the optimal running time of sorting in this model. We also show that both the lower and the upper bounds can be adapted to bridging models that address the issue of limited communication bandwidth: the LogP model and the bulk-synchronous parallel (BSP) model. The lower bounds provide further convincing evidence that efficient parallel algorithms for sorting rely strongly on high communication bandwidth.

Communication-Efficient Sorting Algorithms on Reconfigurable Array of Processors With Slotted Optical Buses

The reconfigurable array with slotted optical buses (RASOB) has recently received a lot of attention from the research community. In this paper, we first discuss the reconfiguration methods and communication capabilities of the RASOB architecture. Then, we use this architecture for the implementation of efficient sorting algorithms on the 1D RASOB and the 2D RASOB. Our parallel sorting algorithm on the 1D RASOB is based on an efficient divide-and-conquer scheme. It sortsNdata items usingNprocessors inO(k) communication cycles where k is the size of the data items to be sorted in bits. We further develop a parallel sorting algorithm on the 2D RASOB based on the sorting algorithm on the 1D RASOB in conjunction with the well known Rotatesort algorithm. Similarly, this algorithm sortsNdata items on a 2D RASOB of sizeNinO(k) communication cycles. These sorting algorithms are much more efficient than state-of-the-art sorting algorithms on reconfigurable arrays of processors withelectronicbuses using the same number of processors.

An Efficient General In-Place Parallel Sorting Scheme

We present a simple and general parallel sorting scheme, ZZ-sort, which can be used to derive a class of efficient in-place sorting algorithms on realistic parallel machine models. We prove a tight bound for the worst case performance of ZZ-sort. We also demonstrate the average performance of ZZ-sort by experimental results obtained on a MasPar parallel computer. Our experiments indicate that ZZ-sort can be incorporated into a distributed memory parallel computer system as a standard routine, and this routine is useful for space critical situations. Finally, we show that ZZ-sort can be used to convert a non-adaptive parallel sorting algorithm into an in-place and adaptive one by considering the problem of sorting an arbitrarily large input on fixed-size reconfigurable meshes.

A new deterministic parallel sorting algorithm with an experimental evaluation

We introduce a new deterministic parallel sorting algorithm for distributed memory machines based on the regular sampling approach. The algorithm uses only two rounds of regular all-to-all personalized communication in a scheme that yields very good load balancing with virtually no overhead. Moreover, unlike previous variations, our algorithm efficiently handles the presence of duplicate values without the overhead of tagging each element with a unique identifier. This algorithm was implemented in SPLIT-C and run on a variety of platforms, including the Thinking Machines CM-5, the IBM SP-2-WN, and the Cray Research T3D. We ran our code using widely different benchmarks to examine the dependence of our algorithm on the input distribution. Our experimental results illustrate the efficiency and scalability of our algorithm across different platforms. In fact, the performance compares closely to that of our random sample sort algorithm, which seems to outperform all similar algorithms known to the authors on these platforms. Together, their performance is nearly invariant over the set of input distributions, unlike previous efficient algorithms. However, unlike our randomized sorting algorithm, the performance and memory requirements of our regular sorting algorithm can be deterministically guaranteed.

A Randomized Parallel Sorting Algorithm with an Experimental Study

Previous schemes for sorting on general-purpose parallel machines have had to choose between poor load balancing and irregular communication or multiple rounds of all-to-all personalized communication. In this paper, we introduce a novel variation on sample sort which uses only two rounds of regular all-to-all personalized communication in a scheme that yields very good load balancing with virtually no overhead. Moreover, unlike previous variations, our algorithm efficiently handles the presence of duplicate values without the overhead of tagging each element with a unique identifier. This algorithm was implemented in Split-C and run on a variety of platforms, including the Thinking Machines CM-5, the IBM SP-2, and the Cray Research T3D. We ran our code using widely different benchmarks to examine the dependence of our algorithm on the input distribution. Our experimental results illustrate the efficiency and scalability of our algorithm across different platforms. In fact, it seems to outperform all similar algorithms known to the authors on these platforms, and its performance is invariant over the set of input distributions unlike previous efficient algorithms. Our results also compare favorably with those reported for the simpler ranking problem posed by the NAS Integer Sorting (IS) Benchmark.

Sorting in Parallel

Although more than a century passed before Menabrea's vision became a reality, today parallel computers with hundreds and even thousands of processors are used in a broad range of applications. One of the great challenges of parallel computing is the design of algorithms that make efficient use of the large number of processors in the machine. In this paper we explore parallel algorithms for one of the most fundamental problems in computer science: the problem of sorting a list of numbers. Specifically, given a list a1, a2, ... , an of integers, the sorting problem is that of finding a permutation of this list, 7T, such that a.(l) < a.(2) < ...< a(n) Before considering parallel sorting algorithms, we examine a sequential sorting algorithm; an algorithm for a computer with a single processor. One sequential algorithm for sorting is known as selection sort. The selection sort algorithm first examines each of the n integers a1, ... , an to find an integer ai with minimum value. Elements a1 and ai are then exchanged so that the element with minimum value is now in the first position in the list. Next, the algorithm examines each of the n 1 integers a2, ... ., an to find an integer with minimum value in this portion of the list. This integer is exchanged with a2 so that now the second smallest integer is in the second position in the list. In general, in the k th iteration the algorithm examines integers ak, .. , an, finds a minimum element in this list, and exchanges it with ak. At the end of nth iteration, the list is sorted. Observe that the first iteration of the algorithm requires n steps to find an element with minimum value and then some constant additional number of steps, c, to exchange this minimum value with a1. The next iteration requires n 1 steps to locate a minimum value in the remaining list and c steps to perform the exchange. In general, the kth iteration requires n k + 1 steps to locate a minimum value in the remaining list and c steps to perform the exchange. Since there are n iterations, the total number of steps or running time of the algorithm is

An Experimental Analysis of Parallel Sorting Algorithms

We have developed a methodology for predicting the performance of parallel algorithms on real parallel machines. The methodology consists of two steps. First, we characterize a machine by enumerating the primitive operations that it is capable of performing along with the cost of each operation. Next, we analyze an algorithm by making a precise count of the number of times the algorithm performs each type of operation. We have used this methodology to evaluate many of the parallel sorting algorithms proposed in the literature. Of these, we selected the three most promising, Batcher's bitonic sort, a parallel radix sort, and a sample sort similar to Reif and Valiant's flashsort, and implemented them on the connection Machine model CM-2. This paper analyzes the three algorithms in detail and discusses the issues that led us to our particular implementations. On the CM-2 the predicted performance of the algorithms closely matches the observed performance, and hence our methodology can be used to tune the algorithms for optimal performance. Although our programs were designed for the CM-2, our conclusions about the merits of the three algorithms apply to other parallel machines as well.

Sorting by parallel insertion on a one-dimensional subbus array

We consider the problem of sorting on a one-dimensional subbus array of processors, an architecture that communicates using a segmentable bus. The subbus broadcast operation makes possible a new class of parallel sorting algorithms whose complexity we analyze with the parallel insertion model. We give per-input lower bounds for sorting in the parallel insertion model and demonstrate sorting strategies that are optimal by matching those lower bounds. For each of our sorting strategies, we discuss the issues involved in implementing them on subbus machines. Finally, we empirically evaluate the performance of our sorting strategies by applying them to shearsort, a common two-dimensional mesh sorting algorithm. Our results suggest that for sorting the subbus broadcast capability gives at most a slight advantage over using only nearest neighbor communication.

A novel sorting algorithm and its application to a gamma-ray telescope asynchronous data acquisition system

In this paper we present a novel parallel sorting algorithm, which works through a cascade of elementary sorting units and leads to a scalable architecture. The algorithm's complexity is analyzed and compared with a classical parallel algorithm. It comes out that, although it may be less efficient than classical approaches, the proposed algorithm is highly suited for VLSI implementation for its simplicity and scalability. The paper describes the applications of such device to the asynchronous data acquisition for a gamma-ray telescope.

Fast Parallel Radix Sort Using a Reconfigurable Mesh

In this paper, we present a parallel SIMD algorithm for radix sorting of N numbers of w bits each, taking O(w + N1/4) time with the VLSI area of O(N3/2 w2), 0 < w < N1/4. For w = log N, our algorithm improves a previous known solution on a similar architecture in time complexity by a factor of log N. Since our algorithm uses only radix sort for sorting of subsets and merging of them, no comparator is needed. Our algorithm satisfies the lower bound of AT2 complexity which mainly restricts the VLSI implementation of most sorting algorithms. The same result is obtained in another previously known solution, but it requires a comparator of size w.

A PRACTICAL ALGORITHM for INTEGER SORTING INTEGER SORTING ON A MESH-CONNECTED COMPUTER*

This paper presents count-sort, a parallel algorithm for mesh-connected computers to sort integers where the range of inputs is known. A straightforward counting technique that has not been implemented previously in parallel sorting algorithms is presented. On a mesh-connected computer with N×N processors we are able to sort N integers in the range 1 sN in timewhere cN is very small. For practical values of N, the algorithm is extremely fast. Further, it is possible to expand the range by a factor k to 1 N so that the slowdown is less than k. We produce two implementations of count-sort on the SIMD MasPar MP-I with 8192 processors. The first sorts 8-bit integers, one per processor, significantly faster than the manufacturer's current library routine for sorting 8-bit integers. The second implementation is a fast version that sorts several elements per processor.