Comparison-based Sorting Algorithm Research Articles

2 mm: A new technique for sorting data

In computation, sorting is highly essential to access and manipulate data efficiently. High-performance sorting algorithms have always been in demand so that computers may process data more quickly. Since the computer has become a vital tool in various domains of human life, the various researchers have investigated and presented numerous sorting algorithms to sort elements of list with minimal execution time and least space. As the volume of data increases, the urgency for efficient data processing algorithms also increases. In this novel work, we present a comparison-based sorting algorithm titled “2 mm” which is a modification of the sorting algorithm Min-Max Bidirectional Parallel Selection Sort (MMBPSS). The proposed algorithm follows divide and conquers method and uses constant space complexity i.e. O(n). Although 2 mm sort differs slightly from MMBPSS sort, its computational cost is very low and it reduces more than 50% comparison cost hence presents a more efficient solution to the sorting problems in the modern era. For performance evaluation and comparison, extensive experimentation has been performed that shows the better performance of the proposed method.

Engineering In-place (Shared-memory) Sorting Algorithms

We present new sequential and parallel sorting algorithms that now represent the fastest known techniques for a wide range of input sizes, input distributions, data types, and machines. Somewhat surprisingly, part of the speed advantage is due to the additional feature of the algorithms to work in-place, i.e., they do not need a significant amount of space beyond the input array. Previously, the in-place feature often implied performance penalties. Our main algorithmic contribution is a blockwise approach to in-place data distribution that is provably cache-efficient. We also parallelize this approach taking dynamic load balancing and memory locality into account. Our new comparison-based algorithm In-place Parallel Super Scalar Samplesort ( IPS 4 o ) , combines this technique with branchless decision trees. By taking cases with many equal elements into account and by adapting the distribution degree dynamically, we obtain a highly robust algorithm that outperforms the best previous in-place parallel comparison-based sorting algorithms by almost a factor of three. That algorithm also outperforms the best comparison-based competitors regardless of whether we consider in-place or not in-place, parallel or sequential settings. Another surprising result is that IPS 4 o even outperforms the best (in-place or not in-place) integer sorting algorithms in a wide range of situations. In many of the remaining cases (often involving near-uniform input distributions, small keys, or a sequential setting), our new In-place Parallel Super Scalar Radix Sort ( IPS 2 Ra ) turns out to be the best algorithm. Claims to have the – in some sense – “best” sorting algorithm can be found in many papers which cannot all be true. Therefore, we base our conclusions on an extensive experimental study involving a large part of the cross product of 21 state-of-the-art sorting codes, 6 data types, 10 input distributions, 4 machines, 4 memory allocation strategies, and input sizes varying over 7 orders of magnitude. This confirms the claims made about the robust performance of our algorithms while revealing major performance problems in many competitors outside the concrete set of measurements reported in the associated publications. This is particularly true for integer sorting algorithms giving one reason to prefer comparison-based algorithms for robust general-purpose sorting.

A General Framework for Sorting Large Data Sets Using Independent Subarrays of Approximately Equal Length

Designing an efficient data sorting algorithm that requires less time and space complexity is essential for computer science, different engineering disciplines, data mining systems, wireless networks, and the Internet of things. This paper proposes a general low-complex data sorting framework that distinguishes the sorted or similar data, makes independent subarrays approximately in equal length, and sorts the subarrays’ data using one of the popular comparison-based sorting algorithms. Two frameworks, one for serial realization and another for parallel realization, are proposed. The time complexity analyses of the proposed framework demonstrate an improvement compared to the conventional Merge and Quick sorting algorithms. Following complexity analysis, the simulation results indicate slight improvements in the elapsed time and the number of swaps of the proposed serial Merge-based and Quick-based frameworks compared to the conventional ones for low/high variance integer/non-integer data sets, in different data sizes and the number of divisions. It is about <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(1-1.6\%)$ </tex-math></inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(3.5-4\%)$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(0.3-1.8\%)$ </tex-math></inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(2-4\%)$ </tex-math></inline-formula> improvements in the elapsed times for 1, 2, 3, and 4 divisions, respectively for small and very large data sets in Merge-based and Quick-based scenarios. Although these improvements in serial realization are minor, making independent low-variance subarrays allows the sorted components to be extracted sequentially and gradually before the end of the sorting process. Also, it proposes a general framework for parallelizing conventional sorting algorithms using non-connected (independent) or connected (dependent) multi-core structures. As the second experiment, the numerical analyses that compare the results of the parallel realization of the proposed framework to the serial one in 1, 2, 3, and 4 divisions, show a speedup factor of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(2-4)$ </tex-math></inline-formula> for small to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(2-16)$ </tex-math></inline-formula> for very large data sets. The third experiment shows the effectiveness of the proposed parallel framework to the parallel sorting based on the random-access machine model. Finally, we prove that the mean-based pivot is as efficient as the median-based and much better than the random pivot for making subarrays of approximately equal length.

Entropy conservation for comparison-based algorithms

Comparison-based algorithms are algorithms for which the execution of each operation is solely based on the outcome of a series of comparisons between elements [5]. Examples include most sorting algorithms, search algorithms, and more general algorithms such as Heapify which constructs a heap data structure from an input list [5]. Comparison-based computations can be naturally represented via the following computational model[12]: (a) model data structures as partially-ordered finite sets; (b) model data on these by topological sorts (aka linear extensions); (c) considering computation states as finite multisets of such data; (d) represent computations by their induced transformations on states.In this view, an abstract specification of a sorting algorithm has input state given by any possible permutation of a finite set of elements (represented, according to (a) and (b), by a discrete partially-ordered set together with its topological sorts given by all permutations) and output state a sorted list of elements (represented, again according to (a) and (b), by a linearly-ordered finite set with its unique topological sort).Entropy is a measure of “randomness” or “disorder.” Based on the computational model we introduce an entropy conservation result for comparison-based sorting algorithms: “quantitative order gained is proportional to positional order lost.” Intuitively, the result bears some relation to the messy office argument advocating a chaotic office where nothing is in the right place yet each item's place is known to the owner, over the case where each item is stored in the right order and yet the owner can no longer locate the items. Formally, we generalise the result to the class of data structures representable via series-parallel partial orders–a well-known computationally tractable class [6,7]. The resulting “denotational” version of entropy conservation will be extended in follow-up work to an “operational” version for a core part of our computational model.

Array sort: an adaptive sorting algorithm on multi‐thread

Sorting is the most fundamental operation in database system. There are many classical sorting algorithms and among them the most commonly-used sorting algorithm in modern database system is merge sort. Merge sort is an efficient, general-purpose, comparison-based sorting algorithm. As merge sort is based on a divide and conquer model, it has found wide use in parallel computing algorithms. In this study, the authors present an adaptive sorting algorithm based on merge sort which is called array sort. Array sort not only takes advantages of merge sort, but also simplify the merge step by using a tag array. The proposed implementation consists of three phases: tag array creation phase, tag array split phase and list merge phase. Instead of just evaluating array sort in a single thread environment, the authors also run their experiments on multi-thread to test how array sort performs.

Complexity verification through design and analysis of computer experiments

This research article is a systematic study towards exploring the parameterised behaviour of smart sort, a comparison based sorting algorithm. Our observation for quick sort led us to conjecture that for sufficiently large samples of fixed size, the average case runtime complexity is: yavg(n, td) = Oemp(td), where yavg denotes the average complexity with parameters n and td denoting the input size and frequency of an element (tie density) respectively. The notation Oemp (also called empirical-O) is the statistical bound estimate obtained by running computer experiments. Performance of heap sort is better for discrete inputs with low k values (or equivalently high td values) and the runtime reaches to maximum beyond a threshold k. These two observations are opposite in their behaviour. The smart sort, which is designed by combining the key functions of standard quick and heap sort algorithms, is expected to behave optimally with respect to the different input parameters. The robustness of average case Oemp(nlog2n) complexity for smart sort is conjectured as result of study for various regression models and factorial design experiments.

Special Issue on Applications for the Heterogeneous Computing Era

As we look forward to the exascale era, heterogeneous parallel machines with accelerators, such as GPUs, FPGAs, and upcoming on-chip accelerator cores, are expected to play a massive role in architecting the largest systems in the world. While there is significant interest in accelerator-based architectures, much of this interest is an artifact of the hype associated with them. This special issue focuses on understanding the implications of accelerators on the architectures and programming environments of future systems. It seeks to ground accelerator research through studies of application kernels or whole applications on such systems, as well as tools and libraries that improve the performance or productivity of applications trying to use these systems. For accelerator-based heterogeneous systems to truly be a successful high-performance computing (HPC) platform, it is important that we obtain a complete picture of HPC applications and learn the opportunities and challenges these architectures raise. We need to learn the characteristics of computational kernels and applications, and how different software stacks impact them, in order to guide future accelerator-based HPC system designs. In this special issue, we presented case studies about accelerating representative kernels and applications on emerging multicore and manycore systems, including Intel MIC (Many Integrated Core) and GPU architectures. We also demonstrated better designs in the programming models to scale the performance of applications running on HPC systems. Finally, we investigated algorithms designs for large-scale systems and studied the power–performance tradeoffs of various optimizations techniques on heterogeneous platforms. In ‘‘Using MIC to accelerate graph traversal’’, Gao et al. describe a highly optimized breadth-first graph traversal algorithm designed for the MIC architecture. The algorithm utilizes both the MIC accelerator and the host CPU, and thus exploits the full capability of the heterogeneous system. Graph traversal is an important kernel for big data analysis, and we believe their optimized algorithm will help other researchers and practitioners in this area. In ‘‘Comparison sorting on hybrid multicore architectures for fixed and variable length keys’’, Banerjee et al. present a hybrid comparison-based sorting algorithm which utilizes a NVidia GPU and an Intel i7 CPU. The algorithm explores ways to divide-andconquer the overall problem. The algorithm achieves a 20% gain over the current best known comparison sorting algorithm. They also use a look-ahead-based approach to sort strings and obtain around 24% performance benefit over the current best known solution. Sorting has been a topic of immense research value and we think the advance in sorting efficiency may have tremendous impact in various types of applications. In ‘‘Composing multiple StarPU applications over heterogeneous machines: a supervised approach’’, Hugo et al. propose an extension of StarPU, a runtime system specifically designed for heterogeneous architectures to allow multiple parallel codes to run concurrently with minimal interference. They introduce a hypervisor that automatically expands or shrinks scheduling contexts (e.g. resource allocation) using feedbacks from the runtime system. Their mechanism can dramatically improve the overall application runtime by 34%. In ‘‘Evaluating the multi-core and many-core architectures through accelerating the 3D LWC stencil’’, You et al. showcase how they accelerate the iterative stencil loops in wave propagation forward modeling, which is a widely used computational method in oil and gas exploration. They experimented with architectures including Intel Sandy Bridge, NVidia Fermi C2070, NVidia Kepler K20x, and the Intel Xeon Phi Coprocessors. Numerous parallel strategies and optimization techniques are employed. The study about crossplatform performance and power analysis is also conducted. In ‘‘Analyzing power efficiency of optimization techniques and algorithm design methods for applications on heterogeneous platforms’’, Ukidave et al. evaluate the power/performance efficiency of different optimizations

Comparison sorting on hybrid multicore architectures for fixed and variable length keys

Sorting has been a topic of immense research value since the inception of computer science. Hybrid computing on multicore architectures involves computing simultaneously on a tightly coupled heterogeneous collection of devices. In this work, we consider a multicore CPU along with a manycore GPU as our experimental hybrid platform. In this work, we present a hybrid comparison based sorting algorithm which utilizes a many-core GPU and a multi-core CPU to perform sorting. The algorithm is broadly based on splitting the input list according to a large number of splitters followed by creating independent sublists. Sorting the independent sublists results in sorting the entire original list. On a CPU + GPU platform consisting of an Intel i7-980X and an NVidia GTX 580, our algorithm achieves a 20% gain over the current best known comparison sort result that was published by (Davidson et al., 2012). On the above experimental platform, our results are better by 40% on average over a similar GPU-alone algorithm proposed by (Leischner et al., 2010). We also extend our sorting algorithm for fixed length keys to variable length keys. We use a look-ahead based approach to sort strings and obtain around a 24% benefit compared to the current best known solution. Our results also show that our algorithm and its implementation scale with the size of the input. We also show that such performance gains can be obtained on other hybrid CPU + GPU platforms.

DETERMINISTIC SAMPLE SORT FOR GPUS

We demonstrate that parallel deterministic sample sort for many-core GPUs (GPU BUCKET SORT) is not only considerably faster than the best comparison-based sorting algorithm for GPUs (THRUST MERGE [Satish et.al., Proc. IPDPS 2009]) but also as fast as randomized sample sort for GPUs (GPU SAMPLE SORT [Leischner et.al., Proc. IPDPS 2010]). However, deterministic sample sort has the advantage that bucket sizes are guaranteed and therefore its running time does not have the input data dependent fluctuations that can occur for randomized sample sort.

An experimental study of sorting and branch prediction

Sorting is one of the most important and well-studied problems in computer science. Many good algorithms are known which offer various trade-offs in efficiency, simplicity, memory use, and other factors. However, these algorithms do not take into account features of modern computer architectures that significantly influence performance. Caches and branch predictors are two such features and, while there has been a significant amount of research into the cache performance of general purpose sorting algorithms, there has been little research on their branch prediction properties. In this paper, we empirically examine the behavior of the branches in all the most common sorting algorithms. We also consider the interaction of cache optimization on the predictability of the branches in these algorithms. We find insertion sort to have the fewest branch mispredictions of any comparison-based sorting algorithm, that bubble and shaker sort operate in a fashion that makes their branches highly unpredictable, that the unpredictability of shellsort's branches improves its caching behavior, and that several cache optimizations have little effect on mergesort's branch mispredictions. We find also that optimizations to quicksort, for example the choice of pivot, have a strong influence on the predictability of its branches. We point out a simple way of removing branch instructions from a classic heapsort implementation and also show that unrolling a loop in a cache-optimized heapsort implementation improves the predicitability of its branches. Finally, we note that when sorting random data two-level adaptive branch predictors are usually no better than simpler bimodal predictors. This is despite the fact that two-level adaptive predictors are almost always superior to bimodal predictors, in general.

Sorting and Searching in Faulty Memories

In this paper we investigate the design and analysis of algorithms resilient to memory faults. We focus on algorithms that, despite the corruption of some memory values during their execution, are nevertheless able to produce a correct output at least on the set of uncorrupted values. In this framework, we consider two fundamental problems: sorting and searching. In particular, we prove that any O(nlog n) comparison-based sorting algorithm can tolerate the corruption of at most O((nlog n)1/2) keys. Furthermore, we present one comparison-based sorting algorithm with optimal space and running time that is resilient to O((nlog n)1/3) memory faults. We also prove polylogarithmic lower and upper bounds on resilient searching.

A simplified derivation of timing complexity lower bounds for sorting by comparisons

We present a simplified derivation of the fact that the complexity-theoretic lower bound of comparison-based sorting algorithms, both for the worst-case and for the average-case time measure, is Ω(...

Adapting Radix Sort to the Memory Hierarchy

We demonstrate the importance of reducing misses in the translation-lookaside buffer (TLB) for obtaining good performance on modern computer architectures. We focus on least-significantbit first (LSB) radix sort, standard implementations of which make many TLB misses. We give three techniques which simultaneously reduce cache and TLB misses for LSB radix sort: reducing working set size, explicit block transfer and pre-sorting. We note that: • All the techniques above yield algorithms whose implementations outperform optimised cache-tuned implementations of LSB radix sort and comparison-based sorting algorithms. The fastest running times are obtained by the pre-sorting approach and these are over twice as fast as optimised cache-tuned implementations of LSB radix sort and quicksort. Even the simplest optimisation, using the TLB size to guide the choice of radix in standard implementations of LSB radix sort, gives good improvements over cache-tuned algorithms. • One of the pre-sorting algorithms and explicit block transfer make few cache and TLB misses in the worst case. This is not true of standard implementations of LSB radix sort.We also apply these techniques to the problem of permuting an array of integers, and obtain gains of over 30% relative to the naive algorithm by using explicit block transfer.

A Wait-Free Sorting Algorithm

Sorting is one of a set of fundamental problems in computer science. In this paper we present the first wait-free algorithm for sorting an input array of size N using P ≤ N processors to achieve optimal running time. We show two variants of the algorithm, one deterministic and one randomized, and prove that, with high probability, the latter suffers no more than $O(\sqrt{P})$ contention when run synchronously. Known sorting algorithms, when made wait-free through previously established transformation techniques, have complexity O(log 3 N) . The algorithm we present here, when run in the CRCW PRAM model, executes with high probability in O(log N) time when P=N , and O((Nlog N)/P) otherwise, which is optimal amongst comparison-based sorting algorithms. The wait-free property guarantees that the sort will complete despite any delays or failures incurred by the processors. This is a very desirable property from an operating systems point of view, since it allows oblivious thread scheduling as well as thread creation and deletion, without fear of losing the algorithm's correctness.

The Influence of Caches on the Performance of Sorting

We investigate the effect that caches have on the performance of sorting algorithms both experimentally and analytically. To address the performance problems that high cache miss penalties introduce we restructure mergesort, quicksort, and heapsort in order to improve their cache locality. For all three algorithms the improvement in cache performance leads to a reduction in total execution time. We also investigate the performance of radix sort. Despite the extremely low instruction count incurred by this linear time sorting algorithm, its relatively poor cache performance results in worse overall performance than the efficient comparison based sorting algorithms. For each algorithm we provide an analysis that closely predicts the number of cache misses incurred by the algorithm.

Communication-Efficient Parallel Sorting

We study the problem of sorting n numbers on a p-processor bulk-synchronous parallel (BSP) computer, which is a parallel multicomputer that allows for general processor-to-processor communication rounds provided each processor sends and receives at most h items in any round. We provide parallel sorting methods that use internal computation time that is $O({n\log n \over p})$ and a number of communication rounds that is $O({\log n \over \log (h+1)})$ for $h=\Theta(n/p)$. The internal computation bound is optimal for any comparison-based sorting algorithm. Moreover, the number of communication rounds is bounded by a constant for the (practical) situations when $p\le n^{1-{1/c}}$ for a constant $c\ge 1$. In fact, we show that our bound on the number of communication rounds is asymptotically optimal for the full range of values for p, for we show that just computing the "or"of n bits distributed evenly to the first O(n/h) of an arbitrary number of processors in a BSP computer requires $\Omega(\log n/\log (h+1))$ communication rounds.

Optimal Time-Space Trade-Offs for Sorting

We study the fundamental problem of sorting in a sequential model of computation and in particular consider the time-space trade-off (product of time and space) for this problem.&lt;br /&gt;Beame has shown a lower bound of Omega(n^2) for this product leaving a gap of a logarithmic factor up to the previously best known upper bound of O(n^2 log n) due to Frederickson. Since then, no progress has been made towards tightening this gap.&lt;br /&gt;The main contribution of this paper is a comparison based sorting algorithm which closes this gap by meeting the lower bound of Beame. The time-space product O(n^2) upper bound holds for the full range of space bounds between log n and n/log n. Hence in this range our algorithm is optimal for comparison based models as well as for the very powerful general models considered by Beame.

Splaysort: Fast, Versatile, Practical

Adaptivity in sorting algorithms is sometimes gained at the expense of practicality. We give experimental results showing that Splaysort - sorting by repeated insertion into a Splay tree - is a surprisingly efficient method for in-memory sorting. Splaysort appears to be adaptive with respect to all accepted measures of presortedness, and it outperforms Quicksort for sequences with modest amounts of existing order. Although Splaysort has a linear space overhead, there are many applications for which this is reasonable. In these situations Splaysort is an attractive alternative to traditional comparison-based sorting algorithms such as Heapsort, Mergesort, and Quicksort.

A special function unit for database operations (SFU-DB): design and performance evaluation

The design and analysis of a special function unit for database operations (SFU-DB) that uses a novel hardware sorting module, the automatic retrieval memory (ARM), are described. The SFU-DB is a functionally independent unit that efficiently performs certain nonnumeric operations. It can function as a coprocessor for a host CPU or as a special processing unit in a highly parallel processing system. The ARM implements in hardware a true distribution-based sort algorithm that requires no comparison operations. Without performing any comparison, the SFU-DB avoids the lower bound constraint on comparison-based sorting algorithms and achieves, for the worst case, a complexity of O(n) for both execution time and main memory size. Using the fundamental sort algorithm with slight modifications. the SFU-DB also uses the ARM as an engine for other primitive database operations such as relational join, elimination of duplicates, set union, set intersection, and set difference, also with complexity of O(n). The SFU-DB/ARM architecture is rather simple and requires only a modest amount of specialized hardware. The specialized hardware has been designed and simulated for fabrication using CMOS gate arrays, and the remainder of the SFU-DB has been simulated in software using Turbo Pascal running on an IBM-PC.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>