Integer Sorting Research Articles

Parallel space‐filling curve generation through sorting

AbstractIn this paper we consider the scalability of parallel space‐filling curve generation as implemented through parallel sorting algorithms. Multiple sorting algorithms are studied and results show that space‐filling curves can be generated quickly in parallel on thousands of processors. In addition, performance models are presented that are consistent with measured performance and offer insight into performance on still larger numbers of processors. At large numbers of processors, the scalability of adaptive mesh refined codes depends on the individual components of the adaptive solver. One such component is the dynamic load balancer. In adaptive mesh refined codes, the mesh is constantly changing resulting in load imbalance among the processors requiring a load‐balancing phase. The load balancing may occur often, requiring the load balancer to perform quickly. One common method for dynamic load balancing is to use space‐filling curves. Space‐filling curves, in particular the Hilbert curve, generate good partitions quickly in serial. However, at tens and hundreds of thousands of processors serial generation of space‐filling curves will hinder scalability. In order to avoid this issue we have developed a method that generates space‐filling curves quickly in parallel by reducing the generation to integer sorting. Copyright © 2007 John Wiley & Sons, Ltd.

On coding labeled trees

We consider the problem of coding labeled trees by means of strings of node labels. Different codes have been introduced in the literature by Prüfer, Neville, and Deo and Micikevičius. For all of them, we show that both coding and decoding can be reduced to integer (radix) sorting, closing several open problems within a unified framework that can be applied both in a sequential and in a parallel setting. Our sequential coding and decoding schemes require optimal O ( n ) time when applied to n -node trees, yielding the first linear time decoding algorithm for a code presented by Neville. These schemes can be parallelized on the EREW PRAM model, so as to work in O ( log n ) time with cost O ( n ) , O ( n log n ) , or O ( n log n ) , depending on the code and on the operation: in all cases, they either match or improve the performances of the best ad hoc approaches known so far.

O(log*n) algorithms on a Sum-CRCW PRAM

We present work- and cost-optimal O(log*n) algorithms for prefix sums and linear integer sorting on a Sum-CRCW PRAM.

RANDOMIZED SORTING ON THE POPS NETWORK

Partitioned Optical Passive Stars (POPS) network has been presented recently as a desirable model of parallel computation. Several papers have been published that address fundamental problems on this architecture. The algorithms presented in this paper are valid for POPS(d,g) where d>g and use randomization. We present an algorithm that solves the problem of sparse enumeration sorting of d∊ keys in [Formula: see text] time and hence performs better than previous algorithms. We also present algorithms that allow us to do stable sorting of integers in the range [1, log n] and [Formula: see text] in [Formula: see text] time. When g=n∊, for any constant 0<∊<½ this allows us to do sorting of integers in the range [1,n] in [Formula: see text] time. We finally use these algorithms to solve the problem of multiple binary search in the case where we have d∊ keys in [Formula: see text] time and in the case where we have integer keys in the range [1,n] in [Formula: see text] time, when g=n∊, for some constant 0<∊<½.

An Analysis of the Cost Effectiveness of an Adaptable Computing Cluster

With a focus on commodity PC systems, Beowulf clusters traditionally lack the cutting edge network architectures, memory subsystems, and processor technologies found in their more expensive supercomputer counterparts. What Beowulf clusters lack in technology, they more than make up for with their significant cost advantage over traditional supercomputers. This paper presents the cost implications of an architectural extension that adds reconfigurable computing to the network interface of Beowulf clusters. This extension is called an intelligent network interface card (INIC). A quantitative description of cost-effectiveness is formulated to compare alternatives. Cost-effectiveness is considered in the context of three applications: the 2D Fast Fourier Transform (2D-FFT), integer sorting, and PNN image classification. It is shown that, for these three representative applications, there is a range of basic hardware costs and cluster sizes for which the INIC is more efficient than a purely serial solution or an ordinary cluster. Furthermore, the cost model has proven useful for designing the next generation INIC.

Linear time algorithms for the ring loading problem with demand splitting

Given a ring of size n and a set K of traffic demands, the ring loading problem with demand splitting (RLPW) is to determine a routing to minimize the maximum load on the edges. In the problem, a demand between two nodes can be split into two flows and then be routed along the ring in different directions. If the two flows obtained by splitting a demand are restricted to integers, this restricted version is called the ring loading problem with integer demand splitting (RLPWI). In this paper, efficient algorithms are proposed for the RLPW and the RLPWI. Both the proposed algorithms require O(| K|+ t s ) time, where t s is the time for sorting | K| nodes. If | K|⩾ n ε for some small constant ε>0, integer sort can be applied and thus t s = O(| K|); otherwise, t s = O(| K|log| K|). For real world applications, | K| is usually not smaller than n and thus our algorithms achieve linear time. The proposed algorithms improve the previous upper bounds from O(min{ n| K|, n 2}) for RLPW and from O( n| K|) for RLPWI.

Deterministic sorting in O( nloglog n) time and linear space

We present a fast deterministic algorithm for integer sorting in linear space. Our algorithm sorts n integers in the range {0,1,2,…, m−1} in linear space in O( nloglog n) time. This improves our previous result [Y. Han, Inform. and Comput. 170 (1) (2001) 81–94] which sorts in O( nloglog nlogloglog n) time and linear space. This also improves previous best deterministic sorting algorithm [A. Andersson, T. Hagerup, S. Nilsson, R. Raman, in: Proc. 1995 Symposium on Theory of Computing (1995) 427–436; Y. Han, X. Shen, in: Proc. 1995 International Computing and Combinatorics Conference, in: Lecture Notes in Comput. Sci. 959 (1995) 324–333] which sorts in O( nloglog n) time but uses O( m ε ) space. Our results also improves the result of Andersson et al. [A. Andersson, T. Hagerup, S. Nilsson, R. Raman, in: Proc. 1995 Symposium on Theory of Computing (1995) 427–436] which sorts in O( nloglog n) time and linear space but uses randomization.

Analysis of a prototype intelligent network interface

AbstractWith a focus on commodity PC systems, Beowulf clusters traditionally lack the cutting edge network architectures, memory subsystems, and processor technologies found in their more expensive supercomputer counterparts. Many users find that what Beowulf clusters lack in technology, they more than make up for with their significant cost advantage. In this paper, an architectural extension that adds reconfigurable computing to the network interface of Beowulf clusters is proposed. The proposed extension, called an intelligent network interface card (or INIC), enhances both the network and processor capabilities of the cluster, which has a significant impact on the performance of a crucial class of applications. Furthermore, for some applications, the proposed extension partially compensates for weaknesses in the PC memory subsystem. A prototype of the proposed architecture was constructed and analyzed. In addition, two applications, the 2D Fast Fourier Transform (2D‐FFT) and integer sorting, which benefit from the resulting architecture, are discussed and analyzed on the proposed architecture. Specifically, results indicate that the 2D‐FFT is performed 15–50% faster when using the prototype INIC rather than the comparable Gigabit Ethernet. Early simulation results also indicate that integer sorting will receive a 21–32% performance boost from the prototype. The significant improvements seen with a relatively limited prototype lead to the conclusion that cluster network interfaces enhanced with reconfigurable computing could significantly improve the Beowulf architecture. Copyright © 2003 John Wiley & Sons, Ltd.

Communication—Processor Tradeoffs in a Limited Resources PRAM

We consider a simple restriction of the PRAM model (called PPRAM), where the input is arbitrarily partitioned between a fixed set of p processors and the shared memory is restricted to m cells. This model allows for investigation of the tradeoffs/ bottlenecks with respect to the communication bandwidth (modeled by the shared memory size m ) and the number of processors p . The model is quite simple and allows the design of optimal algorithms without losing the effect of communication bottlenecks. We have focused on the PPRAM complexity of problems that have $\tilde{O}$ (n) sequential solutions (where n is the input size), and where m ≤ p ≤ n . We show essentially tight time bounds (up to logarithmic factors) for several problems in this model such as summing, Boolean threshold, routing, integer sorting, list reversal and k -selection. We get typically two sorts of complexity behaviors for these problems: One type is $\tilde{O}$ (n/p + p/m) , which means that the time scales with the number of processors and with memory size (in appropriate ranges) but not with both. The other is $\tilde{O}$ (n/m) , which means that the running time does not scale with p and reflects a communication bottleneck (as long as m < p ). We are not aware of any problem whose complexity scales with both p and m (e.g. $O(n/\sqrt{m \cdot p})$ ). This might explain why in actual implementations one often fails to get p -scalability for p close to n .

Improved disk-drive failure warnings

Improved methods are proposed for disk-drive failure prediction. The SMART (self monitoring and reporting technology) failure prediction system is currently implemented in disk-drives. Its purpose is to predict the near-term failure of an individual hard disk-drive, and issue a backup warning to prevent data loss. Two experimental tests of SMART show only moderate accuracy at low false-alarm rates. (A rate of 0.2% of total drives per year implies that 20% of drive returns would be good drives, relative to /spl ap/1% annual failure rate of drives). This requirement for very low false-alarm rates is well known in medical diagnostic tests for rare diseases, and methodology used there suggests ways to improve SMART. Two improved SMART algorithms are proposed. They use the SMART internal drive attribute measurements in present drives. The present warning-algorithm based on maximum error thresholds is replaced by distribution-free statistical hypothesis tests. These improved algorithms are computationally simple enough to be implemented in drive microprocessor firmware code. They require only integer sort operations to put several hundred attribute values in rank order. Some tens of these ranks are added up and the SMART warning is issued if the sum exceeds a prestored limit. These new algorithms were tested on 3744 drives of 2 models. They gave 3-4 times higher correct prediction accuracy than error thresholds on will-fail drives, at 0.2% false-alarm rate. The highest accuracies achievable are modest (40%-60%). Care was taken to test will-fail drive prediction accuracy on data independent of the algorithm design data. Additional work is needed to verify and apply these algorithms in actual drive design. They can also be useful in drive failure analysis engineering. It might be possible to screen drives in manufacturing using SMART attributes. Marginal drives might be detected before substantial final test time is invested in them, thereby decreasing manufacturing cost, and possibly decreasing overall field failure rates.

Parallel Integer Sorting Is More Efficient Than Parallel Comparison Sorting on Exclusive Write PRAMs

We present a significant improvement for parallel integer sorting. On the EREW (exclusive read exclusive write) PRAM our algorithm sorts n integers in the range {0,1, . . . ,m-1 } in time O(log n) with $O(n \sqrt{\frac{\log n}{k}})$ operations using word length $k \log (m+n)$, where $1 \leq k \leq \log n$. In this paper we present the following four variants of our algorithm. \noindent (1) The first variant sorts integers in $\{ 0, 1, \ldots, m-1\}$ in time O(log n) and in linear space with O(n) operations using word length log m log n. (2) The second variant sorts integers in {0, 1, . . , n-1} in time O(log n) and in linear space with $O(n\sqrt{\log n})$ operations using word length log n. (3) The third variant sorts integers in {0, 1, . . . , m-1} in time O(log3/2 n) and in linear space with $O(n\sqrt{\log n})$ operations using word length log (m+n). (4) The fourth variant sorts integers in {0, 1, . . . ,m-1} in time O(log n) and space $O(nm^\epsilon )$ with $O(n\sqrt{\log n})$ operations using word length log (m+n). Our algorithms can then be generalized to the situation where the word length is k log (m+n), $1 \leq k \leq \log n$.

Improved Fast Integer Sorting in Linear Space

We present a fast deterministic algorithm for integer sorting in linear space. Our algorithm sorts n integers in the range {0, 1, 2, …, m−1} in linear space in O(n log log n log log log n) time. When log m≥log2+ϵn, ϵ>0, we can further achieve O(n log log n) time. This improves the O(n(log log n)2) time bound given in M. Thorup (1998) in “Proc. 1998 ACM-SIAM Symp. on Discrete Algorithms (SODA'98),” pp. 550–555). This result is obtained by combining our new technique with that of Thorup's. Signature sorting (A. Andersson, T. Hagerup, S. Nilsson, and R. Raman, 1995, in “Proc. 1995 Symposium on Theory of Computing,” pp. 427–436), A. Andersson's result (1996, in “Proc. 1996 IEEE Symp. on Foundations of Computer Science,” pp. 135–141), R. Raman's result (1996, Lecture Notes in Computer Science, Vol. 1136, pp. 121–137, Springer-Verlag Berlin/New York), and our previous result (Y. Han and X. Shen, 1999, in “Proc. 1999 Tenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA'99),” Baltimore, MD, January, pp. 419–428) are also used for the design of our algorithms. We provide an approach and techniques which are totally different from previous approaches and techniques for the problem. As a consequence our technique can be extended to apply to nonconservative sorting and parallel sorting. Our nonconservative sorting algorithm sorts n integers in {0, 1, …, m−1} in time O(n(log log n)2/(log k+log log log n)) using word length k log(m+n), where k≤log n. Our EREW parallel algorithm sorts n integers in {0, 1, …, m−1} in O((log n)2) time and O(n(log log n)2/log log log n) operations provided log m=Ω((log n)2).

Limited multiple-writer: An approach to dealing with false sharing in software DSMs

False sharing is one of the most important factors impacting the performance of DSM (distributed shared memory) systems. The single-writer approach is simple, but it cannot avoid the ping-pong effect of the data page thrashing, while the multiple-writer approach is effective for false sharing but with high cost. This paper proposes a new approach, called limited multiple-writer (LMW) to handling multiple writers in software DSM. It distinguishes two kinds of multiple-writer as lock-based form and barrier-based form, and handles them with different policies. It discards theTwin andDiff in traditional multiple-writer approach, and simplifies the implementation of multiple-writer in software DSM systems. The implementation of LMW in a CVM (Coherent Virtual Machine) software DSM system, which is based on a network of workstations, is introduced. Evaluation results show that for some applications such as SOR (Successive Over-Relaxation), LU (Lower triangular and Upper triangular), FFT (Fast Fourier Transformation), and IS (Integer Sorting), LMW provides a significant reduction in execution time (11%, 16%, 33% and 46%) compared with the traditional multiple-writer approach on the platform.

A NOTE ON COARSE GRAINED PARALLEL INTEGER SORTING

We observe that for n/p ≥ p, which is usually the case in practice, there exists a very simple, deterministic, optimal coarse grained parallel integer sorting algorithm with 24 communication rounds (6 [Formula: see text]-relations and 18 p-relations), O (n/p) memory per processor and O (n/p) local computation. Experimental data indicates that the algorithm has very good performance in practice.

Sorting in Linear Time?

We show that a unit-cost RAM with a word length ofwbits can sortnintegers in the range 0…2w−1 inO(nloglogn) time for arbitraryw⩾logn, a significant improvement over the bound ofO(nlogn) achieved by the fusion trees of Fredman and Willard. Provided thatw⩾(logn)2+εfor some fixedε>0, the sorting can even be accomplished in linear expected time with a randomized algorithm. Both of our algorithms parallelize without loss on a unit-cost PRAM with a word length ofwbits. The first one yields an algorithm that usesO(logn) time andO(nloglogn) operations on a deterministic CRCW PRAM. The second one yields an algorithm that usesO(logn) expected time andO(n) expected operations on a randomized EREW PRAM, provided thatw⩾(logn)2+εfor some fixedε>0. Our deterministic and randomized sequential and parallel algorithms generalize to the lexicographic sorting of multiple-precision integers represented in several words.

A Randomized Parallel Sorting Algorithm with an Experimental Study

Previous schemes for sorting on general-purpose parallel machines have had to choose between poor load balancing and irregular communication or multiple rounds of all-to-all personalized communication. In this paper, we introduce a novel variation on sample sort which uses only two rounds of regular all-to-all personalized communication in a scheme that yields very good load balancing with virtually no overhead. Moreover, unlike previous variations, our algorithm efficiently handles the presence of duplicate values without the overhead of tagging each element with a unique identifier. This algorithm was implemented in Split-C and run on a variety of platforms, including the Thinking Machines CM-5, the IBM SP-2, and the Cray Research T3D. We ran our code using widely different benchmarks to examine the dependence of our algorithm on the input distribution. Our experimental results illustrate the efficiency and scalability of our algorithm across different platforms. In fact, it seems to outperform all similar algorithms known to the authors on these platforms, and its performance is invariant over the set of input distributions unlike previous efficient algorithms. Our results also compare favorably with those reported for the simpler ranking problem posed by the NAS Integer Sorting (IS) Benchmark.

INTEGER SORTING AND ROUTING IN ARRAYS WITH RECONFIGURABLE OPTICAL BUSES

In this paper we present deterministic algorithms for integer sorting and on-line packet routing on arrays with reconfigurable optical buses. The main objective is to identify the mechanisms specific to this type of architectures that allow us to build efficient integer sorting, partial permutation routing and h-relations algorithms. The consequences of these results on the PRAM simulation complexity are also investigated.

Improved Bounds for Integer Sorting in the EREW PRAM Model

A new simple method of exploiting nonstandard word length in the nonconservative RAM and PRAM models is considered. As a result, improved bounds for parallel integer sorting in the EREW PRAM model with standard and nonstandard word length are obtained.

Improved Parallel Integer Sorting without Concurrent Writing

We show thatnintegers in the range 1, …, ncan be sorted stably on an EREW PRAM usingO(t) time andO(n(lognloglogn+(log n)2/t)) operations, for arbitrary givent⩾log n log log n, and on a CREW PRAM usingO(t) time andO(n(logn+log n/2t/log n)) operations, for arbitrary givent⩾log n. In addition, we are able to sortnarbitrary integers on a randomized CREW PRAM within the same resource bounds with high probability. In each case our algorithm is a factor of almostΘ:(logn) closer to optimality than all previous algorithms for the stated problem in the stated model, and our third result matches the operation count of the best previous sequential algorithm. We also show thatnintegers in the range 1, …, mcan be sorted inO((log n)2) time withO(n) operations on an EREW PRAM using a nonstandard word length ofO(log n log log n log m) bits, thereby greatly improving the upper bound on the word length necessary to sort integers with a linear time–processor product, even sequentially. Our algorithms were inspired by, and in one case directly use, the fusion trees of Fredman and Willard.

Quantifying the Performance Differences between PVM and TreadMarks

This paper compares two systems for parallel programming on networks of workstations: Parallel Virtual Machine (PVM), a message-passing system, and TreadMarks, a software distributed shared-memory (DSM) system. The eight applications used in this comparison are Water and Barnes–Hut from the SPLASH benchmark suite; 3-D FFT, Integer Sort (IS), and Embarrassingly Parallel (EP) from the NAS benchmarks; ILINK, a widely used genetic linkage analysis program; and Successive Over-Relaxation (SOR) and Traveling Salesman (TSP). Two different input data sets are used for five of the applications. We use two execution environments. The first is a 155 Mbps ATM network with eight Sparc-20 model 61 workstations; the second is an eight-processor IBM SP/2. The differences in speedup between TreadMarks and PVM depend mostly on the applications, and only to a much lesser extent on the platform and the data set used. In particular, the TreadMarks speedup for six of the eight applications is within 15% of that achieved with PVM. For one application, the difference in speedup is between 15% and 30%, and for another, the difference is around 50%. We identified four important factors that contribute to the lower performance of TreadMarks: (1) extra messages due to the separation of synchronization and data transfer, (2) extra messages to handle access misses caused by the use of an invalidate protocol, (3) false sharing, and (4) diff accumulation for migratory data. We have quantified the effects of the last three factors by measuring the performance gain when each is eliminated. Of the three factors, TreadMarks' use of a separate request message per page of data accessed is the most important. The effect of false sharing is comparatively low. Reducing diff accumulation benefits migratory data only when the diffs completely overlap. When these performance impediments are removed, all of the TreadMarks programs perform within 25% of PVM, and for six out of eight experiments, TreadMarks is less than 5% slower than PVM.