Bitonic Sort Research Articles

Computations on the massively parallel processor at the Goddard Space Flight Center

Described are four significant algorithms implemented on the massively parallel processor (MPP) at the Goddard Space Flight Center. Two are in the area of image analysis. Of the other two, one is a mathematical simulation experiment and the other deals with the efficient transfer of data between distantly separated processors in the MPP array. The first algorithm presented is the automatic determination of elevations from stereo pairs. The second algorithm solves mathematical logistic equations capable of producing both ordered and chaotic (or random) solutions. This work can potentially lead to the simulation of artificial life processes. The third algorithm is the automatic segmentation of images into reasonable regions based on some similarity criterion, while the fourth is an implementation of a bitonic sort of data which significantly overcomes the nearest neighbor interconnection constraints on the MPP for transferring data between distant processors.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Running algorithms efficiently on faulty hypercubes (extended abstract)

We examine the issue of running algorithms with a constant factor slowdown on a faulty hypercube in a worst case scenario. We present two sets of novel results related to this issue. We first consider edge faults and show how to tolerate faults with a constant factor slow-down in communication and no slowdown in computation. The key to our approach is an efficient method for embedding a fault free Cube Connected Cycles (CCC) graph in the faulty hypercube. Using this embedding we can run ascend-descend algorithms (such as bitonic sort) on the faulty hypercube by implementing them on the embedded CCC. We then consider hypercubes with both edge and node faults. We prove that for any constant c there exists a fault-free subgraph of an n -dimensional hypercube with n c faulty components that can implement a large class of hypercube algorithms with only a constant factor slowdown. To the best of our knowledge, this result is the first in which a hypercube can tolerate more than O ( n ) faults in the worst case sense.

Experimental analysis of a mixed-mode parallel architecture using bitonic sequence sorting

Experimentation aimed at determining the potential benefit of mixed-mode SIMD/MIMD parallel architectures is reported. The experimentation is based on timing measurements made on the PASM system prototype at Purdue utilizing carefully coded synthetic variations of a well-known algorithm. The synthetic algorithms used to measure and evaluate this system were based on the bitonic sorting of sequences stored in the processing elements. This computation was mapped to both the SIMD and MIMD modes of parallelism, as well as to two hybrids of the SIMD and MIMD modes. The computations were coded in these four ways and experiments that explore the trade-offs among them were performed. Consideration is given to the overhead caused by enabling or disabling PEs in SIMD mode and a lower bound on this overhead is shown. Finally, a more efficient PE mask generation scheme for multiple “off-the-shelf” microprocessor-based SIMD systems is emulated and its performance is analyzed. The results of these experiments are presented and are discussed with special consideration of the effects of the system's architecture. The goal is to (as much as possible) obtain implementation independent analyses of the attributes of mixed-mode parallel processing with respect to the computational characteristics of the application being examined. The results are used to gain insight into the impact of computation mode on synchronization and data-conditional aspects of system performance.

A unified algorithm for sorting on multidimensional mesh-connected processors

The algorithm, dubbed MeshSort, is significant in that it is a general algorithm that unifies two well known algorithms that operate on special cases of the multidimensional mesh: Bitonic sort on the binary hypercube, and Shearsort on the two-dimensional mesh.

An experimental ATM switch for BISDN studies

AbstractThis paper describes the architecture, functionality and performance of an experimental ATM switch being developed at the Telecom Australia Research Laboratories as part of its investigations into the broadband ISDN. The proposed switch architecture consists of parallel omega networks preceded by a Batcher bitonic sorting network. The switching fabric has no internal cell buffering. Cell buffering is provided only on the switch outputs for cells simultaneously contending for the same output port. The switch fabric and cell buffers include mechanisms for providing prioritized servicing of queued cells and prioritized discarding of cells based on priority fields contained within the cell header. Components of the switch are currently being implemented in 2 μm CMOS VLSI.

A modular architecture for very large packet switches

Switch modules, the building blocks of this system, are independently operated packet switches. Each module consists of a Batcher sorting network, a stack of binary trees, and a bundle of banyan networks. The modular architecture is a unification of the Batcher-banyan switch and the knockout switch, and can be physically realized as an array of three-dimensional parallel processors. Switch modules are interconnected only at the outputs by multiplexers. The partitioned switch fabric provides a flexible distributed architecture, which is the key to simplify the operation and maintenance of the whole switching system. The modularity implies less stringent synchronization requirements and makes higher-speed implementation possible. The proposed modular switch is intended to meet the needs of broadband telephone offices of all sizes. It is estimated that a modular switch with terabit capacity can be built using current VLSI technologies. >

A new approach to fast control of r 2 × r 2 3-stage benes networks of r × r crossbar switches

The routing control of Benes networks has proven to be costly. This paper introduces a new approach to fast control of N × N 3-stage Benes networks of r × r crossbar switches as building blocks, where N = r 2 and r ≥ 2. The new approach consists of setting the leftmost column of switches to an appropriately chosen configuration so that the network becomes self-routed while still able to realize a given family of permutations. This approach requires that, for any given family of permutations, a configuration for the leftmost column be found. Such a family is called compatible and the configuration of the leftmost column is called the compatibility factor. In this paper, compatibility is characterized and a technique to determine compatibility and the compatibility factor is developed. The technique is used to show the compatibility and find the compatibility factor of Ω-realizable permutations, the permutations needed to emulate a hypercube, and the families of permutations required by FFT, bitonic sorting, tree computations, multidimensional mesh and torus computations, and multigrid computations. An O(log 2 N ) time routing algorithm for the 3-stage Benes will also be developed. Finally, as only 3 compatibility factors are required by the above families of permutations, it will be proposed to replace the first column by 3 multiplexed connections yielding a self-routing network with strong communication capabilities.

Adaptive Bitonic Sorting: An Optimal Parallel Algorithm for Shared-Memory Machines

We propose a parallel algorithm, called adaptive bitonic sorting, which runs on a PRAC, a shared-memory multiprocessor where fetch and store conflicts are disallowed. On a $P$ processors PRAC, our algorithm achieves optimal performance $TP = O(N \log N)$, for any computation time $T$ in the range $\Omega (\log^{2} N) \leq T \leq O(N \log N)$. Adaptive bitonic sorting has also a small constant factor, since it performs less than $2N \log N$ comparisons, and only a handful of operations per comparison.

K-way bitonic sort

The k-way bitonic sort algorithm, a generalization of K.E. Batcher's bitonic sort algorithm (1968), is presented. This variation of the algorithm is based on a k-way decomposition instead of a two-way decomposition. It is proven that Batcher's bitonic sequence decomposition theorem still holds with this multiway decomposition. This leads to applications of sorting networks with bitonic sorters of arbitrary or mixed sizes.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Token relabeling in a tagged token data-flow architecture

A direct structure access approach called token relabeling scheme is presented in which all array operations can be performed without the use of any intermediary structure memory. The graph constructs for both approaches are described. Four numerical algorithms including fast Fourier transform, bitonic sort, LU decomposition, and matrix multiplication are implemented in both approaches. Token relabeling is directly applicable to arrays which are entirely consumed. Graph constructs which allow partial consumption are also designed. The effectiveness and efficiency of this parallel random access construct verifies the feasibility and applicability of the token relabeling scheme. A deterministic simulation of Arvind and K.P. Gostelow's tagged-token data-flow architecture (1982) is undertaken to validate the graph designs and evaluate the performance of the graphs. It is shown that the direct access graphs present better performance with regard to execution time, speedup, and resource utilization.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A balanced bin sort for hypercube multicomputers

We propose a balanced bin sort for hypercube multicomputers. This sorting algorithm has an empirically measured expected run time that is greater than that of hyperquicksort but less than that of bitonic sort. Also, its space requirements are less than that of hyperquicksort but more than that of bitonic sort. So, it is useful in situations in which there is some excess memory but not enough to run hyperquicksort.

Sorting with optical compare-and-exchange modules

Sorting is central to the solution of many knowledge-based and switching problems in advanced computation and communication systems. Parallel-pipelined sorting algorithms are appropriate for applications that demand high throughput, low delay, and many data channels. One such algorithm, the bitonic sort, can be implemented with passive perfect shuffle interconnects between active stages of compare-and-exchange (C&E) elements. In this paper we focus on optical hardware to implement the C&E operation and show that, by taking advantage of a distinctive feature of optical logic, namely, bistability, comparison circuits of remarkable simplicity are attainable. We describe implementations of C&E in a variety of optical device technologies capable of performing latching and nonlatching logic. Based on the device characteristics we outline potential application areas for each technology.

Applicative parallelism on a shared-memory multiprocessor

An applicative language, SISAL (streams and iteration in a single-assignment language), is used to examine how effective the applicative approach can be in programming parallel computers. SISAL is a research language for investigating issues in parallel processing, especially for numerical computing. A prototype implementation of SISAL is described, focusing on loops and streams and on microtasking software. Performance results are reported for Livermore kernel 1, the Sieve prime finder, insertion sort, Batcher sort, the Simple hydrodynamics code, and the PSPhot photon-transport code. >

A Broadband Packet Switch for Integrated Transport

This paper gives a broadband (total throughput approaching 1 terabit/s) self-routing packet switch design for providing flexible multiple bit-rate broadband services for an end-to-end fiber network. The switch fabric for the slotted broadband packet switch delivers exactly one packet to each output port from one of the input ports which request packet delivery to that output port. The denied requests would try again during the next slot. We discover an effective scheme, implemented by CMOS VLSI with manageable complexity, for performing this function. First, each input port sends a request for a port destination through a Batcher Sorting network, which sorts the request destinations in ascending order so that we may easily purge all but one request for the same destination. The winning request acknowledges its originating port from the output of the Batcher network, with the acknowledgment routed through a Batcher-banyan selfrouting switch. The acknowledged input port then sends the full packet through the same Batcher-banyan switch without any conflict. Unacknowledged ports buffer the blocked packet for reentry in the next cycle. We also give several variations for significantly improved performance. We then study switch performance based on some rudimentary protocols for traffic control. For the basic scheme, we analyze the throughput-delay characteristics for random traffic, modeled by random output port requests and a binomial distribution of packet arrival. We demonstrate with a buffer size of around 20 packets, we can achieve a 50 percent loading with almost no buffer overflow. Maximum throughput of switch is 58 percent. Next, we investigate the performance of the switch in the presence of periodic broadband traffic. We then apply circuit switching techniques and packet priority for high bit-rate services in our packet switch environment. We improve the throughput per port to close to 100 percent by means of parallel switch fabric, while maintaining the periodic nature of the traffic.

Best and worst mappings for the omega network

This paper presents a study of the best and worst mappings for the omega network proposed by D. H. Lawrie in 1975. We identify mappings that produce no conflicts in the network and mappings that produce a maximum number of conflicts. The analysis of mappings for some typical applications shows that an initial allocation of data to memory modules determines the contention within the network for all iterations of the algorithm. For the case of the FFT and the bitonic sort algorithm executed on a shared-memory architecture, we prove that if no conflicts are produced during the first iteration of the algorithm, then no conflicts are produced during any other iteration. Moreover, if a maximum number of conflicts are produced during the first iteration, then a maximum number of conflicts are produced during all other iterations of the algorithm. For the d-dimensional grid computations where communication is required with 2d nearest neighbors, we prove that if the initial allocation produces no conflicts within the network, then communication with all the neighbors is conflict-free. Also, if the initial allocation produces a maximum number of conflicts, then communication with all the neighbors is maximum-conflict. We show that the omega network cannot produce without conflicts some of the bit-permute mappings such as the perfect shuffle and the bit reversal. The network can produce both of these mappings provided that data items are accessed from memories according to a specific skewed scheme.

Some parallel sorts on a mesh-connected processor array and their time efficiency

For each value to be sorted in the process of the parallel bubble sort computation, we evaluate the exact time necessary to route the value to its final position. Using this evaluation we design some efficient parallel sorting algorithms that can be implemented on a mesh-connected processor array and analyze their time complexities. Our algorithms are some combinations of the parallel bubble sorts in different directions, and their control hardware is very simple. Although the time complexities of our algorithms are O(N 1 2 log N) , they are as fast as the implementations of Batcher's bitonic sort and odd-even merge sort on the mesh-connected processor array for practical values of N, 1 ≤ N ≤ 128 2. We also show a parallel sort that is very fast in the average case for practical values of N, 1 ≤ N ≤ 128 2.

The automated proof of a trace transformation for a bitonic sort

In his third volume of The Art of Computer Programming, Knuth presents Batcher's bitonic sorting network. With concurrency, this sorting network can be executed in logarithmic time. Knuth suggests a formal argument for the correctness of the bitonic sorting algorithm (as an exercise), but addresses the question of concurrency only informally. We develop a program for the bitonic sort by (1) deriving a stepwise refinement from Knuth's informal description of the algorithm, (2) deriving from the refinement a sequential execution or ‘trace’ of order O ( n log n) in the length n of the sequence to be sorted, and (3) transforming the sequential trace into a parallel trace of order O(log n) while preserving its semantics. We shall be informal in Steps 1 and 2—although these steps can be formalized. But we will provide a formal treatment of Step 3 and report on the certification of this treatment in a mechanized logic. This work is a contribution to the optimization of programs (via concurrency) through transformation and the automation of program proofs.

K-Fold bitonic sort on a mesh-connected parallel computer

External sorting is very expensive; the problem of sorting more data items than the number of PEs is thus important. Let us assume that each PE has m memory locations for storing data items in addition to some working registers. We adapt Batcher's bitonic sort [1] to process k* N data items, where k ~< m, on a two-dimensional mesh-connected parallel computer with N processing elements. Performances are analyzed and compared between two algorithms of different data folding schemes. On a two-dimensional mesh, the location of each PE is uniquely represented by an ordered pair (i, j) of integers. The PE placed at the coordinates (i, j) is generally connected to four other PEs at (i + 1, j), (i 1, j), (i, j + 1), and (i, j 1). Assuming no 'wrap-around' connections, those PEs at the boundaries are connected only to two or three other PEs. In this paper we shall consider a SIMD computer of a square, mesh-connected array of N PEs. A sorting problem on a parallel computer can be defined as reordering data items according to a pre-determined PE indexing. The PE index function can be thought of as a one-to-one mapping from the coordinate space {0, 1 . . . . . ~ } x {0, 1 . . . . . ~ } onto the index space {0, 1 . . . . . N 1}. Discussed in [6] are three PE index functions: (1) row-major indexing, (2) shuffled row-major indexing, and (3) snake-like row-major indexing. Examples of these index functions are shown in Fig. 1 for a 4 x 4 mesh. Batcher's bitonic sort, based on pair-wise compare-and-interchange operations, can be informally represented as a sorting network [4]. As shown in Fig. 2, it requires a total of log N merging stages to sort a sequence of N elements by merging sub-sequences of lengths 1, 2 . . . . . and so on. (We use the notation log for log 2 throughout this paper.) Two adapted algorithms of the bitonic sort can be found in [6,5] which require O(¢'-N-) time to sort N data items using N PEs on a mesh-connected parallel computer. One maps the bitonic sort relatively straightforwardly onto a computer with the shuffled row-major PE indexing [6]. The other adapts, with some additional complications, the bitonic sort to a computer with the row-major PE indexing [5]. Knuth [4] pointed out that any sorting network for N data items can be generalized to sort k * N data items if the comparison operations are replaced by k-way merge operations. One implementation, using merge-splitting operations, of Knuth 's generalization was proposed for several sorting algorithms by

Particle simulation of 3D galactic hydrodynamics on the ICL DAP

A non-self-gravitating galactic hydrodynamics code based on a quasi-particle technique and making use of a mesh for force evaluation and sorting purposes is described. The short-range nature of the interparticle pressure forces, coupled with the use of a mesh allows a particularly fast algorithm. The 3D representation of the galaxy is mapped onto the “3D” main store of the ICL DAP in a natural way, the 2 spatial dimensions in the plane of the galaxy becoming the 2 dimensions of the processor plane on the DAP and the third dimension varying within individual processor storage elements. This leads to a fairly straightforward implementation and a high degree of parallelism in the crucial parts of the code. The particle shuffling which is necessary after each timestep is facilitated by the use of a parallel variant of the bitonic sorting algorithm. Some results of simulations using a 63 × 63 × 16 mesh and about 50000 particles to follow the evolution of a model disk galaxy are presented.

A Robust Sorting Network

Beginning with the recently introduced balanced sorting network, we propose a shuffle-exchange type layout consisting of a single block with the output recirculated back as input until sorting is achieved. Although this network has essentially the same performance bounds as Batcher's bitonic sort, our design has the property that no comparator in the network is critical in the sense that any faulty comparator can be bypassed without disturbing the functionality of the network (just its speed). The novelty of the design is that the robustness is derived from the underlying algorithm. The network will sort in the presence of many faulty comparators. Moreover, of the (N log N)/2 comparators, only N pairs of comparators are critical. That is, the network fails only when both comparators in a pair fail. Our results enable one to build large sorting networks on a single wafer so that a high percentage of the fabricated wafers can be used; some of the wafers will sort very quickly (the ones with no faulty components), most will sort at somewhat slower than optimal speeds, but only a few will fail to be useful as sorting networks (due to too many badly placed faults).