Bitonic Sorting Network Research Articles

Federated Data Cleaning: Collaborative and Privacy-Preserving Data Cleaning for Edge Intelligence

As an important driving factor of emerging Internet-of-Things (IoT) applications, machine learning algorithms are currently facing the challenge of how to “clean” data noise, that is introduced during the training process (e.g., asynchronous execution and lossy data compression and quantization). In an attempt to guarantee data quality, various data cleaning approaches have been proposed to filter out abnormal data entries based on the global data distribution. However, most existing data cleaning approaches are based on a centralized paradigm and thus cannot be applied to future edge-based IoT applications, where each edge node (EN) has only a limited view of the global data distribution. Moreover, the increasing demand for privacy preservation largely prevents ENs from combining their data for centralized cleaning. In this study, we propose a federated data cleaning protocol, coined as FedClean, for edge intelligence (EI) scenarios that is designed to achieve data cleaning without compromising data privacy. More specifically, different ENs first generate Boolean shares of their data and distribute them to two noncolluding servers. These two servers then run the FedClean protocol to privately and efficiently compute the attribute value frequency (AVF) scores of the collected data entries, which are then sorted in ascending order via a bitonic sorting network without revealing their values. As a result, data entries with lower AVF scores are considered as abnormal and filtered out. The security, efficiency, and effectiveness of the proposed approach are then demonstrated via concrete security analysis and comprehensive experiments.

RTHS: A Low-Cost High-Performance Real-Time Hardware Sorter, Using a Multidimensional Sorting Algorithm

This paper proposes a novel hardware-based multidimensional sorting algorithm and its respective architecture, called real-time hardware sorter (RTHS), for emerging data intensive processing applications where performance and resource conservation are serious concerns. The basic idea behind RTHS is to reduce the hardware complexity of parallel hardware sorting architectures (PHSAs) through a high-performance scalable matrix-based sorting method. The proposed method can also be used for implementing Min/Max queues or finding the largest/smallest records exclusively in the big data application. Implementing the RTHS design on a Virtex-7 field-programmable gate array (FPGA) reveals that the number of lookup tables (LUTs) of the proposed method has decreased by 66.3% and 87.3% compared to the conventional Bitonic sorting network (CBSN) and the state-of-the-art PHSA, respectively. In addition, the number of required registers for the proposed method has decreased by 94.8% compared to the state-of-the-art PHSA.

Design space exploration of multi-core RTL via high level synthesis from OpenCL models

As more and more powerful integrated circuits are appearing on the market, more and more applications, with very different requirements and workloads, are making use of available computing power. Designing optimized accelerators that can meet particular requirements has always presented a tremendous challenge to hardware engineers. To do so, designers have to trade off performance for power consumption in a manner such that the final RTL consumes minimum energy to meet the required performance (e.g. FLOPS) target. Moreover, the growing trend towards heterogeneous platforms is crucial to meet time and power consumption constraints of high-performance computing (HPC) applications. The OpenCL parallel programming language and framework enables programming CPU, GPU and recently FPGAs using the high-level synthesis (HLS) methodology. This work presents a design space exploration flow based on execution time, resource utilization and power consumption of OpenCL kernels mapped on FPGAs using the Xilinx high-level synthesis tool chain. Our experiments suggest that the quality of generated solutions, in terms of performance-per-watt, can be determined using analytical formulas prior to implementation, thus enabling fast and accurate DSE by considering on-chip and off-chip sources of parallelism. Moreover, the automated flow suggests design hints to meet a given time constraint within available resources. The proposed technique is demonstrated by optimizing the well known bitonic sorting network from NVIDIA’s OpenCL benchmark. Our results report that FPGAs have at least 20% higher performance-per-watt with respect to two high-end GPUs manufactured in the same technology (28 nm). Additionally, FPGAs with more available resources and using a more modern process (20 nm) can outperform the tested GPUs while consuming at least 55% less power at the cost of more expensive devices.

The Half Cleaner Lemma: Constructing Efficient Interconnection Networks from Sorting Networks

In general, efficient non-blocking interconnection networks can be derived from sorting networks, and to this end, one may either follow the merge-based or the radix-based sorting paradigm. Both paradigms require special modifications to handle partial permutations. In this article, we present a general lemma about half cleaner modules that were introduced as building blocks in Batcher’s bitonic sorting network. This lemma is the key to prove the correctness of many known optimizations of interconnection networks. In particular, we first show how to use any ternary sorter and a half cleaner for implementing an efficient split module as required for radix-based sorting networks for partial permutations. Second, our lemma formally proves the correctness of another known optimization of the Batcher-Banyan network.

Computer Generation of High Throughput and Memory Efficient Sorting Designs on FPGA

Accelerating sorting using dedicated hardware to fully utilize the memory bandwidth for Big Data applications has gained much interest in the research community. Recently, parallel sorting networks have been widely employed in hardware implementations due to their high data parallelism and low control overhead. In this paper, we propose a systematic methodology for mapping large-scale bitonic sorting networks onto FPGA. To realize data permutations in the sorting network, we develop a novel RAM-based design by vertically “folding” the classic Clos network. By utilizing the proposed design for data permutation, we develop a hardware generator to automatically build bitonic sorting architectures on FPGAs. For given input size, data width and data parallelism, the hardware generator specializes both the datapath and the control unit for sorting and generates a design in high level hardware description language. We demonstrate trade-offs among throughput, latency and area using two illustrative sorting designs including a high throughput design and a resource efficient design. With a data parallelism of p (2 ≤ p ≤ N/2), the high throughput design sorts an N-key sequence with latency 6N=p + o(N), throughputp results per cycle and uses 6N + o(N) memory. This achieves optimal memory efficiency (defined as the ratio of throughput to the amount of on-chip memory used by the design) and outperforms the state-of-the-art. Experimental results show that the designs obtained by our proposed hardware generator achieve 49 to 112 percent improvement in energy efficiency and 56 to 430 percent higher memory efficiency compared with the state-of-the-art.

Streaming Sorting Networks

Sorting is a fundamental problem in computer science and has been studied extensively. Thus, a large variety of sorting methods exist for both software and hardware implementations. For the latter, there is a trade-off between the throughput achieved and the cost (i.e., the logic and storage invested to sort n elements). Two popular solutions are bitonic sorting networks with O ( n log 2 n ) logic and storage, which sort n elements per cycle, and linear sorters with O ( n ) logic and storage, which sort n elements per n cycles. In this article, we present new hardware structures that we call streaming sorting networks , which we derive through a mathematical formalism that we introduce, and an accompanying domain-specific hardware generator that translates our formal mathematical description into synthesizable RTL Verilog. With the new networks, we achieve novel and improved cost-performance trade-offs. For example, assuming that n is a two-power and w is any divisor of n , one class of these networks can sort in n /; w cycles with O ( w log 2 n ) logic and O ( n log 2 n ) storage; the other class that we present sorts in n log 2 n /; w cycles with O ( w ) logic and O ( n ) storage. We carefully analyze the performance of these networks and their cost at three levels of abstraction: (1) asymptotically, (2) exactly in terms of the number of basic elements needed, and (3) in terms of the resources required by the actual circuit when mapped to a field-programmable gate array. The accompanying hardware generator allows us to explore the entire design space, identify the Pareto-optimal solutions, and show superior cost-performance trade-offs compared to prior work.

Optimized Merge Sort on Modern Commodity Multi-core CPUs

Sorting is a kind of widely used basic algorithms. As the high performance computing devices are increasingly common, more and more modern commodity machines have the capability of parallel concurrent computing. A new implementation of sorting algorithms is proposed to harness the power of newer SIMD operations and multi-core computing provided by modern CPUs. The algorithm is hybrid by optimized bitonic sorting network and multi-way merge. New SIMD instructions provided by modern CPUs are used in the bitonic network implementation, which adopted a different method to arrange data so that the number of SIMD operations is reduced. Balanced binary trees are used in multi-way merge, which is also different with former implementations. Efforts are also paid on minimizing data moving in memory since merge sort is a kind of memory-bound application. The performance evaluation shows that the proposed algorithm is twice as fast as the sort function in C++ standard library when only single thread is used. It also outperforms radix sort implemented in Boost library.

Fast in‐place, comparison‐based sorting with CUDA: a study with bitonic sort

AbstractState‐of‐the‐art graphics processors provide high processing power and furthermore, the high programmability of GPUs offered by frameworks like CUDA (Compute Unified Device Architecture) increases their usability as high‐performance co‐processors for general‐purpose computing. Sorting is well investigated in Computer Science in general, but (because of this new field of application for GPUs) there is a demand for high‐performance parallel sorting algorithms that fit with the characteristics of the modern GPU‐architecture. We present a high‐performance in‐place implementation of Batcher's bitonic sorting networks for CUDA‐enabled GPUs. Therefore, we assigned compare/exchange operations to threads in a way that decreases low‐performance global‐memory access and makes efficient use of high‐performance shared memory. This greatly increases the performance of this in‐place, comparison‐based sorting algorithm. Our implementation outperforms all other algorithms in our tests when sorting 64‐bit keys. It is the fastest comparison‐based GPU sorting algorithm for 32‐bit keys, being only outperformed by (non‐comparison‐based) radix sort when sorting sequences larger than 223. Copyright © 2011 John Wiley & Sons, Ltd.

Sorting on GPUs for large scale datasets: A thorough comparison

Although sort has been extensively studied in many research works, it still remains a challenge in particular if we consider the implications of novel processor technologies such as manycores (i.e. GPUs, Cell/BE, multicore, etc.). In this paper, we compare different algorithms for sorting integers on stream multiprocessors and we discuss their viability on large datasets (such as those managed by search engines). In order to fully exploit the potentiality of the underlying architecture, we designed an optimized version of sorting network in the K-model, a novel computational model designed to consider all the important features of many-core architectures. According to K-model, our bitonic sorting network mapping improves the three main aspects of many-core architectures, i.e. the processors exploitation, and the on-chip/off-chip memory bandwidth utilization. Furthermore we are able to attain a space complexity of Θ(1). We experimentally compare our solution with state-of-the-art ones (namely, Quicksort and Radixsort) on GPUs. We also compute the complexity in the K-model for such algorithms. The conducted evaluation highlight that our bitonic sorting network is faster than Quicksort and slightly slower than radix, yet being an in-place solution it consumes less memory than both algorithms.

A calculus for hardware description

AbstractIn efforts to overcome the complexity of the syntax and the lack of formal semantics of conventional hardware description languages, a number of functional hardware description languages have been developed. Like conventional hardware description languages, however, functional hardware description languages eventually convert all source programs into netlists, which describe wire connections in hardware circuits at the lowest level and conceal all high-level descriptions written into source programs. We develop a calculus, calledlλ (linear lambda), which may serve as an intermediate functional language just above netlists in the hierarchy of hardware description languages. In order to support higher-order functions,lλ uses a linear type system, which enforces the linear use of variables of function type. The translation oflλ into structural descriptions of hardware circuits is sound and complete in the sense that it maps expressions only to realizable hardware circuits, and that every realizable hardware circuit has a corresponding expression inlλ. To illustrate the use oflλ as a practical intermediate language for hardware description, we design a simple hardware description language that extendslλ with polymorphism, and use it to implement a fast Fourier transform circuit and a bitonic sorting network.

Minimizing communication in the bitonic sort

This paper presents bitonic sorting schemes for special-purpose parallel architectures such as sorting networks and for general-purpose parallel architectures such as SIMD and/or MIMD computers. First, bitonic sorting algorithms for shared-memory SIMD and/or MIMD computers are developed. Shared-memory accesses through the interconnection network of shared memory SIMD and/or MIMD computers can be very time consuming. A scheme is introduced which reduces the number of such accesses. This scheme is based on the parity strategy which is the main idea of the paper. By reducing the communication through the network, a performance improvement is achieved. Second, a recirculating bitonic sorting network is presented, which is composed of one level of N/2 comparators plus an /spl Omega/-network of (log N-1) switch levels. This network reduces the cost complexity to O(N log N) compared with the O(N log/sup 2/ N) of the original bitonic sorting network, while preserving the same time complexity. Finally, a simplified multistage bitonic sorting network, is presented. For simplifying the interlevel wiring, the parity strategy is used, so N/2 keys are wired straight through the network.

An optimal hardware-algorithm for sorting using a fixed-size parallel sorting device

We present a hardware-algorithm for sorting N elements using either a p-sorter or a sorting network of fixed I/O size p while strictly enforcing conflict-free memory accesses. To the best of our knowledge, this is the first realistic design that achieves optimal time performance, running in /spl Theta/(NlogN/plogp) time for all ranges of N. Our result completely resolves the problem of designing an implementable, time-optimal algorithm for sorting N elements using a p-sorter. More importantly, however, our result shows that, in order to achieve optimal time performance, all that is needed is a sorting network of depth O(log/sup 2/p) such as, for example, Batcher's classic bitonic sorting network.

Parallel multiplication using fast sorting networks

A recent paper describes the use of Svoboda's binary counter in the construction of fast parallel multipliers. The resulting approach was shown to be faster than the conventional Dadda multiplier when the wordlength N was small. Unfortunately, the growth in the number of gates of that method was O(N/sup 3/) and the speed was O(N). In this paper, Batcher's bitonic sorting network and other efficient networks replace the Svoboda counter. The asymptotic growth rate in gates of these new methods is O(N/sup 2/ log/sup 2/ N), and the speed is O(log/sup 2/ N).

Design, analysis, and evaluation of concurrent checking sorting networks

In this brief, we propose two new concurrent error-detection (CED) schemes for a class of sorting networks, e.g., odd-even transposition, bitonic, and perfect shuffle sorting networks. A probabilistic method is developed to analyze the fault coverage, and the hardware overhead is evaluated. We first propose a CED scheme by which all errors caused by single faults in a concurrent checking sorting network can be detected. This scheme is the first one available to use significantly less hardware overhead than duplication without compromising throughput. From this scheme, we develop another fault detection scheme which sharply reduces the hardware overhead (using an additional 10%/spl sim/30% hardware) but still achieves virtually 1001 fault coverage.

An experimental ATM switch for BISDN studies

AbstractThis paper describes the architecture, functionality and performance of an experimental ATM switch being developed at the Telecom Australia Research Laboratories as part of its investigations into the broadband ISDN. The proposed switch architecture consists of parallel omega networks preceded by a Batcher bitonic sorting network. The switching fabric has no internal cell buffering. Cell buffering is provided only on the switch outputs for cells simultaneously contending for the same output port. The switch fabric and cell buffers include mechanisms for providing prioritized servicing of queued cells and prioritized discarding of cells based on priority fields contained within the cell header. Components of the switch are currently being implemented in 2 μm CMOS VLSI.

The automated proof of a trace transformation for a bitonic sort

In his third volume of The Art of Computer Programming, Knuth presents Batcher's bitonic sorting network. With concurrency, this sorting network can be executed in logarithmic time. Knuth suggests a formal argument for the correctness of the bitonic sorting algorithm (as an exercise), but addresses the question of concurrency only informally. We develop a program for the bitonic sort by (1) deriving a stepwise refinement from Knuth's informal description of the algorithm, (2) deriving from the refinement a sequential execution or ‘trace’ of order O ( n log n) in the length n of the sequence to be sorted, and (3) transforming the sequential trace into a parallel trace of order O(log n) while preserving its semantics. We shall be informal in Steps 1 and 2—although these steps can be formalized. But we will provide a formal treatment of Step 3 and report on the certification of this treatment in a mechanized logic. This work is a contribution to the optimization of programs (via concurrency) through transformation and the automation of program proofs.