Parallel Fast Fourier Transform Research Articles

Low-Power Very-Large-Scale Integration Implementation of Fault-Tolerant Parallel Real Fast Fourier Transform Architectures Using Error Correction Codes and Algorithm-Based Fault-Tolerant Techniques

As technology advances, electronic circuits are more vulnerable to errors. Soft errors are one among them that causes the degradation of a circuit’s reliability. In many applications, protecting critical modules is of main concern. One such module is Fast Fourier Transform (FFT). Real FFT (RFFT) is a memory-based FFT architecture. RFFT architecture can be optimized by its processing element through employing several types of adder and multipliers and an optimized memory usage. It has been seen that various blocks operate simultaneously in many applications. For the protection of parallel FFTs using conventional Error Correction Codes (ECCs), algorithmic-based fault tolerance (ABFT) techniques like Parseval checks and its combination are seen. In this brief, the protection schemes are applied to the single RAM-based parallel RFFTs and dual RAM-based parallel RFFTs. This work is implemented on platforms such as field programmable gate arrays (FPGAs) using Verilog HDL and on application-specific integrated circuit (ASIC) using a cadence encounter digital IC implementation tool. The synthesis results, including LUTs, slices registers, LUT–Flip-Flop pairs, and the frequency of two types of protected parallel RFFTs, are analyzed, along with the existing FFTs. The two proposed architectures with the combined protection scheme Parity-SOS-ECC present an 88% and 33% reduction in area overhead when compared to the existing parallel RFFTs. The performance metrics like area, power, delay, and power delay product (PDP) in an ASIC of 45 nm and 90 nm technology are evaluated, and the proposed single RAM-based parallel RFFTs architecture presents a 62.93% and 57.56% improvement of PDP in 45 nm technology and a 67.20% and 60.31% improvement of PDP in 90 nm technology compared to the dual RAM-based parallel RFFTs and the existing architecture, respectively.

MFFT: A GPU Accelerated Highly Efficient Mixed-Precision Large-Scale FFT Framework

Fast Fourier transform (FFT) is widely used in computing applications in large-scale parallel programs, and data communication is the main performance bottleneck of FFT and seriously affects its parallel efficiency. To tackle this problem, we propose a new large-scale FFT framework, MFFT, which optimizes parallel FFT with a new mixed-precision optimization technique, adopting the “high precision computation, low precision communication” strategy. To enable “low precision communication”, we propose a shared-exponent floating-point number compression technique, which reduces the volume of data communication, while maintaining higher accuracy. In addition, we apply a two-phase normalization technique to further reduce the round-off error. Based on the mixed-precision MFFT framework, we apply several optimization techniques to improve the performance, such as streaming of GPU kernels, MPI message combination, kernel optimization, and memory optimization. We evaluate MFFT on a system with 4,096 GPUs. The results show that shared-exponent MFFT is 1.23 × faster than that of double-precision MFFT on average, and double-precision MFFT achieves performance 3.53× and 9.48× on average higher than open source library 2Decomp&FFT (CPU-based version) and heFFTe (AMD GPU-based version), respectively. The parallel efficiency of double-precision MFFT increased from 53.2% to 78.1% compared with 2Decomp&FFT, and shared-exponent MFFT further increases the parallel efficiency to 83.8%.

A New Multichannel Parallel Real-time FFT Algorithm for a Solar Radio Observation System Based on FPGA

The real-time fast Fourier transform (FFT) is the essential algorithm for signal processing in a solar radio receiver. However, field-programmable gate array (FPGA) computation resources have become the limitation of real-time processing of signals with increasing time and spectral resolutions. It is necessary to design a real-time parallel FFT algorithm with reduced resource occupation in the development of future receiving systems. In this paper, we developed a multichannel parallel FFT algorithm named the multichannel parallel real-time fast Fourier transform (MPR-FFT), which can greatly reduce FPGA resource occupation while increasing the real-time processing speed. In this algorithm, the 4L simultaneous N-point FFTs are first converted into L simultaneous 4N-point FFTs. Fusion processing is then performed to obtain the 4 ∗ L ∗ N-point spectrum. This method has been used in developing a solar radio spectrometer, which works in the frequency range of 0.5–15 GHz in the Chashan Observatory. In this spectrometer, 16 channel MPR-FFT with 8k-point data is realized in a Xilinx UltraScale KU115 FPGA. The MPR-FFT algorithm reduced the computational resources to a large extent compared to the Cooley-Tukey-based parallel FFT method; for instance, the Look-Up-Table, Look-Up-Table RAM, Flip-Flop, and Digital Signal Process slices were reduced by 37%, 50%, 17%, and 2.48%, respectively. Although the MPR-FFT consumes 14 block RAM resources more than the Cooley-Tukey-based parallel FFT, the MPR-FFT algorithm presents an overall reduction in resource usage.

Parallel FFT algorithms for high-order approximations on three-dimensional compact stencils

The recent development of multicore technologies on modern desktop computers makes parallelization of the proposed numerical approaches a priority in algorithmic research. The main performance improvement of personal computers in the upcoming years will be made based on the increasing number of cores on modern CPUs. This shifts the focus of algorithmic research from the development of sequential numerical methods to parallel methodology. This paper presents an efficient parallel direct algorithm with near-optimal complexity for the compact fourth and sixth-order approximation of the three-dimensional Helmholtz equations (Turkel et al., 2013) with the problem coefficient depending on only one of the coordinate directions. The developed method is based on a combination of the separation of variables technique and a Fast Fourier Transform (FFT) type method. Similar direct solvers for the lower-order approximations of the two and three-dimensional Helmholtz equation were considered in several previous publications by the authors and other researchers (see, e.g. Gryazin et al. (2000); Gryazin (2014); Elman and O’Leary (1998); Elman and O’Leary (1999); Toivanen and Wolfmayr (2020)). The authors also consider a generalization of the presented algorithm to the solution of a wide class of linear systems obtained from approximation on the compact 27-point three-dimensional stencils on the rectangular grids with similar requirements on the stencil coefficients. The general restrictions on the coefficients in the considered class of compact schemes are developed and presented. This class includes the second, fourth and sixth-order compact approximation schemes for the three-dimensional Helmholtz equation considered in this paper and our previous publications (Gryazin et al., 2000; Gryazin, 2014; Gryazin, 2014). As an example of the diversity of applications of the developed general method, the direct parallel implementation of a compact fourth-order approximation scheme for a convection–diffusion equation is considered. Another goal of this paper is to investigate the scalability of the proposed technique in the case of a large linear system using different parallel programming extensions. The results of the implementation of this method in OpenMP, MPI and hybrid programming environments on the multicore computers and multiple node clusters are presented and discussed. The results demonstrate the high efficiency of the proposed direct solvers for many important applications on the structured grid with the corresponding 27-diagonal matrices of sizes up to 1011 by 1011.

PARALLEL PIPELINED MULTI RADIX VARIABLE LENGTH FAST FOURIER TRANSFORM ARCHITECTURE

There is an enormous demand for high speed data communication or high speed internet using Long Term Evolution (LTE) or LTE-Advanced communication methods. To achieve the high speed data rate in the receiver side, it is necessary to achieve high throughput in the Fast Fourier transform (FFT) architecture. Hence there is demand to improve the throughput of the FFT architectures used for high speed data communication. The FFT architecture is designed and optimized for LTE-A applications. However, the high throughput has been achieved by sacrificing hardware resources of the Field Programmable Gate Array (FPGA). Many fixed and variable length FFT processors are proposed by the researchers with improved performance focusing either on algorithmic modifications, novel architectural optimizations or radix selection. Among these FFT processors, the goal and main objectives of this paper are: 1. To design and implement a pipelined FFT architecture to give high throughput for LTE-A MIMO applications 2. To develop an intellectual property (IP) core for FFT computation with variable FFT size 3. To propose a parallel implementation to increase the performance of LTE-A baseband processing system. In an Orthogonal Frequency Division Multiplexing (OFDM) baseband communication system, FFT operation is one of the highest computationally serious tasks which directly influence the communication system performance factor. The baseband hardware has to be capable as well as efficient enough to calculate FFT in specific timing restrictions. Thus, the parallel pipelined multi radix Variable Length FFT architectures for LTE-A Multiple-Input-Multiple-Output (MIMO) applications have been designed. The proposed FFT architecture delivers a throughput of 550 MSPS at the maximum clock rate of 550 MHz. The proposed FFT support the FFT length of 64, 128, 256, 512, 1024 and 2048 for LTE-Advanced MIMO standard. The proposed FFT architecture has been implemented in Xilinx Artix-7 FPGA device and the performance metrics have been analysed. Even though the proposed FFT architecture consumes additional Block-RAMs (BRAM) and quite an amount of Xilinx-Xtreme Digital Signal Processor (DSP)/ DSP48 resources, the Power Delay Product (PDP) of the proposed FFT is excellent compared to the existing FFT architectures. The proposed multi radix mixed 2/3/5 parallel pipelined 2048 point FFT architectures exhibits very less latency of 11.382 µs at 550 MHz clock frequency compared to the existing system with the latency of 56.88 µs at 200 MHz clock frequency. Moreover, the designed FFT architecture utilizes 96 Xilinx Xtreme DSP blocks 25040 clock cycles to complete the FFT operation with an excellent throughput of 2200 MSPS with FFT computation time of 45.528 µs at the clock frequency of 550 MHz for the 4x4 MIMO- OFDM architecture. Therefore, the designed FFT provides high throughput rate in order to meet the modern wireless standard specifications. The proposed FFT architecture outperforms well in terms high throughput, low latency and better PDP with extra hardware as trade-off for both for Single FFT and for MIMO technology.

Two Parallel Pipelined FFT Architecture After Third Stage for Low Complexity and Latency

Complexity is the major issue in the development of pipelined Fast Fourier Transform (FFT) structure. A novel parallel pipelined FFT design constructed for low complexity and Latency. The proposed structure efficiently uses pipeline and parallel pipeline in the design. The first three stages of the structure follow the pipelining and from fourth stage there are two parallel paths simultaneously compute four complex additions, the first three single path stages that reduces complexity, parallel pipeline stages at the end of structure also reduces the Latency. The proposed structure is an optimized solution to reduce the trade-off between Latency and complexity. The comparative performance of the proposed structure with several other existing structures shows that it has Low latency with small cost of hardware.

Reconfigurable Multi-Butterfly Parallel Radix-r FFT Processor

The design of reconfigurable multi-butterfly parallel radix-r FFT (Fast Fourier Transform) processors is proposed. FFT is widely used in signal processing, and the application needs real-time and high performance, while most of the traditional designs are limited to the power of two, which wastes the buffers and multipliers in big data. In response to the problem, we improve the parallel FFT algorithm with the design of reconfigurable control machine combined with buffer/multiplier, and the cost function with the input of radix/number/paddling number/time consuming is deduced. Constrained with the number of buffer and multipliers, the radix and number can be computed with the optimum cost function, and the resolution space of computing performance and hardware cost is presented. The proposed guarantees the real-time performance with better flexibility compared with the previous literature, and the comparison also suggests the effectiveness of the design.

Parallelized Tube Rearrangement Algorithm for Online Video Synopsis

Video synopsis allows us to analyze security videos efficiently by condensing or shortening a long video into a short one. To generate a condensed video, moving objects (a.k.a. object tubes) in the video are rearranged in the temporal domain using a predefined objective function. The objective function consists of several energy terms which play important roles in making a visually appealing condensed video. One of the energy terms, collision energy, creates a bottleneck in the computation because it requires two object tubes to calculate the degree of collision between them. Existing approaches try to reduce the computation time of the collision energy calculation by reducing the number of tubes processed at once. However, those approaches are not sufficient to generate condensed video when the number of object tubes becomes large. In this letter, we propose a fast Fourier transform (FFT)-based parallelized tube rearrangement algorithm. To take advantage of both parallel processing and FFT, we represent object tubes as three-dimensional binary matrices (occupation matrices). An objective function of the tube rearrangement problem is defined on the occupation matrix, and a starting position for each tube in the temporal domain is then determined by optimizing the objective function. Throughout the experiments, the proposed algorithm took a much shorter time to condense the video than existing algorithms, while other performance metrics were similar.

Efficient GPS Signal Acquisition Based on Parallel FFT in FPGA

Acquisition in GPS system is crucial step to get the code phase in Pseudo Random Noise (PRN) in continuous tracking of the satellite. Fast Fourier Transform (FFT) in conventional GPS receivers usually is implemented in convolution algorithm in the acquisition process. In this paper, a parallel FFT algorithm is described that segments the fast Fourier transform algorithm into groups of identical parallel operations that can be performed concurrently and independently. The proposed algorithm is carried out using the LABVIEW FPGA module and those obtained from the GPS Signal simulation using LABVIEW verify the results. This process execution provides a superior performance in fast acquisition and determinism compared with conventional processors based software solutions.

Generic functional parallel algorithms: scan and FFT

Parallel programming, whether imperative or functional, has long focused on arrays as the central data type. Meanwhile, typed functional programming has explored a variety of data types, including lists and various forms of trees. Generic functional programming decomposes these data types into a small set of fundamental building blocks: sum, product, composition, and their associated identities. Definitions over these few fundamental type constructions then automatically assemble into algorithms for an infinite variety of data types—some familiar and some new. This paper presents generic functional formulations for two important and well-known classes of parallel algorithms: parallel scan (generalized prefix sum) and fast Fourier transform (FFT). Notably, arrays play no role in these formulations. Consequent benefits include a simpler and more compositional style, much use of common algebraic patterns and freedom from possibility of run-time indexing errors. The functional generic style also clearly reveals deep commonality among what otherwise appear to be quite different algorithms. Instantiating the generic formulations, two well-known algorithms for each of parallel scan and FFT naturally emerge, as well as two possibly new algorithms.

Implementation of parallel FFT algorithm on a ternary optical computer

The Fast Fourier Transform (FFT) is widely used in modern digital signal processing systems. In order to improve the efficiency of FFT, parallel methods are often used to speed up FFT in high-speed real-time application environments. However, due to the restrictions of size, energy consumption, heat dissipation etc., traditional electronic methods are becoming unsuitable for FFT applications in certain areas, such as aviation, aerospace, etc. Due to its characteristics of a massive number of data bits and low energy consumption, the Ternary Optical Computer (TOC) is a potential solution for these special cases. To verify this possibility, we studied the design and implementation of a high-speed parallel FFT on a TOC. Through analysis of the traditional radix-2, radix-4, and radix-8 DIT FFT operation processes, several FFT algorithms with higher parallelism, implemented on TOC, are designed by taking advantage of the characteristics of TOC. The implementation processes of these algorithms and the differences between them are presented. Meanwhile, the clock cycles and hardware resources required for each algorithm are also discussed. Simulation results reveal that the FFT implementation method is accurate. The algorithms require less power and fewer clock cycles when implemented on a TOC compared to the traditional methods implemented on an FPGA. This provides a new possible solution for high-speed low-power FFT implementation.

Stage-Dependent DSP Operation Range Clipping-Induced Bit Resolution Reductions of Full Parallel 64-Point FFTs Incorporated in FPGA-Based Optical OFDM Receivers

The fast Fourier transform (FFT) is the fundamental algorithm at the heart of an optical OFDM (OOFDM) transceiver. The high digital signal processing (DSP) complexity has become one of the most significant obstacles to experimentally demonstrating real-time high-capacity OOFDM transceivers. In this paper, stage-dependent clipping of FFT DSP operation dynamic range is proposed and extensively explored, for the first time, based on which an improved stage-dependent minimum bit resolution map is numerically identified by taking into account the DSP operation dynamic range-clipping and precision of FFT DSP operations. The validity and high accuracy of the identified minimum bit resolution map is experimentally verified in 25 km SSMF OOFDM transmission systems based on intensity modulation and direct detection. Experimental results show that, compared to a previously reported FFT stage-dependent bit resolution map including the FFT DSP operation precision only, the improved minimum bit resolution map offers significant bit resolution reductions of up to 3-bits for full-parallel pipelined 64-point FFT architectures. As a direct result, an approximately 30% reduction in FPGA logic resource usage is achievable compared to the spiral FPGA design.

Efficient Circuit for Parallel Bit Reversal

Bit reversal is an essential part of the fast Fourier transform (FFT). However, compared to the amount of works on FFT architectures, far fewer works are dedicated to bit-reversal circuits until recent years. In this brief, the minimum latency and memory required for calculating the bit reversal of continuous-flow parallel data are formulated. The formulas are generic for all power of two parallelism, including the serial bit reversal. Furthermore, an efficient circuit for calculating the parallel bit reversal is proposed. The circuit not only achieves the lowest latency but also uses the minimum memory. This is achieved by breaking the bit-reversal permutation into two subpermutations, which are implemented by a sub-bit-reversal module and a group of $P$ buffer banks. In addition, two commutators are adopted to access the $P$ buffer banks efficiently. The proposed circuit is simple and efficient for reordering the output samples of parallel pipelined FFT processors.

Multi-mode parallel and folded VLSI architectures for 1D-fast Fourier transform

The modern real time applications like orthogonal frequency division multiplexing and etc., demand high performance fast Fourier transform (FFT) design with less area and clock cycles. This paper proposes efficient FFT VLSI architectures using folded/parallel implementation. In the proposed folded FFT architecture, the number of cycles required to complete the operation is less than single path delay feedback (SDF)/multi-path delay commutator (MDC) architectures. In the proposed parallel FFT architecture, N-point FFT is implemented by using one N/2-point FFT without much extra hardware. Both the proposed architectures are implemented for radix-2, 22, and 4 using 45nm technology library. The proposed parallel architecture achieves 56.7% and 40.6% of area reduction as compared with the existing parallel architecture based 16-point radix-2 and radix-22 DIF FFTs respectively. The proposed folded architecture achieves 65.5%, 51.1%, and 35.8% of worst path delay reduction as compared with the existing SDF based 16-point radix-2, radix-22, and radix-4 DIF FFTs respectively.

High performance Python for direct numerical simulations of turbulent flows

Direct Numerical Simulations (DNS) of the Navier Stokes equations is an invaluable research tool in fluid dynamics. Still, there are few publicly available research codes and, due to the heavy number crunching implied, available codes are usually written in low-level languages such as C/C++ or Fortran. In this paper we describe a pure scientific Python pseudo-spectral DNS code that nearly matches the performance of C++ for thousands of processors and billions of unknowns. We also describe a version optimized through Cython, that is found to match the speed of C++. The solvers are written from scratch in Python, both the mesh, the MPI domain decomposition, and the temporal integrators. The solvers have been verified and benchmarked on the Shaheen supercomputer at the KAUST supercomputing laboratory, and we are able to show very good scaling up to several thousand cores.A very important part of the implementation is the mesh decomposition (we implement both slab and pencil decompositions) and 3D parallel Fast Fourier Transforms (FFT). The mesh decomposition and FFT routines have been implemented in Python using serial FFT routines (either NumPy, pyFFTW or any other serial FFT module), NumPy array manipulations and with MPI communications handled by MPI for Python (mpi4py). We show how we are able to execute a 3D parallel FFT in Python for a slab mesh decomposition using 4 lines of compact Python code, for which the parallel performance on Shaheen is found to be slightly better than similar routines provided through the FFTW library. For a pencil mesh decomposition 7 lines of code is required to execute a transform.

Hybrid and 4-D FFT implementations of an open-source parallel FFT package OpenFFT

The fast Fourier transform (FFT) is a fundamental kernel in a wide variety of science and engineering fields from electronic structure calculations to medical imaging. OpenFFT is an open-source package for parallel 3-D FFTs, built on a domain decomposition method with optimal communication for minimizing the volume of communication. However, OpenFFT version 1.1 does not support hybrid calculations for fully utilizing common multi-core machines and parallel 4-D FFTs. In addition, there exist several state-of-the-art open-source packages for parallel FFTs, and performance comparison among them would be interesting and helpful for potential users. In this paper, we (1) extend the functionality of OpenFFT by developing a hybrid MPI/OpenMP version and a parallel 4-D FFT, and (2) conduct performance comparison among the currently available parallel FFT packages. In the former, we first analyze the computational parts of OpenFFT to explore the opportunities for taking advantage of fine-grained parallelization with OpenMP. Based on the analysis, we implement and make comparison to choose between the two most promising hybrid options. We then develop the parallel 4-D FFT by extending the hybrid implementation of the parallel 3-D FFT. The decomposition method for 4-D FFTs preserves its 3-D original features to maximize the localization of data when transposing. In the latter, we evaluate and compare the performance of FFTE, P3DFFT, PFFT, 2DECOMP&FFT, and OpenFFT for both 3-D and 4-D FFTs on a number of different machines with varied computational scales. The evaluation results assert the benefit of the hybrid feature in improving the scalability of OpenFFT, and confirm its minimal volume of communication for 4-D FFTs in practice. Also, although no significant difference is observed in overall performance in general, there are specific cases when some packages have the edge over the others.

English

A well-known problem of orthogonal frequency division multiplexing (OFDM) is its sensitivity to offset between the transmitted and received carrier frequencies. In OFDM communication systems, the frequency offsets in mobile radio channels deform the orthogonality between subcarriers which causes inter carrier interference (ICI). ICI causes power leakage among subcarriers and it further degrades the system performance. In this paper a novel bandwidth efficient method for combating the effects of ICI is presented. In the present scheme, the self cancellation mechanism is used to compress ICI signals without any equalization. At transmitter and receiver parallel Fast Fourier Transform (FFT) factorization is performed using Radix-2 decimation in frequency (DIF) FFT algorithm derived from the well known Cooley-turkey factorization. The proposed scheme cancels the ICI coefficient effectively and further improves system performance as shown through extensive simulations. Key words: Orthogonal frequency division multiplexing (OFDM), intercarrier interference, carrier frequency offset, Radix2, fast Fourier transform (FFT).

Computation-Communication Overlap Techniques for Parallel Spectral Calculations in Gyrokinetic Vlasov Simulations

One of the important phenomena in magnetically-confined fusion plasma is plasma turbulence, which causes particle and heat transport and degrades plasma confinement. To address multi-scale turbulence including temporal and spatial scales of electrons and ions, we extend our gyrokinetic Vlasov simulation code GKV to run efficiently on peta-scale supercomputers. A key numerical technique is the parallel Fast Fourier Transform (FFT) required for parallel spectral calculations, where masking of the cost of inter-node transpose communications is essential to improve strong scaling. To mask communication costs, computation-communication overlap techniques are applied for FFTs and transpose with the help of the hybrid parallelization of message passing interface and open multi-processing. Integrated overlaps including whole spectral calculation procedures show better scaling than simple overlaps of FFTs and transpose. The masking of communication costs significantly improves strong scaling of the GKV code, and makes substantial speed-up toward multi-scale turbulence simulations.

PFFT: An Extension of FFTW to Massively Parallel Architectures

We present an MPI based software library for computing fast Fourier transforms (FFTs) on massively parallel, distributed memory architectures based on the Message Passing Interface standard (MPI). Similar to established transpose FFT algorithms, we propose a parallel FFT framework that is based on a combination of local FFTs, local data permutations, and global data transpositions. This framework can be generalized to arbitrary multidimensional data and process meshes. All performance-relevant building blocks can be implemented with the help of the FFTW software library. Therefore, our library offers great flexibility and portable performance. Similarly to FFTW, we are able to compute FFTs of complex data, real data, and even- or odd-symmetric real data. All the transforms can be performed completely in place. Furthermore, we propose an algorithm to calculate pruned FFTs more efficiently on distributed memory architectures. For example, we provide performance measurements of FFTs of sizes between $512^3$ ...

Pipelined Parallel FFT Architectures via Folding Transformation

This paper presents a novel approach to develop parallel pipelined architectures for the fast Fourier transform (FFT). A formal procedure for designing FFT architectures using folding transformation and register minimization techniques is proposed. Novel parallel-pipelined architectures for the computation of complex and real valued fast Fourier transform are derived. For complex valued Fourier transform (CFFT), the proposed architecture takes advantage of under utilized hardware in the serial architecture to derive <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$L$</tex></formula> -parallel architectures without increasing the hardware complexity by a factor of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$L$</tex></formula> . The operating frequency of the proposed architecture can be decreased which in turn reduces the power consumption. Further, this paper presents new parallel-pipelined architectures for the computation of real-valued fast Fourier transform (RFFT). The proposed architectures exploit redundancy in the computation of FFT samples to reduce the hardware complexity. A comparison is drawn between the proposed designs and the previous architectures. The power consumption can be reduced up to 37% and 50% in 2-parallel CFFT and RFFT architectures, respectively. The output samples are obtained in a scrambled order in the proposed architectures. Circuits to reorder these scrambled output sequences to a desired order are presented.