Data-parallel Paradigm Research Articles

Hybrid deterministic non-deterministic data-parallel algorithm for real-time unmanned aerial vehicle trajectory planning in CUDA

The development of autonomous Unmanned Aerial Vehicles (UAVs) is a priority to many civilian and military organizations. Real time optimal trajectory planning is an essential element for the autonomy of UAVs. The use of metaheuristic algorithms for solving such complex optimization problems with non-linearity and multimodality has gained popularity recently. In this paper, we use a non-deterministic Flower Pollination Algorithm (FPA) to deal with the problem's complexity and compute feasible and quasi-optimal trajectories for fixed-wing UAVs in complex 3D environments, taking into account the vehicle's flight properties. The global optimization algorithm is improved with the addition of a deterministic 2-opt local search providing a significant improvement. To achieve real-time performance, the proposed trajectory planner in implemented and parallelized following the data-parallel paradigm on a Graphics Processing Unit (GPU) using the Compute Unified Device Architecture (CUDA) resulting in a 253.4x speedup compared to the sequential implementation on CPU. The parallel implementation is able to compute quasi-optimal trajectories in just 0.369 s.

To share or not to share vector registers?

Query execution techniques in database systems constantly adapt to novel hardware features to achieve high query performance, in particular for analytical queries. In recent years, vectorization based on the Single Instruction Multiple Data parallel paradigm has been established as a state-of-the-art approach to increase single-query performance. However, since concurrent analytical queries running in parallel often access the same columns and perform a same set of vectorized operations, data accesses and computations among different queries may be executed redundantly. Various techniques have already been proposed to avoid such redundancy, ranging from concurrent scans via the construction of materialized views to applying multiple query optimization techniques. Continuing this line of research, we investigate the opportunity of sharing vector registers for concurrently running queries in analytical scenarios in this paper. In particular, our novel sharing approach relies on processing data elements of different queries together within a single vector register. As we are going to show, sharing vector registers to optimize the execution of concurrent analytical queries can be very beneficial in single-threaded as well as multi-thread environments. Therefore, we demonstrate the feasibility and applicability of such a novel work sharing strategy and thus open up a wide spectrum of future research opportunities.

Joint coflow routing and scheduling in leaf-spine data centers

Communication in data centers often involves many parallel flows that all share the same performance goal (e.g. to minimize the average completion time). A useful abstraction, coflow, is proposed to express the communication requirements of prevalent data parallel paradigms such as MapReduce and Spark. The multiple coflow routing and scheduling problem makes it challenging to derive a good theoretical performance ratio, as coexisting coflows may compete for the same network resources such as link bandwidths. In this paper, we focus on the coflow problem in one popular data center infrastructure: the Leaf-Spine topology. We first formulate the problem and study the path selection issue on this two-tier structure. In order to minimize the average coflow completion time (CCT), we propose the Multi-hop Coflow Routing and Scheduling strategy (MCRS) and prove that our method has a reasonably good competitive ratio. Extensive experiments and large-scale simulations show that MCRS outperforms the state-of-the-art heuristic schemes under the Leaf-Spine topology.

DGST: Efficient and scalable suffix tree construction on distributed data-parallel platforms

The suffix tree is a fundamental data structure for string processing. It is widely used in many important scenarios such as text processing, information retrieval, and bioinformatics. With the rapid growth of data volume, constructing the suffix tree for large-scale datasets is very time-consuming. To solve this problem, a number of MPI-based parallel algorithms were proposed, but they have limitations in fault tolerance and scalability for large-scale datasets. Recently, there are ever-increasing application demands on efficient algorithms for constructing the suffix tree for large-scale datasets on distributed data-parallel platforms, such as Hadoop and Spark. In this paper, we present DGST, which is an efficient and scalable algorithm for generalized suffix tree construction on distributed data-parallel platforms. DGST consists of two major stages: parallel sub-tree partitioning and parallel sub-tree construction. We first design a novel data partitioning strategy for both two stages in the data-parallel paradigm. Then, we propose an efficient sub-tree partitioning algorithm based on parallel frequency counting. To improve the load balance and amortize the disk I/O costs, we propose an efficient Bin-Packing and Number-Partitioning based task allocation strategy for the sub-tree construction. At the sub-tree construction stage, we further propose a novel data structure LCP-Range and a multi-way LCP-Merge sorting algorithm for parallel LCP array construction. The experimental results on Apache Spark reveal that DGST outperforms the state-of-the-art ERa algorithm with approximately 3 times speedup on both the DNA and English text datasets. Furthermore, DGST achieves near-linear data and node scalability.

Big data mining with parallel computing: A comparison of distributed and MapReduce methodologies

Mining with big data or big data mining has become an active research area. It is very difficult using current methodologies and data mining software tools for a single personal computer to efficiently deal with very large datasets. The parallel and cloud computing platforms are considered a better solution for big data mining. The concept of parallel computing is based on dividing a large problem into smaller ones and each of them is carried out by one single processor individually. In addition, these processes are performed concurrently in a distributed and parallel manner. There are two common methodologies used to tackle the big data problem. The first one is the distributed procedure based on the data parallelism paradigm, where a given big dataset can be manually divided into n subsets, and n algorithms are respectively executed for the corresponding n subsets. The final result can be obtained from a combination of the outputs produced by the n algorithms. The second one is the MapReduce based procedure under the cloud computing platform. This procedure is composed of the map and reduce processes, in which the former performs filtering and sorting and the later performs a summary operation in order to produce the final result. In this paper, we aim to compare the performance differences between the distributed and MapReduce methodologies over large scale datasets in terms of mining accuracy and efficiency. The experiments are based on four large scale datasets, which are used for the data classification problems. The results show that the classification performances of the MapReduce based procedure are very stable no matter how many computer nodes are used, better than the baseline single machine and distributed procedures except for the class imbalance dataset. In addition, the MapReduce procedure requires the least computational cost to process these big datasets.

Object Identification in Binary Tomographic Images Using GPGPUs

The authors present a hybrid OpenCL CPU/GPU algorithm for identification of connected structures inside black and white 3D scientific data. This algorithm exploits parallelism both at CPU and GPGPU levels, but the work is predominantly done in GPUs. The underlying context of this work is the structural characterization of composite materials via tomography. The algorithm allows us to later infer location and morphology of objects inside composite materials. Moreover, execution times are very low thus allowing us to process large data sets, but within acceptable running times. Intermediate solutions are computed independently over a partition of the spatial domain, following the data parallelism paradigm, and then integrated both at GPU and CPU levels, using parallel multi-cores. The authors consistently explore parallelism both at the CPU level, by allowing the CPU stage to run in multiple concurrent threads, and at the GPU level with massive parallelism and concurrent data transfers and kernel executions.

Scheduling parallel programs by work stealing with private deques

Work stealing has proven to be an effective method for scheduling parallel programs on multicore computers. To achieve high performance, work stealing distributes tasks between concurrent queues, called deques, which are assigned to each processor. Each processor operates on its deque locally except when performing load balancing via steals. Unfortunately, concurrent deques suffer from two limitations: 1) local deque operations require expensive memory fences in modern weak-memory architectures, 2) they can be very difficult to extend to support various optimizations and flexible forms of task distribution strategies needed many applications, e.g., those that do not fit nicely into the divide-and-conquer, nested data parallel paradigm. For these reasons, there has been a lot recent interest in implementations of work stealing with non-concurrent deques, where deques remain entirely private to each processor and load balancing is performed via message passing. Private deques eliminate the need for memory fences from local operations and enable the design and implementation of efficient techniques for reducing task-creation overheads and improving task distribution. These advantages, however, come at the cost of communication. It is not known whether work stealing with private deques enjoys the theoretical guarantees of concurrent deques and whether they can be effective in practice. In this paper, we propose two work-stealing algorithms with private deques and prove that the algorithms guarantee similar theoretical bounds as work stealing with concurrent deques. For the analysis, we use a probabilistic model and consider a new parameter, the branching depth of the computation. We present an implementation of the algorithm as a C++ library and show that it compares well to Cilk on a range of benchmarks. Since our approach relies on private deques, it enables implementing flexible task creation and distribution strategies. As a specific example, we show how to implement task coalescing and steal-half strategies, which can be important in fine-grain, non-divide-and-conquer algorithms such as graph algorithms, and apply them to the depth-first-search problem.

A data parallel strategy for aligning multiple biological sequences on multi-core computers

In this paper, we address the large-scale biological sequence alignment problem, which has an increasing demand in computational biology. We employ data parallelism paradigm that is suitable for handling large-scale processing on multi-core computers to achieve a high degree of parallelism. Using the data parallelism paradigm, we propose a general strategy which can be used to speed up any multiple sequence alignment method. We applied five different clustering algorithms in our strategy and implemented rigorous tests on an 8-core computer using four traditional benchmarks and artificially generated sequences. The results show that our multi-core-based implementations can achieve up to 151-fold improvements in execution time while losing 2.19% accuracy on average. The source code of the proposed strategy, together with the test sets used in our analysis, is available on request.

Strategies for Parallel Genetic Computation of Optimization Problems

ABSTRACTThis paper explores the efficiency of various strategies for parallel genetic computation of optimization problems on multicomputer platforms. The strategies for designing parallel genetic algorithms on multicomputer platforms are investigated considering the correlations of the algorithmic and the architectural spaces. Two parallel genetic computational models are considered based on the manager/workers and Single Program Multiple Data parallel paradigms. Parallelism profiling and analysis of parallel system performance have been made for the different parallel computational models in respect to the scalability of the application and the scalability of the parallel machine size.

Falcon: A Production Quality Distributed Memory Reservoir Simulator

Summary We describe a new production model, Falcon, that has achieved speeds on parallel computers that are 100 times faster on real world problems than current production models on a vector computer. Falcon has been used to conduct the largest, geostatistical reservoir study ever conducted within Amoco. In this paper we discuss the following: Falcon's data parallel paradigm with FORTRAN 90 and high performance FORTRAN (HPF); its single program, multiple data (SPMD) paradigm with message passing; efficient memory management that enables simulation of enormous studies; a numerical formulation that reconciles the generalized compositional approach (based on component masses and pressure) with earlier approaches (based on pressures and saturations), in a more general and more efficient approach. We also discuss Falcon's scalability up to 512 processor nodes and performance (timings and memory) achieved on a number of parallel platforms, including Cray Research's T3D and T3E, SGI's Power Challenge and Origin 2000, Thinking Machines' CM5, and IBM's SP2. Falcon also runs on single processor computers such as PC's and IBM's RS6000. We discuss a new parallel linear solver technology based on a fully parallel scalable implementation of incomplete lower-upper (ILU) preconditioning coupled with a GMRES or Orthomin iteration process. This naturally ordered global ILU preconditioner is scalable to hundreds of processors, efficiently solving the matrix problems arising from large scale simulations The use of the techniques described in this paper has enabled us to run problem sizes of up to 16.5 million gridblocks. Falcon was used to simulate fifty geostatistically derived realizations of a large, black oil waterflood system. The realizations, each with 2.3 million cells and 1,039 wells, took an average of 4.2 hours to execute on a 128-node CM5 computer, thus enabling the simulation study to finish in less than a month. In this field study, we bypassed upscaling through the use of fine vertical resolution gridding. Our focus has been on the applicability of Falcon to real world problems. Falcon can be used for modeling both small and very large reservoirs, including reservoirs characterized by geostatistics. It can be used to simulate black oil, gas/water, and dry gas reservoirs. And, a fully compositional feature is being developed.

Comparison of MIMD and SIMD strategies for Monte Carlo modelling of kinetically coupled reactions

The atomic-level description of a complex heavy hydrocarbon feedstock and its complete temporal product behavior is a challenging and computationally intensive problem which requires robust reaction simulations to accurately account for kinetic interactions and efficient utilization of available computer hardware for solution. Herein we discuss the parallel implementation of different Monte Carlo reaction algorithms for two prototype kinetic problems and examine their efficiency on MIMD and SIMD parallel architectures. The concepts developed are subsequently applied to a complex resid pyrolysis model. Both fixed- and variable-time step Monte Carlo approaches to the stochastic simulation algorithm were investigated. The prototype kinetic problems consisted of A i⇌B i with Ldngmuir-Hinshelwood-Hougen-Watson kinetics for i = 1–3 and a hypothetical 1–5 ring aro matic hydrogenation simulation. The two parallel architectures utilized were BBN's TC2000 (MIMD) and Thinking Machines CM-2 (SIMD). A new modelling approach that stochastically samples the reaction environment to provide an estimate of species' interactions is introduced. This “partial system” approach exhibited excellent parallel efficiency ( > 90%) on the TC2000. The simple prototype A i⇌B i kinetic problem was also successfully implemented on the SIMD CM-2. While the 1–5 ring hydrogenation problem efficiently took advantage of the MIMD architecture, its performance on the CM-2 was quite poor due to the the lack of a match between the data focus of complex model simulations and the CM-2 data parallel paradigm. The resid pyrolysis model performed extremely well on the MIMD machine.

C++ and Massively Parallel Computers

Our goal is to apply the software engineering advantages of object‐oriented programming to the raw power of massively parallel architectures. To do this we have constructed a hierarchy of C++ classes to support the data‐parallel paradigm. Feasibility studies and initial coding can be supported by any serial machine that has a C++ compiler. Parallel execution requires an extended Cfront, which understands the data‐parallel classes and generates C* code. (C* is a data‐parallel superset of ANSI C developed by Thinking Machines Corporation). This approach provides potential portability across parallel architectures and leverages the existing compiler technology for translating data‐parallel programs onto both SIMD and MIMD hardware.