Hierarchical Parallel Research Articles

Plasticine

Reconfigurable architectures have gained popularity in recent years as they allow the design of energy-efficient accelerators. Fine-grain fabrics (e.g. FPGAs) have traditionally suffered from performance and power inefficiencies due to bit-level reconfigurable abstractions. Both fine-grain and coarse-grain architectures (e.g. CGRAs) traditionally require low level programming and suffer from long compilation times. We address both challenges with Plasticine, a new spatially reconfigurable architecture designed to efficiently execute applications composed of parallel patterns. Parallel patterns have emerged from recent research on parallel programming as powerful, high-level abstractions that can elegantly capture data locality, memory access patterns, and parallelism across a wide range of dense and sparse applications. We motivate Plasticine by first observing key application characteristics captured by parallel patterns that are amenable to hardware acceleration, such as hierarchical parallelism, data locality, memory access patterns, and control flow. Based on these observations, we architect Plasticine as a collection of Pattern Compute Units and Pattern Memory Units. Pattern Compute Units are multi-stage pipelines of reconfigurable SIMD functional units that can efficiently execute nested patterns. Data locality is exploited in Pattern Memory Units using banked scratchpad memories and configurable address decoders. Multiple on-chip address generators and scatter-gather engines make efficient use of DRAM bandwidth by supporting a large number of outstanding memory requests, memory coalescing, and burst mode for dense accesses. Plasticine has an area footprint of 113 mm2 in a 28nm process, and consumes a maximum power of 49 W at a 1 GHz clock. Using a cycle-accurate simulator, we demonstrate that Plasticine provides an improvement of up to 76.9x in performance-per-Watt over a conventional FPGA over a wide range of dense and sparse applications.

Basker: Parallel sparse LU factorization utilizing hierarchical parallelism and data layouts

Abstract Transient simulation in circuit simulation tools, such as SPICE and Xyce, depend on scalable and robust sparse LU factorizations for efficient numerical simulation of circuits and power grids. As the need for simulations of very large circuits grow, the prevalence of multicore architectures enable us to use shared memory parallel algorithms for such simulations. A parallel factorization is a critical component of such shared memory parallel simulations. We develop a parallel sparse factorization algorithm that can solve problems from circuit simulations efficiently, and map well to architectural features. This new factorization algorithm exposes hierarchical parallelism to accommodate irregular structure that arise in our target problems. It also uses a hierarchical two-dimensional data layout which reduces synchronization costs and maps to memory hierarchy found in multicore processors. We present an OpenMP based implementation of the parallel algorithm in a new multithreaded solver called Basker in the Trilinos framework. We present performance evaluations of Basker on the Intel SandyBridge and Xeon Phi platforms using circuit and power grid matrices taken from the University of Florida sparse matrix collection and from Xyce circuit simulation. Basker achieves a geometric mean speedup of 5.91× on CPU (16 cores) and 7.4× on Xeon Phi (32 cores) relative to state-of-the-art solver KLU. Basker outperforms Intel MKL Pardiso solver (PMKL) by as much as 30× on CPU (16 cores) and 7.5× on Xeon Phi (32 cores) for low fill-in circuit matrices. Furthermore, Basker provides 5.4× speedup on a challenging matrix sequence taken from an actual Xyce simulation.

High-Speed Target Tracking System Based on a Hierarchical Parallel Vision Processor and Gray-Level LBP Algorithm

Visual target tracking has made significant advances in past decades. However, fast and robust vision target tracking systems are still greatly demanded. This paper proposes a novel high-speed target tracking system based on hierarchical parallel vision processor architecture. This system contains three main parts: 1) a CMOS image sensor; 2) a vision processor; and 3) an actuator with two degrees of freedom. The vision processor integrates a pixel-parallel processing element (PE) array, a row-parallel row processor (RP) array, dual-core microprocessor unit (MPU) and motor controller. The PE array and RP array can speed up low-level and middle-level image processing operations by ${O(M^{2})}$ and ${O}$ ( ${M}$ ), respectively. The MPU is responsible for the high-level image processing and the overall chip management. A novel tracking algorithm based on a gray-level local binary pattern descriptor is proposed. The descriptor describes not only local texture feature but also distribution of luminance. The algorithm increases the robustness of the tracking system under low resolution scenery and complex background. It can be carried out by the vision processor with very high efficiency. Experiment results demonstrate that the system can track a fast moving target under complex conditions and the vision processor can achieve over 2000 frames/s processing speed of the target tracking algorithm with $ {750\times 480}$ image resolution.

EQE: An open‐source density functional embedding theory code for the condensed phase

AbstractIn this work, we present the main features and algorithmic details of a novel implementation of the frozen density embedding (FDE) formulation of subsystem density functional theory (DFT) that is specifically designed to enable ab initio molecular dynamics (AIMD) simulations of large‐scale condensed‐phase systems containing 1000s of atoms. This code (available at http://eqe.rutgers.edu) has been given the moniker of embedded Quantum ESPRESSO (eQE) as it is a generalization of the open‐source Quantum ESPRESSO (QE) suite of programs. The strengths of eQE reside in a hierarchical parallelization scheme that allows for an efficient and fully self‐consistent treatment of the electronic structure (via the addition of an additional DIIS extrapolation layer) while simultaneously exploiting the inherent symmetries and periodicities in the system (via sampling of subsystem‐specific first Brillouin zones and utilization of subsystem‐specific basis sets). While bulk liquids and molecular crystals are two classes of systems that exemplify the utility of the FDE approach (as these systems can be partitioned into weakly interacting subunits), we show that eQE has significantly extended this regime of applicability by outperforming standard semilocal Kohn–Sham DFT (KS‐DFT) for large‐scale heterogeneous catalysts with quite different layer‐specific electronic structure and intrinsic periodicities. eQE features very favorable strong parallel scaling for a model system of bulk liquid water composed of 256 water molecules, which allows for a significant decrease in the overall time to solution when compared to KS‐DFT. We show that eQE achieves speedups greater than one order of magnitude ( ) when performing AIMD simulations of such large‐scale condensed‐phase systems as: (1) molecular liquids via bulk liquid water represented by 1024 independent water molecules (3072 atoms with a 25.3× speedup over KS‐DFT), (2) polypeptide/biomolecule solvation via (gly)6 solvated in (H2O)395 (1230 atoms with a 38.6× speedup over KS‐DFT), and (3) molecular crystals via a 3 × 3 × 3 periodic supercell of pentacene (1940 atoms with a 12.0× speedup over KS‐DFT). These results represent a significant improvement over the current state‐of‐the‐art and now enable subsystem DFT‐based AIMD simulations of realistically sized condensed‐phase systems of interest throughout chemistry, physics, and materials science.

A Two-Layered Parallel Static Security Assessment for Large-Scale Grids Based on GPU

The development of smart grid and the increasing scale of power system brings more and more pressure to the conventional power system simulators. The graphic processing unit which features the massive concurrent threads and excellent floating point performance brings a new chance to the area of power system simulation. This paper introduces a hierarchical parallel LU decomposition algorithm based on stratified elimination tree, and develops a graphic processing unit (GPU)-based parallel linear system solver on the algorithm. To boost the effectiveness of the solver, a hybrid improved matrix ordering algorithm is proposed which can reduce the height of the elimination tree. On the basis of the solver, this paper implements a coarse-grained task parallelism which can analyze several contingencies of static security assessment (SSA) in parallel, and develops a two-layered parallel SSA program. The performance of the proposed parallel SSA program is benchmarked on a Tesla C2050 GPU using several real world cases, whose scale is up to 21801 nodes, compared with a serial SSA program which runs on a Xeon E5620 CPU. The results of the case studies show that the introduced method’s speedup reaches 9.2 times, which can greatly accelerate SSA solution.

H-PARAFAC: Hierarchical Parallel Factor Analysis of Multidimensional Big Data

It has long been an important issue in various disciplines to examine massive multidimensional data superimposed by a high level of noises and interferences by extracting the embedded multi-way factors. With the quick increases of data scales and dimensions in the big data era, research challenges arise in order to (1) reflect the dynamics of large tensors while introducing no significant distortions in the factorization procedure and (2) handle influences of the noises in sophisticated applications. A hierarchical parallel processing framework over a GPU cluster, namely H-PARAFAC, has been developed to enable scalable factorization of large tensors upon a “divide-and-conquer” theory for Parallel Factor Analysis (PARAFAC). The H-PARAFAC framework incorporates a coarse-grained model for coordinating the processing of sub-tensors and a fine-grained parallel model for computing each sub-tensor and fusing sub-factors. Experimental results indicate that (1) the proposed method breaks the limitation on the scale of multidimensional data to be factorized and dramatically outperforms the traditional counterparts in terms of both scalability and efficiency, e.g., the runtime increases in the order of $n^2$ when the data volume increases in the order of $n^3$ , (2) H-PARAFAC has potentials in refraining the influences of significant noises, and (3) H-PARAFAC is far superior to the conventional window-based counterparts in preserving the features of multiple modes of large tensors.

A Scalable Parallel Architecture Based on Many-Core Processors for Generating HTTP Traffic

The past years have witnessed the significant development of the Internet. Numerous emerging network architectures and protocols have triggered the demand for traffic generators which stand in stark contrast to previous schemes. Namely, fixed test content is inefficient in the presence of such a dynamic and realistic demand. Moreover, the requirement of high-performance has raised the stakes on developing a new concurrent system. In this paper, we present a hierarchical parallel design for a Web traffic generator on a TILERAGX36 processor, called TGMP. We discuss the challenges in developing its hierarchical architectural design, and elaborate on its implementation details. Specifically, in order to generate a realistic network workload over a long and large time scale, we propose a user-control scheme based on cubic spline interpolation. To better improve the scalability of the system and satisfy the required flow rate, we adopt techniques, including optimization of parameters under the Linux kernel, event-driven concurrency, and parallel architectures of a TILERAGX36 processor. The experimental results demonstrate that TGMP is able to create real traffic and simulate 50,000 users accessing the Web server simultaneously.

Fast and Adaptive 3D Reconstruction With Extensively High Completeness

The seed-and-expand scheme is appropriate for multiple view stereo, since it can build dense point clouds adaptively by avoiding unnecessary computation. However, due to the irregularity of the algorithm, it is not suitable for parallel computing on general public utilities (GPU). This paper is the first attempt to implement the irregular seed-and-expand method on GPU for multiple view stereo problems. Meanwhile, a hierarchical parallel computing architecture is also proposed to maximize the usage of both CPU and GPU. The adaptivity of the seed-and-expand scheme is pushed further by processing a pixel several rounds while, in order to maintain regularity for GPU implementation, every seed has exactly the same behavior in a single round of optimization. The high adaptivity also improves the robustness of the proposed method, thus aggressive matching score and a view selection method can be used to improve the reconstruction completeness extensively, without smearing out local details and lowering the accuracy. Compared with the state of the art, the proposed method achieves higher accuracy and completeness on standard datasets. The proposed method is also very fast. It is maximally five times faster than other methods running on a CPU and is on par with the regular depth map-based methods on GPU, which are naturally suitable for GPU acceleration.

Some remarks about a community open source Lagrangian pollutant transport and dispersion model

Nowadays fishes and mussels farming is very important, from an economical point of view, for the local social background of the Bay of Naples. Hence, the accurate forecast of marine pollution becomes crucial to have reliable evaluation of its adverse effects on coastal inhabitants’ health. The use of connected smart devices for monitoring the sea water pollution is getting harder because of the saline environment, the network availability and the maintain and calibration costs2. To this purpose, we designed and implemented WaComM (Water Community Model), a community open source model for sea pollutants transport and dispersion. WaComM is a model component of a scientific workflow which allows to perform, on a dedicated computational infrastructure, numerical simulations providing spatial and temporal high-resolution predictions of weather and marine conditions of the Bay of Naples leveraging on the cloud based31 FACE-IT workflow engine27. In this paper we present some remarks about the development of WaComM, using hierarchical parallelism which implies distributed memory, shared memory and GPGPUs. Some numerical details are also discussed.

Toward Dynamic Load Balancing across OpenMP Thread Teams for Irregular Workloads

In the field of high performance computing, massively-parallel many-core processors such as Intel Xeon Phi coprocessors are becoming popular because they can significantly accelerate various applications. In order to efficiently parallelize applications for such many-core processors, several high-level programming models have been proposed. The de facto standard programming model mainly for shared-memory parallel processing is OpenMP. For hierarchical parallel processing, OpenMP version 4.0 or later allows programmers to create multiple thread teams. Each thread team contains a bunch of newly-created synchronizable threads. When multiple thread teams are used to execute an application, it is important to have dynamic load balancing across thread teams, since static load balancing easily encounters load imbalance across teams, and thus degrades performance. In this paper, we first motivate our work by clarifying the benefit of using multiple thread teams to execute an irregular workload on a many-core processor. Then, we demonstrate that dynamic load balancing across those thread teams has a potential of significantly improving the performance of irregular workloads on a many-core processor, with considering the scheduling overhead. Although such a dynamic load balancing mechanism has not been provided by the current OpenMP specification, the benefits of dynamic load balancing across thread teams are discussed through experiments using the Intel Xeon Phi coprocessor. We evaluate the performance gain of dynamic load balancing across thread teams using a ray tracing code. The results show that such a dynamic load balancing mechanism can improve the performance by up to 14% compared to static load balancing across teams, with considering scheduling overhead.

Massively parallel algorithm and implementation of RI-MP2 energy calculation for peta-scale many-core supercomputers

A new parallel algorithm and its implementation for the RI-MP2 energy calculation utilizing peta-flop-class many-core supercomputers are presented. Some improvements from the previous algorithm (J. Chem. Theory Comput. 2013, 9, 5373) have been performed: (1) a dual-level hierarchical parallelization scheme that enables the use of more than 10,000 Message Passing Interface (MPI) processes and (2) a new data communication scheme that reduces network communication overhead. A multi-node and multi-GPU implementation of the present algorithm is presented for calculations on a central processing unit (CPU)/graphics processing unit (GPU) hybrid supercomputer. Benchmark results of the new algorithm and its implementation using the K computer (CPU clustering system) and TSUBAME 2.5 (CPU/GPU hybrid system) demonstrate high efficiency. The peak performance of 3.1 PFLOPS is attained using 80,199 nodes of the K computer. The peak performance of the multi-node and multi-GPU implementation is 514 TFLOPS using 1349 nodes and 4047 GPUs of TSUBAME 2.5. © 2016 Wiley Periodicals, Inc.

A low‐communication, parallel algorithm for solving PDEs based on range decomposition

SummaryThis paper proposes a new, low‐communication algorithm for solving PDEs on massively parallel computers. The range decomposition (RD) algorithm exposes coarse‐grain parallelism by applying nested iteration and adaptive mesh refinement locally before performing a global communication step. Just a few such steps are observed to be sufficient to obtain accuracy within a small multiple of discretization error. The target applications are petascale and exascale machines, where hierarchical parallelism is required and traditional parallel numerical PDE communication patterns are costly because of message latency. The RD algorithm uses a partition of unity to equally distribute the error, and thus, the work. The computational advantages of this approach are that the decomposed problems can be solved in parallel without any communication until the partitioned solutions are summed. This offers potential advantages in the paradigm of expensive communication but very cheap computation. This paper introduces the method and explains the details of the communication step. Two performance models are developed, showing that the latency cost associated with a traditional parallel implementation of nested iteration is proportional to log(P)2, whereas the RD method reduces the communication latency to log(P), while maintaining similar bandwidth costs. Numerical results for two problems, Laplace and advection diffusion, demonstrate the enhanced performance, and a heuristic argument explains why the method converges quickly. Copyright © 2016 John Wiley & Sons, Ltd.

Multi-Variable Agents Decomposition for DCOPs

The application of DCOP models to large problems faces two main limitations: (i) Modeling limitations, as each agent can handle only a single variable of the problem; and (ii) Resolution limitations, as current approaches do not exploit the local problem structure withineach agent. This paper proposes a novel Multi-Variable Agent (MVA) DCOP decompositiontechnique, which: (i) Exploits the co-locality of each agent's variables, allowing us to adopt efficient centralized techniques within each agent; (ii) Enables the use of hierarchical parallel models and proposes the use of GPUs; and (iii) Reduces the amount of computation and communication required in several classes of DCOP algorithms.

CUDAMPF: a multi-tiered parallel framework for accelerating protein sequence search in HMMER on CUDA-enabled GPU.

BackgroundHMMER software suite is widely used for analysis of homologous protein and nucleotide sequences with high sensitivity. The latest version of hmmsearch in HMMER 3.x, utilizes heuristic-pipeline which consists of MSV/SSV (Multiple/Single ungapped Segment Viterbi) stage, P7Viterbi stage and the Forward scoring stage to accelerate homology detection. Since the latest version is highly optimized for performance on modern multi-core CPUs with SSE capabilities, only a few acceleration attempts report speedup. However, the most compute intensive tasks within the pipeline (viz., MSV/SSV and P7Viterbi stages) still stand to benefit from the computational capabilities of massively parallel processors.ResultsA Multi-Tiered Parallel Framework (CUDAMPF) implemented on CUDA-enabled GPUs presented here, offers a finer-grained parallelism for MSV/SSV and Viterbi algorithms. We couple SIMT (Single Instruction Multiple Threads) mechanism with SIMD (Single Instructions Multiple Data) video instructions with warp-synchronism to achieve high-throughput processing and eliminate thread idling. We also propose a hardware-aware optimal allocation scheme of scarce resources like on-chip memory and caches in order to boost performance and scalability of CUDAMPF. In addition, runtime compilation via NVRTC available with CUDA 7.0 is incorporated into the presented framework that not only helps unroll innermost loop to yield upto 2 to 3-fold speedup than static compilation but also enables dynamic loading and switching of kernels depending on the query model size, in order to achieve optimal performance.ConclusionsCUDAMPF is designed as a hardware-aware parallel framework for accelerating computational hotspots within the hmmsearch pipeline as well as other sequence alignment applications. It achieves significant speedup by exploiting hierarchical parallelism on single GPU and takes full advantage of limited resources based on their own performance features. In addition to exceeding performance of other acceleration attempts, comprehensive evaluations against high-end CPUs (Intel i5, i7 and Xeon) shows that CUDAMPF yields upto 440 GCUPS for SSV, 277 GCUPS for MSV and 14.3 GCUPS for P7Viterbi all with 100 % accuracy, which translates to a maximum speedup of 37.5, 23.1 and 11.6-fold for MSV, SSV and P7Viterbi respectively. The source code is available at https://github.com/Super-Hippo/CUDAMPF.

Performance optimizations for scalable implicit RANS calculations with SU2

In this paper, we present single- and multi-node optimizations of SU2, a widely-used, open-source Computational Fluid Dynamics application, aimed at improving performance and scalability for implicit Reynolds-averaged Navier–Stokes calculations on unstructured grids. Typical industry-standard implementations are currently limited by unstructured accesses, variable degrees of parallelism, as well as the global synchronizations inherent in traditionally used Krylov linear solvers. Therefore, we rely on aggressive single-node optimizations, such as hierarchical parallelism, dynamic threading, compacted memory layout, and vectorization, along with a communication-friendly agglomeration (geometric) linear multigrid solver. Based on results with the well-known ONERA M6 geometry, our single core and shared memory optimizations result in a speedup of 2.6X on the latest 14-core Intel® Xeon™ 11Intel, Xeon, and Intel Xeon Phi are trademarks of Intel Corporation in the U.S. and/or other countries. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to http://www.intel.com/performance E5-2697v3 processor when compared to the baseline SU2 implementation with 14 MPI ranks. In multi-node settings, the hybrid OpenMP+MPI multigrid implementation achieves 2X higher parallel efficiency on 256 nodes over conventional Krylov-based (GMRES) methods.

A Scalable Parallel PWTD-Accelerated SIE Solver for Analyzing Transient Scattering From Electrically Large Objects

A scalable parallel plane-wave time-domain (PWTD) algorithm for efficient and accurate analysis of transient scattering from electrically large objects is presented. The algorithm produces scalable communication patterns on very large numbers of processors by leveraging two mechanisms: 1) a hierarchical parallelization strategy to evenly distribute the computation and memory loads at all levels of the PWTD tree among processors and 2) a novel asynchronous communication scheme to reduce the cost and memory requirement of the communications between the processors. The efficiency and accuracy of the algorithm are demonstrated through its applications to the analysis of transient scattering from a perfect electrically conducting (PEC) sphere with a diameter of 70 wavelengths and a PEC square plate with a dimension of 160 wavelengths. Furthermore, the proposed algorithm is used to analyze transient fields scattered from realistic airplane and helicopter models under high frequency excitation.

On the hierarchical parallelization of ab initio simulations

A hierarchical parallelization has been implemented in a new unified code PIMD-SMASH for ab initio simulation where the replicas and the Born–Oppenheimer forces are parallelized. It is demonstrated that ab initio path integral molecular dynamics simulations can be carried out very efficiently for systems up to a few tens of water molecules. The code was then used to study a Diels–Alder reaction of cyclopentadiene and butenone by ab initio string method. A reduction in the reaction energy barrier is found in the presence of hydrogen-bonded water, in accordance with experiment.

A Tile-Based EGPU with a Fused Universal Processing Engine and Graphics Coprocessor Cluster

As various applied sensors have been integrated into embedded devices, the Embedded Graphics Processing Unit (EGPU) has assumed more processing tasks, which requires an EGPU with higher performance. A tile-based EGPU is proposed that can be used in both general-purpose computing and 3D graphics rendering. With fused, scalable, and hierarchical parallelism architecture, the EGPU has the ability to address nearly 100 million vertices or fragments and achieves 1 GFLOPS per second at a clock frequency of 200 MHz. A fused and scalable architecture, constituted by Universal Processing Engine (UPE) and Graphics Coprocessor Cluster (GCC), ensures that the EGPU can adapt to various graphic processing scenes and situations, achieving more efficient rendering. Moreover, hierarchical parallelism is implemented via the UPE. Additionally, tiling brings a significant reduction in both system memory bandwidth and power consumption. A 0.18 µm technology library is used for timing and power analysis. The area of the proposed EGPU is 6.5 mm∗6.5 mm, and its power consumption is approximately 349.318 mW. Experimental results demonstrate that the proposed EGPU can be used in a System on Chip (SoC) configuration connected to sensors to accelerate its processing and create a proper balance between performance and cost.

Hierarchical Parallel Genetic Optimization Fuzzy ARTMAP Ensemble

In this paper, a framework for designing optimum pattern classifiers is proposed. The fuzzy ARTMAP (FAM) is first used as a base classifier. Multiple FAM classifiers form an ensemble to improve classification accuracy. Multi-objective genetic algorithms (GAs) are then used to search for the best combinations of variables, for the FAM classifiers. Based on the population of potential solutions, another GA selects the best combination of FAM classifiers to create an ensemble. Individual decisions are combined using a probabilistic voting scheme. To increase the inter-classifier diversity, a hierarchical parallel GA variant and a negative correlation method is employed during the genetic optimization phase for the ensemble evaluation. The proposed framework is evaluated using benchmark and real-world data sets, and the results compared with literature. Results positively indicate the proposed framework is effective in undertaking data classification tasks.

High-Level Heterogeneous and Hierarchical Parallel Systems (HLPGPU 2014)

High-Level Heterogeneous and Hierarchical Parallel Systems (HLPGPU 2014)