Perfect Load Balancing Research Articles

Benchmarking 50-Photon Gaussian Boson Sampling on the Sunway TaihuLight

Boson sampling is expected to be an important milestone that will demonstrate quantum computational advantage (or quantum supremacy). This work establishes the benchmarking of Gaussian boson sampling (GBS) with threshold detection based on the Sunway TaihuLight supercomputer. To achieve the best performance and provide a competitive scenario for future quantum computing studies, the selected simulation algorithm is fully optimized based on a set of innovative approaches, including a parallel framework with almost perfect load balance and an instruction-level optimizing scheme based on a shortest-path-based instruction scheduling. In addition, data precision is carefully processed by an integer-instruction-based and multiple-precision fixed-point implementation, including 128- and 256-bit precison mode, which can be appropriately selected based on an adaptive precision optimizing scheme. Based on these methods, a highly efficient parallel quantum sampling algorithm is designed. The largest run enables us to obtain one Torontonian function of a <inline-formula><tex-math notation="LaTeX">$100\times 100$</tex-math></inline-formula> submatrix from 50-photon GBS within 20 hours in 128-bit precision and 2 days in 256-bit precision. To our knowledge, this was the largest quantum computing simulation based on Boson Sampling by using modern supercomputers.

A generic Private Information Retrieval scheme with parallel multi‐exponentiations on multicore processors

SummaryPrivate Information Retrieval (PIR) enables the data owners to share and/or retrieve data on remote repositories without leaking any information as to which a data item is requested. Although it is always possible to download the entire dataset, this is clearly a waste of bandwidth. A fundamental approach in the literature for PIR is exploiting homomorphic cryptosystems. In these approaches, not one but many modular exponentiations need to be computed and multiplied to obtain the desired result. This multi‐exponentiation operation can be implemented by exponentiating the bases to their corresponding exponents one‐by‐one. However, when the operation is considered as a whole, it can be performed in a more efficient way. Although individual exponentiations are pleasingly parallelizable, the combined multi‐exponentiation requires a careful parallel implementation. In this work, we propose a generic tensor‐based PIR scheme and efficient and novel techniques to parallelize multi‐exponentiations on multicore processors with perfect load balance. The experimental results show that our load balancing techniques make a parallel multi‐exponentiation up to %27 faster when the size of the bases and the exponents are 4096 bits and the number of threads is 16.

A load-balancing workload distribution scheme for three-body interaction computation on Graphics Processing Units (GPU)

Three-body effects play an important role for obtaining quantitatively high accuracy in a variety of molecular simulation applications. However, evaluation of three-body potentials is computationally costly, generally of O(N3) where N is the number of particles in a system. In this paper, we present a load-balancing workload distribution scheme for calculating three-body interactions by taking advantage of the Graphics Processing Units (GPU) architectures. Perfect load-balancing is achieved if N is not divisible by 3 and nearly perfect load-balancing is obtained if N is divisible by 3. The workload distribution scheme is particularly suitable for the GPU’s Single Instruction Multiple Threads (SIMT) architecture, where particle’s data accessed by threads can be coalesced into efficient memory transactions. We use two potential energy functions with three-body terms, the Axilrod–Teller potential and the Context-based Secondary Structure Potential, as examples to demonstrate the effectiveness of our workload distribution scheme.

Superset: a non-uniform replica placement strategy towards perfect load balance and fine-grained power proportionality

Load balance and power proportionality are both important aspects in constructing high-performance and cost-effective distributed storage systems. However, traditional replica placement strategies which provide load balance usually produce scattered replica layouts, thus disabling power proportionality, while recent strategies that enable power proportionality are typically based on uniform replication, thus compromising the ability of load balance. In this article, we introduce Superset (an organized non-uniform replica placement strategy) which takes both load balance and power proportionality into consideration. The main idea is to partition the whole system into several uniform replication based subsystems with the accommodated file subsets satisfying the `superset' condition. We proved Superset to be theoretically effective, and conducted a series of simulations with real-world distributions of data popularity. Our results show that, compared to state of the art solutions, Superset consumes less energy to fulfill the same performance requirement while offers higher performance subject to the same energy consumption constraint.

BALANCING THE LOAD: A VORONOI BASED SCHEME FOR PARALLEL COMPUTATIONS

The use of numerical simulations in science is ever increasing and with it the computational size. In many cases single processors are no longer adequate and simulations are run on multiple core machines or supercomputers. One of the key issues when running a simulation on multiple CPUs is maintaining a proper load balance throughout the run and minimizing communications between CPUs. We propose a novel method of utilizing a Voronoi diagram to achieve a nearly perfect load balance without the need of any global redistributions of data. As a show case, we implement our method in RICH, a 2D moving mesh hydrodynamical code, but it can be extended trivially to other codes in 2D or 3D. Our tests show that this method is indeed efficient and can be used in a large variety of existing hydrodynamical codes as well as other applications.

Efficient breadth first search on multi-GPU systems

Simple algorithms for the execution of a Breadth First Search on large graphs lead, running on clusters of GPUs, to a situation of load unbalance among threads and un-coalesced memory accesses, resulting in pretty low performances. To obtain a significant improvement on a single GPU and to scale by using multiple GPUs, we resort to a suitable combination of operations to rearrange data before processing them. We propose a novel technique for mapping threads to data that achieves a perfect load balance by leveraging prefix-sum and binary search operations. To reduce the communication overhead, we perform a pruning operation on the set of edges that needs to be exchanged at each BFS level. The result is an algorithm that exploits at its best the parallelism available on a single GPU and minimizes communication among GPUs. We show that a cluster of GPUs can efficiently perform a distributed BFS on graphs with billions of nodes.

Juggle: addressing extrinsic load imbalances in SPMD applications on multicore computers

We investigate proactive dynamic load balancing on multicore systems, in which threads are continually migrated to reduce the impact of processor/thread mismatches. Our goal is to enhance the flexibility of the SPMD-style programming model and enable SPMD applications to run efficiently in multiprogrammed environments. We present Juggle, a practical decentralized, user-space implementation of a proactive load balancer that emphasizes portability and usability. In this paper we assume perfect intrinsic load balance and focus on extrinsic imbalances caused by OS noise, multiprogramming and mismatches of threads to hardware parallelism. Juggle shows performance improvements of up to 80 % over static load balancing for oversubscribed UPC, OpenMP, and pthreads benchmarks. We also show that Juggle is effective in unpredictable, multiprogrammed environments, with up to a 50 % performance improvement over the Linux load balancer and a 25 % reduction in performance variation. We analyze the impact of Juggle on parallel applications and derive lower bounds and approximations for thread completion times. We show that results from Juggle closely match theoretical predictions across a variety of architectures, including NUMA and hyper-threaded systems.

Accelerating knowledge-based energy evaluation in protein structure modeling with Graphics Processing Units

Evaluating the energy of a protein molecule is one of the most computationally costly operations in many protein structure modeling applications. In this paper, we present an efficient implementation of knowledge-based energy functions by taking advantage of the recent Graphics Processing Unit (GPU) architectures. We use DFIRE, a knowledge-based all-atom potential, as an example to demonstrate our GPU implementations on the latest NVIDIA Fermi architecture. A load balancing workload distribution scheme is designed to assign computations of pair-wise atom interactions to threads to achieve perfect or near-perfect load balancing in the symmetric N -body problem in DFIRE. Reorganizing atoms in the protein also improves the cache efficiency in Fermi GPU architecture, which is particularly effective for small proteins. Our DFIRE implementation on GPU (GPU-DFIRE) has exhibited a speedup of up to ∼ 150 on NVIDIA Quadro FX3800M and ∼ 250 on NVIDIA Tesla M2050 compared to the serial DFIRE implementation on CPU. Furthermore, we show that protein structure modeling applications, including a Monte Carlo sampling program and a local optimization program, can benefit from GPU-DFIRE with little programming modification but significant computational performance improvement.

Two-constraint domain decomposition with Space Filling Curves

In scientific computing, Space Filling Curves are a widely used tool for one-constraint domain decomposition. They provide a mechanism to sort multi-dimensional data in a locality preserving way, and, in this way, a (one dimensional) list of mesh elements is established which is subsequently split into 3 partitions under consideration of the constraint. This procedure has a runtime of O( Nlog N) ( N is the number of mesh elements) while nearly perfect load balancing can be established with reasonable partition surface sizes. In this work, we discuss the extensibility of this procedure to two-constraint settings which is desirable, since the methodology is extremely fast. Here, the splitting operation is subject to two constraints, and, unlike to the one-constraint case, obtaining near perfect balancing is often hard to establish, and is, even more as in the one-constraint case, in conflict with the induced surface sizes (or edge-cuts). We discuss multiple strategies to tackle the splitting, and we present a fast, O( Nlog N) splitting heuristic algorithm which provides an integer σ that allows to trade off between balancing and surface sizes which results in a O( Nlog N) two-constraint decomposition method. Results are compared to the multi-constraint domain decomposition abilities implemented in the Metis software package, and show that the method produces higher surface sizes, but is orders of magnitudes faster which makes the method superior for certain applications.

QUANTUM DYNAMICS ON MASSIVELY PARALLEL COMPUTERS: EFFICIENT NUMERICAL IMPLEMENTATION FOR PRECONDITIONED LINEAR SOLVERS AND EIGENSOLVERS

The eigenvalue/eigenvector and linear solve problems arising in computational quantum dynamics applications (e.g. rovibrational spectroscopy, reaction cross-sections, etc.) often involve large sparse matrices that exhibit a certain block structure. In such cases, specialized iterative methods that employ optimal separable basis (OSB) preconditioners (derived from a block Jacobi diagonalization procedure) have been found to be very efficient, vis-à-vis reducing the required CPU effort on serial computing platforms. Recently,1,2 a parallel implementation was introduced, based on a nonstandard domain decomposition scheme. Near-perfect parallel scalability was observed for the OSB preconditioner construction routines up to hundreds of nodes; however, the fundamental matrix–vector product operation itself was found not to scale well, in general. In addition, the number of nodes was selectively chosen, so as to ensure perfect load balancing. In this paper, two essential improvements are discussed: (1) new algorithm for the matrix–vector product operation with greatly improved parallel scalability and (2) generalization for arbitrary number of nodes and basis sizes. These improvements render the resultant parallel quantum dynamics codes suitable for robust application to a wide range of real molecular problems, running on massively parallel computing architectures.

Particle-field decomposition and domain decomposition in parallel particle-in-cell beam dynamics simulation

Particle-in-cell (PIC) simulation is widely used in many branches of physics and engineering. In this paper, we give an analysis of the particle-field decomposition method and the domain decomposition method in parallel particle-in-cell beam dynamics simulation. The parallel performance of the two decomposition methods was studied on the Cray XT4 and the IBM Blue Gene/P Computers. The domain decomposition method shows better scalability but is slower than the particle-field decomposition in most cases (up to a few thousand processors) for macroparticle dominant applications. The particle-field decomposition method also shows less memory usage than the domain decomposition method due to its use of perfect static load balance. For applications with a smaller ratio of macroparticles to grid points, the domain decomposition method exhibits better scalability and faster speed. Application of the particle-field decomposition scheme to high-resolution macroparticle-dominant parallel beam dynamics simulation for a future light source linear accelerator is presented as an example.

Potentialities of the Robust Multigrid Technique

AbstractThe present paper discusses the parallelization of the robust multigrid technique (RMT) and the possible way of applying this to unstructured grids. As opposed to the classical multigrid methods, the RMT is a trivial method of parallelization on coarse grids independent of the smoothing iterations. Estimates of the minimum speed-up and parallelism efficiency are given. An almost perfect load balance is demonstrated in a 3D illustrative test. To overcome the geometric nature of the technique, the RMT is used as a preconditioner in solving PDEs on unstructured grids. The procedure of auxiliary structured grids generation is considered in details.

Comparison via parallel performance models of angular and spatial domain decompositions for solving neutral particle transport problems

A previously reported parallel performance model for angular domain decomposition (ADD) of the discrete ordinates approximation for solving multidimensional neutral particle transport problems is revisited for stronger validation. Three communication schemes, native MPI, the bucket algorithm, and the distributed bucket algorithm, are included in the validation exercise that is successfully conducted on a Beowulf cluster. The parallel component of the parallel performance model is largely independent of the communication scheme, in contrast with the communication component that is strongly dependent on the global reduce algorithm. Correct trends for each component and each communication scheme are measured for the Arbitrarily High Order Transport (AHOT) code, thus validating the performance models. Furthermore, extensive experiments illustrate the superiority of the bucket algorithm, in the sense that it incurs a smaller communication penalty compared to the native MPI and distributed bucket algorithms. The primary question addressed in this work is for a given problem size, which domain decomposition scheme, angular or spatial, is best suited to parallelize discrete ordinates methods on a specific computational platform? We address this question for three-dimensional applications via parallel performance models for the abovementioned ADD, and a previously constructed and validated spatial domain decomposition (SDD) model. The constructed parallel performance models include parameters specifying the problem size and system performance. We conclude that for large problems the parallel component dwarfs the communication component even on moderately large numbers of processors. The main advantages of SDD are (a) scalability to higher numbers of processors of the order of the number of computational cells; (b) smaller memory requirement; (c) better performance than ADD on high-end platforms and large number of processors. On the other hand, the main advantages of ADD are (a) perfect load balance; (b) simple implementation, even on unstructured grids; (c) better performance than SDD on medium- and low-end platforms and large number of discrete ordinates. It follows that programmers and users of discrete ordinates codes must carefully select the appropriate domain decomposition method for the class of problems and multiprocessor platforms they wish to target.

Energy-efficient policies for embedded clusters

Power conservation has become a key design issue for many systems, including clusters deployed for embedded systems, where power availability ultimately determines system lifetime. These clusters execute a high rate of requests of highly-variable length, such as in satellite-based multiprocessor systems. The goal of power management in such systems is to minimize the aggregate energy consumption of the whole cluster while ensuring timely responses to requests. In the past, dynamic voltage scaling (DVS) and on/off schemes have been studied under the assumptions of continuously tunable processor frequencies and perfect load-balancing. In this work, we focus on the more realistic case of discrete processor frequencies and propose a new policy that adjusts the number of active nodes based on the system load , not system frequency. We also design a threshold scheme which prevents the system from reacting to short-lived temporary workload changes in the presence of unstable incoming workload. Simulation and implementation results on real hardware show that our policy is very effective in reducing the overall power consumption of clusters executing embedded applications.

Variable order revised binary treecode

Three essential improvements are described to the treecode in terms of the expansion formula, the choice of the expansion order as well as the tree structure. Firstly, the multipole expansion is based on the real spherical harmonic functions to reduce the CPU time. Secondly, the expansion order is given in terms of the ratio of the distance of a field point to a source box to the box size, which reflects the relative error of the expansion. With that, a large portion of the sources has been evaluated by the multipole expansion at low levels of the source tree, which is around two-thirds of sources at the first two levels of the tree averagely. The algorithm reduces the CPU time dependency on expansion order p from O( p 2) of the classical treecode to be lower than a linear dependency in p max, where p max is the maximum expansion order used in the variable order expansion. Thirdly, a revised binary tree is built by performing the bisections thrice at each tree level, discarding the boxes generated in the first two bisections and remaining only the boxes generated in the last one. This tree avoids the disadvantage of a binary treecode demanding significantly more CPU time than an oct-treecode. It has high adaptiveness to the source distribution and perfect load balancing for performing the parallelization. Simulations are carried out for N vortex elements and N field points distributed randomly in a cube, a 5:1:1 parallelepiped, and a 10:1:1 parallelepiped, using the oct-tree and revised binary tree, respectively. The algorithm is an order of magnitude faster than those of Strickland et al. [ESAIM: Proceedings 7 (1999) 408], Warren and Salmon [Comput. Phys. Commun. 87 (1995) 266], and Lindsay and Krasny [J. Comput. Phys. 172 (2001) 879]. Simulations also demonstrate the efficiency of the revised binary treecode for an inhomogeneous source distribution.

Varying the domain size of the dynamic load-balancing algorithm DASUD for SPMD and MPMD programming scenarios

Dynamic load balancing is a key problem for the efficient use of parallel systems when solving applications with unpredictable load estimates. However, depending on the underlying programming paradigm Single Program Multiple Data (SPMD) or Multiple Program Multiple Data (MPMD) the balancing requirements vary. In SPMD scenarios, a perfect load balance is desired, whereas in MPMD scenarios it might be better to quickly obtain a large reduction in load imbalance in a short period of time. We propose extending the local domain of a given processor in the load-balancing algorithms to find a better scope for each paradigm. For that purpose, we present a generalised version of the Diffusion Algorithm Searching Unbalanced Domains (called ds-DASUD), which extends the local domain of each processor beyond its immediate neighbour. ds-DASUD belongs to the iterative distributed load-balancing (IDLB) class and, in its original formulation, operates in a diffusion scheme where a processor balances its load with all its immediate neighbours (ds=1). We evaluate this algorithm for the two programming paradigms varying the domain size. The evaluation was carried out using two simulators (load-balancing and network simulators) for a large set of load distributions that exhibit different degrees of initial workload unbalancing. These distributions were applied to torus and hypercube topologies, and the number of processors ranged from 8 to 128. From these experiments, we conclude that the 1-DASUD fits well for SPMD scenarios, whereas for MPMD 3-DASUD and ((d/2)+1)-DASUD for hypercube and torus topologies, respectively, obtain the best trade-off between the imbalance reduction (up to 85%) and the cost incurred in reaching it.

A parallel finite element procedure for contact-impact problems

An efficient parallel finite element procedure for contact-impact problems is presented within the framework of explicit finite element analysis with thepenalty method. The procedure concerned includes a parallel Belytschko-Lin-Tsay shell element generation algorithm and a parallel contact-impact algorithm based on the master-slave slideline algorithm. An element-wise domain decomposition strategy and a communication minimization strategy are featured to achieve almost perfect load balancing among processors and to show scalability of the parallel performance. Throughout this work, a prototype code, named GT-PARADYN, is developed on the IBM SP2 to implement the procedure presented, under message-passing paradigm. Some examples are provided to demonstrate the timing results of the algorithms, discussing the accuracy and efficiency of the code.

A fuzzy genetic multiobjective optimization algorithm for a multilevel generalized assignment problem

We consider a task-operator-machine assignment problem where we seek to minimize the total execution time, to come as close as possible to a perfect load balance among the operators and not to exceed neither predefined inter-operator communication costs nor a prefixed number of resources. Besides, in an industrial environment where work force frequently changes, manufacturing systems need to be flexible and critical decisions have to be quickly taken. In this context, a fuzzy genetic multiobjective optimization algorithm is developed to solve a multilevel generalized assignment problem usually encountered in the clothing industry.

An Efficient Algorithm for Perfect Load Balancing on Hypercube Multiprocessors

This paper proposes a simple yet efficient algorithm to distribute loads evenly on multiprocessor computers with hypercube interconnection networks. This algorithm was developed based upon the well-known dimension exchange method. However, the error accumulation suffered by other algorithms based on the dimension exchange method is avoided by exploiting the notion of regular distributions, which are commonly deployed for data distributions in parallel programming. This algorithm achieves a perfect load balance over P processors with an error of 1 and the worst-case time complexity of O(M log2P), where M is the maximum number of tasks initially assigned to each processor. Furthermore, perfect load balance is achieved over subcubes as well—once a hypercube is balanced, if the cube is decomposed into two subcubes by the lowest bit of node addresses, then the difference between the numbers of the total tasks of these subcubes is at most 1.

Static LU Decomposition on Heterogeneous Platforms

In this paper, the authors deal with algorithmic issues on heterogeneous platforms. They concentrate on dense linear algebra kernels, such as matrix multiplication or LU decomposition. Block-cyclic distribution techniques used in ScaLAPACK are no longer sufficient to balance the load among processors running at different speeds. The main result of this paper is to provide a static data distribution scheme that leads to an asymptotically perfect load balancing for LU decomposition, thereby providing solid foundations toward the design of a cluster-oriented version of ScaLAPACK.