High Parallel Efficiency Research Articles

An Optimized Efficient Galerkin Averaging-Incremental Harmonic Balance Method for High-Dimensional Spatially Discretized Models of Continuous Systems Based on Parallel Computing

Abstract Modern computers are generally equipped with multicore central processing units (CPUs). There has been a great interest to introduce a parallelized harmonic balance method to make full use of computing resources and improve the efficiency in dealing with nonlinear dynamic analysis of high-dimensional spatially discretized models of continuous systems. In this work, an optimized efficient Galerkin averaging-incremental harmonic balance (EGA-IHB) method for solving high-dimensional spatially discretized models of continuous systems based on parallel computing is introduced. Independent sampling and EGA procedures are introduced to achieve generality. Optimized parallel implementation based on tensor contraction is introduced in time-domain series calculations and the quasi-Newton method is used in the iteration procedure, which greatly accelerates computational speeds of both serial and parallel implementations. Especially, the parallel implementation achieves high parallel efficiency when multiple CPU cores are used. Due to its high computational efficiency and good robustness, the proposed method has the potential to be used as a powerful universal solver and analyzer for general types of continuous systems.

Parallel‐in‐time‐and‐space electromagnetic transient simulation of multi‐terminal DC grids with device‐level switch modelling

The electromagnetic transient (EMT) simulation of multi-terminal DC (MTDC) grids requires a detailed device-level modular multilevel converter (MMC) model, which can have thousands of state variables and complex internal structures. The fast device-level insulated gate bipolar transistor (IGBT) transient requires a very small time-step, making the computational overhead prohibitive. Based on the analysis of the parallel-in-time (PiT) implementation of detailed modelled MMCs, this paper proposes a task-based hybrid PiT algorithm to achieve high parallel efficiency and speed-up of MMC with device-level modelling. Moreover, a transmission line model(TLM)-based parallel-in-time-and-space (PiT+PiS) method is proposed to connect PiT grids to conventional or other PiT grids and exploit the maximum parallelism. Simulation results show greater than 30 × speed-up and 60% parallel efficiency on a 48 cores computer for the hybrid PiT method in a 201-level three-phase MMC test case, and 20 × speed-up in the transient simulation of CIGRÉ B4 DC grid test system for the PiT+PiS method.

Effective Distribution of Tasks in Multiprocessor and Multi-Computers Distributed Homogeneous Systems

Nowadays, a promising is the direction associated with the use of a large number of processors to solve the resource-intensive tasks. The enormous potential of multiprocessor and multicomputer systems can be fully revealed only when we apply effective methods for organizing the distribution of tasks between processors or computers. However, the problem of efficient distribution of tasks between processors and computers in similar computing systems remains relevant. Two key factors are critical and have an impact on system performance. This is load uniformity and interprocessor or intercomputer interactions. These conflicting factors must be taken into account simultaneously in the distribution of tasks in multiprocessor computing systems. A uniform loading plays a key role in achieving high parallel efficiency, especially in systems with a large number of processors or computers. Efficiency means not only the ability to obtain the result of computations in a finite number of iterations with the necessary accuracy, but also to obtain the result in the shortest possible time. The number of tasks intended for execution on each processor or each computer should be determined so that the execution time is minimal. This study offers a technique that takes into account the workload of computers and intercomputer interactions, and allows one to minimize the execution time of tasks. The technique proposed by the authors allows the comparison of different architectures of computers and computing modules. In this case, a parameter is used that characterizes the behavior of various models with a fixed number of computers, as well as a parameter that is necessary to compare the effectiveness of each computer architecture or computing module when a different number of computers are used. The number of computers can be variable at a fixed workload. The mathematical implementation of this method is based on the problem solution of the mathematical optimization or feasibility.

Harnessing the Power of Multi-GPU Acceleration into the Quantum Interaction Computational Kernel Program.

We report a new multi-GPU capable ab initio Hartree-Fock/density functional theory implementation integrated into the open source QUantum Interaction Computational Kernel (QUICK) program. Details on the load balancing algorithms for electron repulsion integrals and exchange correlation quadrature across multiple GPUs are described. Benchmarking studies carried out on up to four GPU nodes, each containing four NVIDIA V100-SXM2 type GPUs demonstrate that our implementation is capable of achieving excellent load balancing and high parallel efficiency. For representative medium to large size protein/organic molecular systems, the observed parallel efficiencies remained above 82% for the Kohn-Sham matrix formation and above 90% for nuclear gradient calculations. The accelerations on NVIDIA A100, P100, and K80 platforms also have realized parallel efficiencies higher than 68% in all tested cases, paving the way for large-scale ab initio electronic structure calculations with QUICK.

An efficient parallel iteration algorithm for nonlinear diffusion equations with time extrapolation techniques and the Jacobi explicit scheme

To numerically simulate the radiation diffusion problem with high parallel efficiency, we present a new parallel iteration algorithm for nonlinear diffusion equations. The algorithm is based on the domain decomposition method, and it integrates time extrapolation techniques and an advanced Jacobi explicit scheme. The domain decomposition method decomposes a large global problem into multiple sub-problems which can be solved on multiple processors in parallel. The time extrapolation technique gives the prediction values with the second or third order accuracy in time for the current time layer by specific combinations of the previous two or three time layers. The advanced Jacobi explicit scheme further improves the precision of the prediction values. Overall, the proposed algorithm makes the prediction values of the inner boundary more reasonable and offers accurate iterative initial values for all cells, which reduces the number of nonlinear iterations and improves the parallel computation efficiency remarkably.

Surface-based single-subject morphological brain networks: Effects of morphological index, brain parcellation and similarity measure, sample size-varying stability and test-retest reliability

Morphological brain networks, in particular those at the individual level, have become an important approach for studying the human brain connectome; however, relevant methodology is far from being well-established in their formation, description and reproducibility. Here, we extended our previous study by constructing and characterizing single-subject morphological similarity networks from brain volume to surface space and systematically evaluated their reproducibility with respect to effects of different choices of morphological index, brain parcellation atlas and similarity measure, sample size-varying stability and test-retest reliability. Using the Human Connectome Project dataset, we found that surface-based single-subject morphological similarity networks shared common small-world organization, high parallel efficiency, modular architecture and bilaterally distributed hubs regardless of different analytical strategies. Nevertheless, quantitative values of all interregional similarities, global network measures and nodal centralities were significantly affected by choices of morphological index, brain parcellation atlas and similarity measure. Moreover, the morphological similarity networks varied along with the number of participants and approached stability until the sample size exceeded ~70. Using an independent test-retest dataset, we found fair to good, even excellent, reliability for most interregional similarities and network measures, which were also modulated by different analytical strategies, in particular choices of morphological index. Specifically, fractal dimension and sulcal depth outperformed gyrification index and cortical thickness, higher-resolution atlases outperformed lower-resolution atlases, and Jensen-Shannon divergence-based similarity outperformed Kullback-Leibler divergence-based similarity. Altogether, our findings propose surface-based single-subject morphological similarity networks as a reliable method to characterize the human brain connectome and provide methodological recommendations and guidance for future research.

LES of a turbulent swirl flame using a mesh adaptive level-set method with dynamic load balancing

A turbulent lean premixed swirl flame is numerically investigated by solving the Navier–Stokes equations with a finite-volume large-eddy simulation (LES) method. A combined G-equation progress variable approach is applied to model the combustion process using a solution adaptive level-set solver. The finite-volume and the level-set solver are parallelized and coupled on a joint hierarchical Cartesian mesh, where each solver can individually use and adapt a subset of the mesh cells. The level-set solver adapts the mesh locally in a band close to the flame front location to automatically satisfy the high accuracy requirements of the level-set solution. The numerical results for the turbulent swirl flame are in excellent agreement with experimental findings. Since the flame shape varies during the computation, the number of cells associated with the flame shifts between the individual subdomains which changes the workload distribution on the parallel computing processes. To achieve a high parallel efficiency, a dynamic load balancing method is applied which determines a new partitioning of the grid using estimated cell weights based on measured computing times to redistribute the cells among all subdomains accordingly. The efficiency of the dynamic load balancing scheme to reduce load imbalances is demonstrated for a large-scale simulation of the investigated swirl flame on 170,000 compute cores. That is, the dynamic load balancing scheme reduces the computing time of this simulation by approximately 30%.

THE IMPROVEMENT AND REALIZATION OF FINITE-DIFFERENCE LATTICE BOLTZMANN METHOD

The Lattice Boltzmann Method (LBM) is a numerical method developed in recent decades. It has the characteristics of high parallel efficiency and simple boundary processing. The basic idea is to construct a simplified dynamic model so that the macroscopic behavior of the model is the same as the macroscopic equation. From the perspective of micro-dynamics, LBM treats macro-physical quantities as micro-quantities to obtain results by statistical averaging. The Finite-difference LBM (FDLBM) is a new numerical method developed based on LBM. The first finite-difference LBE (FDLBE) was perhaps due to Tamura and Akinori and was examined by Cao et al. in more detail. Finite-difference LBM was further extended to curvilinear coordinates with nonuniform grids by Mei and Shyy. By improving the FDLBE proposed by Mei and Shyy, a new finite difference LBM is obtained in the paper. In the model, the collision term is treated implicitly, just as done in the Mei-Shyy model. However, by introducing another distribution function based on the earlier distribution function， the implicitness of the discrete scheme is eliminated, and a simple explicit scheme is finally obtained, such as the standard LBE. Furthermore, this trick for the FDLBE can also be easily used to develop more efficient FVLBE and FELBE schemes. To verify the correctness and feasibility of this improved FDLBM model, which is used to calculate the square cavity model, and the calculated results are compared with the data of the classic square cavity model. The comparison result includes two items: the velocity on the centerline of the square cavity and the position of the vortex center in the square cavity. The simulation results of FDLBM are very consistent with the data in the literature. When Re=400, the velocity profiles of u and v on the centerline of the square cavity are consistent with the data results in Ghia's paper, and the vortex center position in the square cavity is also almost the same as the data results in Ghia's paper. Therefore, the verification of FDLBM is successful and FDLBM is feasible. This improved method can also serve as a reference for subsequent research.

Improved block-Jacobi parallel algorithm for the SN nodal method with unstructured mesh

The substantial time required for solving the neutron transport equation can be reduced through parallel calculation. The block-Jacobi algorithm is a common method for parallel neutron transport calculation in an unstructured mesh; however, iteration degradation is an inevitable problem that limits the application of this algorithm by reducing the parallel efficiency. In this study, we applied the block-Jacobi algorithm to the SN nodal method with a triangular-z mesh and proposed an improvement to achieve high parallel efficiency. The interface prediction (IP) method was developed to prevent iteration degradation. This method is based on extrapolating the interface information instead of using the information from the preceding iteration at the interfaces in sub-domains. Meanwhile, the prediction was used recursively to accelerate the self-group scattering source iteration in the entire space; this process is referred to as the inner iteration prediction method (IIP). These two methods effectively prevent iteration degradation and reduce time required for self-group scattering source iteration. A more stable and improved parallel performance was thus achieved.

Critique of “Planetary Normal Mode Computation: Parallel Algorithms, Performance, and Reproducibility” by SCC Team From Tsinghua University

In this article we present our results from the SC19 Student Cluster Competition Reproducibility Challenge. The challenge entails reproducing the article entitled “Computing Planetary Interior Normal Modes with A Highly Parallel Polynomial Filtering Eigensolver” presented at SC'18, which proposes a parallel polynomial filtered Lanczos algorithm to directly calculate the planetary normal modes of heterogeneous planets. The proposed algorithm showed excellent performance with relatively low memory consumption and high parallel efficiency. In this work, we reproduce the scaling tests in that article on a cluster using Intel Cascade Lake architecture and use the proposed algorithm to illustrate specific normal modes of Mars. We compare the results obtained on our cluster with those in the original article. We also design a new metric to better analyze the results. In addition, we use the profiling tool Intel VTune Amplifier to explain our discoveries. Our results demonstrate that the given models show great scalability, which is similar to the original article. The required normal modes of Mars are also successfully calculated and visualized.

Accelerated K-Means Algorithms for Low-Dimensional Data on Parallel Shared-Memory Systems

This paper considers the problem of exact accelerated algorithms for the K-means clustering of low-dimensional data on modern multi-core systems. A version of the filtering algorithm parallelized using the OpenMP (Open Multi-Processing) standard is proposed. The algorithm employs a kd-tree structure to skip some unnecessary calculations between cluster centroids and feature vectors. In our approach, both the kd-tree construction and the iterations of the K-means are parallelized using the OpenMP tasking mechanism. A new task is created for a recursive call performed during kd-tree construction and traversal. The tasks are executed in parallel by the cores of a shared-memory system. In computational experiments, we evaluated the parallel efficiency of our approach and compared its performance to the parallel Lloyd’s method, a GPU (Graphics Processing Unit) formulation of the K-means algorithm, and two parallel triangle inequality-based algorithms intended for low-dimensional data. The evaluation was performed on six synthetic datasets from two distributions and seven real-life datasets. The experiments, executed on a 24-core system, indicated that our version of the filtering algorithm had satisfactory or high parallel efficiency. Its runtime was much shorter than those of competing algorithms. However, the advantage of the parallel filtering algorithm decreased rapidly as the dimension of data increased.

Enhancing Parallel Scheduling of Grid Jobs in a Multicored Environment

The computing Grid has emerged as a platform to solve the complex and ever-increasing processing need of man and advances in computing technology have birthed the multicore era aimed for high throughput and efficient parallel computing. However, most systems still rely on the underlying hardware for parallelism despite the hard evidence that sequential algorithms do not optimally exploit parallel systems. This research seeks to harness the benefits of multicore systems using job and machine grouping methods to enhance parallelism in the scheduling of Grid jobs. The paper presents the result of two separate experiments on a method that parallelize scheduling algorithm on two multicore platforms. An arbitrary method was employed to group machines; a summation of the total processing power of machines in each group was made. To ensure load balancing, jobs were allocated to machine groups based on the ratio of the total processing power of the machines in each group. The MinMin Grid scheduling algorithm was implemented independently within the groups using a range of threads varied in powers of two. Also, the numbers of groups were varied between 2, 4, and 8. The same experiment was executed on a single processor computer; a duocore machine and a quadcore machine. A performance improvement of 16% to 85% was recorded by the group method against the best ordinary MinMin results and an improvement of 50% to 84% was recorded by the group method against the ordinary MinMin on corresponding machines. We prove that an increase in the number of groups results in improved performance on corresponding machines (approximately 2 times using 2 groups, approximately 3 times using four groups, and approximately 6 times using 8 groups). And most importantly, we established that as the number of processors increases, the grouping method makes more significant improvements over the ordinary MinMin scheduling algorithm executed on the multicore systems.

Large-Scale Discrete Fourier Transform on TPUs

In this work, we present two parallel algorithms for the large-scale discrete Fourier transform (DFT) on Tensor Processing Unit (TPU) clusters. The two parallel algorithms are associated with two DFT formulations: one formulation, denoted as KDFT, is based on the Kronecker product; the other is based on the famous Cooley-Tukey algorithm and phase adjustment, denoted as FFT. Both KDFT and FFT formulations take full advantage of TPU's strength in matrix multiplications. The KDFT formulation allows direct use of nonuniform inputs without additional step. In the two parallel algorithms, the same strategy of data decomposition is applied to the input data. Through the data decomposition, the dense matrix multiplications in KDFT and FFT are kept local within TPU cores, which can be performed completely in parallel. The communication among TPU cores is achieved through the one-shuffle scheme in both parallel algorithms, with which sending and receiving data takes place simultaneously between two neighboring cores and along the same direction on the interconnect network. The one-shuffle scheme is designed for the interconnect topology of TPU clusters, minimizing the time required by the communication among TPU cores. Both KDFT and FFT are implemented in TensorFlow. The three-dimensional complex DFT is performed on an example of dimension 8192 ×8192 ×8192 with a full TPU Pod: the run time of KDFT is 12.66 seconds and that of FFT is 8.3 seconds. Scaling analysis is provided to demonstrate the high parallel efficiency of the two DFT implementations on TPUs.

Large-scale first-principles quantum transport simulations using plane wave basis set on high performance computing platforms

As the characteristic lengths of advanced electronic devices are approaching the atomic scale, ab initio simulation method, with full consideration of quantum mechanical effects, becomes essential to study the quantum transport phenomenon in them. The widely used non-equilibrium Green’s function (NEGF) combined with the density functional theory (DFT) approach prefers a localized basis set. As many states of the art DFT calculations for solid state systems are carried out in plane waves, it is thus worth to investigate the feasibility of using the plane wave basis set for large-scale quantum transport calculations. Here we present a plane wave method for large-scale transport calculations based on a previously developed scattering state calculation approach (Wang, 2005). We address the unique computational challenges of applying that approach for large-scale systems where it is too expensive to calculate all the occupied eigenstates of the system as in conventional DFT calculations. By applying several high-efficiency parallel algorithms, including linear-scaling DFT algorithm, folded spectrum method, and Chebyshev filter technique, we demonstrate that it is possible to use this approach to simulate a system with several thousand atoms on high performance computing platforms. This method is not only used to study nanowire interconnects, showing how the shape and point defect affects their transport properties, but also used to study nanoscale Si transistor. Such quantum transport simulation method will be useful for investigating and designing nanoscale devices.

Parallel discontinuous Galerkin finite element method for computing hyperbolic conservation law on unstructured meshes

PurposeThe discontinuous Galerkin finite element method (DGFEM) is very suited for realizing high order resolution approximations on unstructured grids for calculating the hyperbolic conservation law. However, it requires a significant amount of computing resources. Therefore, this paper aims to investigate how to solve the Euler equations in parallel systems and improve the parallel performance.Design/methodology/approachDiscontinuous Galerkin discretization is used for the compressible inviscid Euler equations. The multi-level domain decomposition strategy was used to deal with the computational grids and ensure the calculation load balancing. The total variation diminishing (TVD) Runge–Kutta (RK) scheme coupled with the multigrid strategy was employed to further improve parallel efficiency. Moreover, the Newton Block Gauss–Seidel (GS) method was adopted to accelerate convergence and improve the iteration efficiency.FindingsNumerical experiments were implemented for the compressible inviscid flow problems around NACA0012 airfoil, over M6 wing and DLR-F6 configuration. The parallel acceleration is near to a linear convergence. The results indicate that the present parallel algorithm can reduce computational time significantly and allocate memory reasonably, which has high parallel efficiency and speedup, and it is well-suited to large-scale scientific computational problems on multiple instruction stream multiple data stream model.Originality/valueThe parallel DGFEM coupled with TVD RK and the Newton Block GS methods was presented for hyperbolic conservation law on unstructured meshes.

An efficient scheduling algorithm for dataflow architecture using loop-pipelining

Dataflow architecture has native advantages in achieving high instruction parallelism and power efficiency for today’s emerging applications such as high performance computing and deep neural network. In dataflow computing, the execution of instructions is driven by data, so the data transfer efficiency of the network on chip (NoC) is a key factor affecting performance. However, the NoC performance degrades due to the increasing use of multicast communications in many applications. The existing dataflow architecture instruction scheduling algorithms do not optimize multicast communication between the instruction and its successor instructions, so the routing paths of many multicast packets have forks which cause bandwidth waste and potential network congestion. We propose a sharing path awareness (SPA) algorithm to optimize multicast communication in the dataflow architecture. The algorithm shares the routing paths from the instruction to its child node to reduce the NoC bandwidth waste through the instruction scheduler. For applications using software iteration, we further extend the loop optimization to the SPA algorithm to sufficiently exploit instruction-level parallelism. Compared with the state-of-the-art algorithm, we show that the SPA algorithm achieves 20.21% average performance improvement and 15.11% energy consumption reduction for our experimental workloads.

Localized Topological Simplification of Scalar Data.

This paper describes a localized algorithm for the topological simplification of scalar data, an essential pre-processing step of topological data analysis (TDA). Given a scalar field f and a selection of extrema to preserve, the proposed localized topological simplification (LTS) derives a function g that is close to f and only exhibits the selected set of extrema. Specifically, sub- and superlevel set components associated with undesired extrema are first locally flattened and then correctly embedded into the global scalar field, such that these regions are guaranteed-from a combinatorial perspective-to no longer contain any undesired extrema. In contrast to previous global approaches, LTS only and independently processes regions of the domain that actually need to be simplified, which already results in a noticeable speedup. Moreover, due to the localized nature of the algorithm, LTS can utilize shared-memory parallelism to simplify regions simultaneously with a high parallel efficiency (70%). Hence, LTS significantly improves interactivity for the exploration of simplification parameters and their effect on subsequent topological analysis. For such exploration tasks, LTS brings the overall execution time of a plethora of TDA pipelines from minutes down to seconds, with an average observed speedup over state-of-the-art techniques of up to ×36. Furthermore, in the special case where preserved extrema are selected based on topological persistence, an adapted version of LTS partially computes the persistence diagram and simultaneously simplifies features below a predefined persistence threshold. The effectiveness of LTS, its parallel efficiency, and its resulting benefits for TDA are demonstrated on several simulated and acquired datasets from different application domains, including physics, chemistry, and biomedical imaging.

Neutronic calculations of the China dual-functional lithium–lead test blanket module with the parallel discrete ordinates code Hydra

The China dual-functional lithium–lead test blanket module (DFLL-TBM) is a liquid LiPb blanket concept developed by the Institute of Nuclear Energy Safety Technology of the Chinese Academy of Sciences for testing in ITER to validate relevant tritium breeding and shielding technologies. In this study, neutronic calculations of DFLL-TBM were carried out using a massively parallel three-dimensional transport code, Hydra, with the Fusion Evaluated Nuclear Data Library/MG. Hydra was developed by the Nuclear Engineering Computational Physics Lab based on the discrete ordinates method and has been devoted to neutronic analysis and shielding evaluation for nuclear facilities. An in-house Monte Carlo code (MCX) was employed to verify the discretized calculation model used by Hydra for the DFLL-TBM calculations. The results showed two key aspects: (1) In most material zones, Hydra solutions are in good agreement with the reference MCX results within 1%, and the maximal relative difference of the neutron flux is merely 3%, demonstrating the correctness of the calculation model; (2) while the current DFLL-TBM design meets the operation shielding requirement of ITER for 4 years, it does not satisfy the tritium self-sufficiency requirement. Compared to the two-step approach, Hydra produces higher accuracies as it does not rely on the homogenization technique during the calculation process. The parallel efficiency tests of Hydra using the DFLL-TBM model also showed that this code maintains a high parallel efficiency on O(100) processors and, as a result, is able to significantly improve computing performance through parallelization. Parameter studies have been carried out by varying the thickness of the beryllium armor layer and the tritium breeding zone to understand the influence of the beryllium layer and breeding zone thickness on tritium breeding performance. This establishes a foundation for further improvement in the tritium production performance of DFLL-TBM.

Massively Parallel CFD Simulation Software: CCFD Development and Optimization Based on Sunway TaihuLight

A parallel framework software, CCFD, based on the structure grid, and suitable for parallel computing of super-large-scale structure blocks, is designed and implemented. An overdecomposition method, in which the load balancing strategy is based on the domain decomposition method, is designed for the graph subdivision algorithm. This method takes computation and communication as the limiting condition and realizes the load balance between blocks by dividing the weighted graph. The fast convergence technique of a high-efficiency parallel geometric multigrid greatly improves the parallel efficiency and convergence speed of CCFD software. This paper introduces the software structure, process invocations, and calculation method of CCFD and introduces a hybrid parallel acceleration technology based on the Sunway TaihuLight heterogeneous architecture. The results calculated by Onera-M6 and DLR-F6 standard model show that the software structure and method in this paper are feasible and can meet the requirements of a large-scale parallel solution.

Benchmark of the Compute-in-Memory-Based DNN Accelerator With Area Constraint

Compute-in-memory (CIM) is a promising computing paradigm to accelerate the inference of deep neural network (DNN) algorithms due to its high processing parallelism and energy efficiency. Prior CIM-based DNN accelerators mostly consider full custom design, which assumes that all the weights are stored on-chip. For lightweight smart edge devices, this assumption may not hold. In this article, CIM-based DNN accelerators are designed and benchmarked under different chip area constraints. First, a scheduling strategy and dataflow for DNN inference is investigated when only part of the weights can be stored on-chip. Two weight reload schemes are evaluated: 1) reload partial weights and reuse input/output feature maps and 2) load a batch of input and reuse the partial weights on-chip across the batch. Then, system-level performance benchmark is performed for the inference of ResNet-18 on ImageNet data set. The design tradeoffs with different area constraints, dataflow, and device technologies [static random access memory (SRAM) versus ferroelectric field-effect transistor (FeFET)] are discussed.