Tianhe-2 Supercomputer Research Articles

Improving Pareto Local Search Using Cooperative Parallelism Strategies for Multiobjective Combinatorial Optimization.

Pareto local search (PLS) is a natural extension of local search for multiobjective combinatorial optimization problems (MCOPs). In our previous work, we improved the anytime performance of PLS using parallel computing techniques and proposed a parallel PLS based on decomposition (PPLS/D). In PPLS/D, the solution space is searched by multiple independent parallel processes simultaneously. This article further improves PPLS/D by introducing two new cooperative process techniques, namely, a cooperative search mechanism and a cooperative subregion-adjusting strategy. In the cooperative search mechanism, the parallel processes share high-quality solutions with each other during the search according to a distributed topology. In the proposed subregion-adjusting strategy, a master process collects useful information from all processes during the search to approximate the Pareto front (PF) and redivide the subregions evenly. In the experimental studies, three well-known NP-hard MCOPs with up to six objectives were selected as test problems. The experimental results on the Tianhe-2 supercomputer verified the effectiveness of the proposed techniques.

Tracking Carbon Dioxide with Lagrangian Transport Simulations: Case Study of Canadian Forest Fires in May 2021

The large amounts of greenhouse gases, such as carbon dioxide, produced by severe forest fires not only seriously affect the ecosystems in the area where the fires occur but also cause a greenhouse effect that has a profound impact on the natural environment in other parts of the world. Numerical simulations of greenhouse gas transport processes are often affected by uncertainties in the location and timing of the emission sources and local meteorological conditions, and it is difficult to obtain accurate and credible predictions by combining remote sensing satellite data with given meteorological forecasts or reanalyses. To study the regional transport processes and impacts of greenhouse gases produced by sudden large-scale forest fires, this study applies the Lagrangian particle dispersion model Massive-Parallel Trajectory Calculations (MPTRAC) to conduct forward simulations of the CO2 transport process of greenhouse gases emitted from forest fires in the central region of Saskatchewan, Canada, during the period of 17 May to 25 May 2021. The simulation results are validated with the Orbiting Carbon Observatory-2 Goddard Earth Observing System (OCO-2 GEOS) Level 3 daily gridded CO2 product over the study area. In order to leverage the high computational costs of the numerical simulations of the model, we implement the forward simulations on the Tianhe-2 supercomputer platform and the JUWELS HPC system, which greatly improves the computational efficiency through parallel computation and makes near-real-time predictions of atmospheric transport processes feasible.

Comparison of Parallel Genetic Algorithm and Particle Swarm Optimization for Parameter Calibration in Hydrological Simulation

ABSTRACT Parameter calibration is an important part of hydrological simulation and affects the final simulation results. In this paper, we introduce heuristic optimization algorithms, genetic algorithm (GA) to cope with the complexity of the parameter calibration problem, and use particle swarm optimization algorithm (PSO) as a comparison. For large-scale hydrological simulations, we use a multilevel parallel parameter calibration framework to make full use of processor resources, and accelerate the process of solving high-dimensional parameter calibration. Further, we test and apply the experiments on domestic supercomputers. The results of parameter calibration with GA and PSO can basically reach the ideal value of 0.65 and above, with PSO achieving a speedup of 58.52 on TianHe-2 supercomputer. The experimental results indicate that using a parallel implementation on multicore CPUs makes high-dimensional parameter calibration in large-scale hydrological simulation possible. Moreover, our comparison of the two algorithms shows that the GA obtains better calibration results, and the PSO has a more pronounced acceleration effect.

Dynamic Allocation Method of Economic Information Integrated Data Based on Deep Learning Algorithm.

In view of the low efficiency of traditional data fusion algorithms in wireless sensor networks and the difficulty in processing high-dimensional data, a new algorithm CNNMDA, based on the deep learning model is proposed to realize data fusion. Firstly, the algorithm trains the constructed feature extraction model CNNM at the sink node; then each terminal node extracts the original data features through CNNM and finally sends the fused data to the sink node, so as to reduce the data transmission amount and prolong the network life. Simulation experiments show that compared with similar fusion algorithms, the CNNMDA can greatly reduce network energy consumption under the same data amount, and effectively improve the efficiency and accuracy of data fusion. In order to solve the problem that parameter synchronization takes too long in synchronous parallel, a dynamic training data allocation algorithm in multimachine synchronous parallel is proposed. Based on the computing efficiency of compute nodes, the amount of sample data to be processed by nodes will be dynamically adjusted after each iteration. This mechanism not only enables the model to be synchronized and parallel but also reduces the time of waiting for gradient updates. Finally, a comparative experiment is carried out on the Tianhe-2 supercomputer, and the experimental results show that the proposed optimization mechanism achieves the expected effect.

A three-dimensional finite difference model for ocean acoustic propagation and benchmarking for topographic effects.

A three-dimensional (3D) finite difference (FD) model with formal fourth-order accuracy has been developed for the ocean acoustic Helmholtz equation (HE), which can be used to address arbitrary bathymetry and provide more accurate benchmark solutions for other 3D underwater acoustic approximate models. The derivatives in the acoustic HE are numerically discretized based on regular grids, and the perfectly matched layer is introduced to absorb unphysical reflections from the boundaries where Sommerfeld radiation conditions are deployed. The system of linear equations is solved using a parallel matrix-free geometric multigrid preconditioned biconjugate gradient stabilized iteration method, and the code (named COACH) is run on the Tianhe-2 supercomputer in China. Four 3D topographic benchmark acoustic cases-a wedge waveguide, Gaussian canyon, conical seamount, and corrugated seabed-are simulated to test the present FD model, and the maximum number of grid points reaches 33.15 × 109 in the wedge waveguide case, running in parallel with 988 central processing unit cores. Furthermore, the accuracy and generality of the present model have been verified by solution comparisons with other available 3D acoustic propagation models, and the two-dimensional and 3D transmission loss contours are presented to facilitate the distinguishing among the acoustic field features of these cases.

Memory-Efficient and Skew-Tolerant MapReduce Over MPI for Supercomputing Systems

Data analytics has become an integral part of large-scale scientific computing. Among various data analytics frameworks, MapReduce has gained the most traction. Although some efforts have been made to enable efficient MapReduce for supercomputing systems, they are often limited to fairly homogeneous workloads where equal partitioning of input data across tasks results in essentially equal output or temporary data generated on each task. For workloads that are more skewed, however, current implementations can result in imbalance in memory usage and, consequently, can cause a slowdown in execution time and a loss in data scalability. To tackle this problem, we enhance a previously published memory-conscious MapReduce over MPI framework called Mimir. Our enhancements to Mimir include combiner and dynamic repartition optimizations to minimize and balance memory usage and to achieve close to optimal balance of the memory usage across processes and to reduce the execution time by up to 12 times. Experimental results show that Mimir can scale to at least 3072 processes on the Tianhe-2 supercomputer on skewed datasets.

A three-dimensional unified gas-kinetic wave-particle solver for flow computation in all regimes

In this paper, the unified gas-kinetic wave-particle (UGKWP) method has been constructed on a three-dimensional unstructured mesh with parallel computing for multiscale flow simulation. Based on the direct modeling methodology, the unified gas-kinetic scheme (UGKS) models the flow dynamics directly on the numerical mesh size and time step scales, and it is able to capture the flow dynamics from the kinetic scale particle transport to the hydrodynamic wave propagation seamlessly according to the local cell Knudsen number. Instead of discretizing the particle velocity space in UGKS, the UGKWP method is composed of evolution of deterministic wave and stochastic particles. With dynamic wave-particle decomposition according to the cell Knudsen number, the UGKWP method is able to capture the continuum wave interaction and rarefied particle transport under a unified framework and achieves high efficiency in different flow regimes. The UGKWP flow solver is constructed in three-dimensional space and is validated by many test cases at different Mach and Knudsen numbers. The examples include a 3D shock tube problem, lid-driven cubic cavity flow, high-speed flow passing through a cubic object, and hypersonic flow around a space vehicle. The parallel performance has been tested on the Tianhe-2 supercomputer, and reasonable parallel performance has been observed up to 1000 cores. With the wave-particle formulation, the UGKWP method has great potential in solving three-dimensional multiscale transport problems with the co-existence of continuum and rarefied flow regimes, especially for the high-speed rarefied and continuum flow simulation around a space vehicle in near-space flight, where the local Knudsen number can vary significantly with five or six orders of magnitude differences.

Parallel multilevel restricted Schwarz preconditioners for implicit simulation of subsurface flows with Peng-Robinson equation of state

Parallel algorithms and simulators with good scalabilities are particularly important for large-scale reservoir simulations on modern supercomputers with a large number of processors. In this paper, we introduce and study a family of highly scalable multilevel restricted additive Schwarz (RAS) methods for the fully implicit solution of subsurface flows with Peng-Robinson equation of state in two and three dimensions. With the use of a second-order fully implicit scheme, the proposed simulator is unconditionally stable with the relaxation of the time step size by the stability condition. The investigation then focuses on the development of several types of multilevel overlapping additive Schwarz methods for the preconditioning of the resultant linear system arising from the inexact Newton iteration, and some fast solver technologies are presented for the assurance of the multilevel approach efficiency and scalability. We numerically show that the proposed fully implicit framework is highly efficient for solving both standard benchmarks as well as realistic problems with several hundreds of millions of unknowns and scalable to 8192 processors on the Tianhe-2 supercomputer.

A hybrid modeling approach for optimal design of non-woven membrane channels in brackish water reverse osmosis process with high-throughput computation

Optimal design of feed spacers in spiral wound membrane (SWM) modules has attracted extensive attention in industry and academia due to their widespread applications. However, it is still mathematically difficult and computationally expensive to solve many CFD models with various design parameters. In this work, a hybrid framework that couples a system-level model with a CFD model neglecting concentration polarization (CP) phenomenon is developed to analyze the performance of brackish water reverse osmosis (BWRO) processes under industrial operating conditions using non-woven SWM modules. The simulation results are in satisfactory agreement with measurement data, while computational efficiency is about ten times of that of a previous model accounting for the CP phenomenon. Furthermore, an efficient heuristic optimal design workflow that couples a high-throughput computation (HTC) strategy with the consideration of the field synergy equation is proposed to tackle the sophisticated hydraulic optimization problems within the scope of the considered design space. For this purpose, a total number of 1958 CFD models are solved with various geometries and inlet velocities at 51 nodes (1224 cores) on Tianhe-2 supercomputer by applying the proposed HTC strategy. Using the optimized spacer structure, the average permeation flux of water is enhanced by about 9%.

Study and Verification of Large-Scale Parallel Mesh Generation Algorithm for Centrifugal Pump

In order to reveal the details of the internal flow in a centrifugal pump, a large-scale mesh is needed. However, the mesh generated by the serial grid algorithm cannot meet the calculation requirements due to the huge amount of time. A large-scale parallel mesh generation algorithm of a centrifugal pump for high-performance computers is presented in this paper. First, a grid point set for the 3D Delaunay triangular mesh on the surface of the centrifugal pump is generated. Then, the S-H (Sutherland–Hodgman) algorithm for cropping and segmenting these grid point sets on the surface is employed. A uniform boundary mesh is generated and is divided into different subregions. In addition, in order to ensure the consistency of the interface mesh and to avoid the boundary mesh intersection overlap error, a parallel constrained Delaunay mesh generation algorithm based on region numbering is proposed, which can improve the quality and efficiency of the generated parallel mesh. Finally, the centrifugal pump is tested for verifying the parallel mesh algorithm in the Tianhe-2 supercomputer. PIV (particle image velocimetry) internal flow experiment is comparatively analyzed with the numerical simulation of large-scale mesh. The results show that the algorithm can generate 108 3D unstructured grid elements in 5 minutes, and the parallel efficiency can achieve 80%. The proposed algorithm not only ensures high grid quality with the serial grid algorithm but also accurately simulates the flow law in the centrifugal pump. The double-vortex structure which is obtained by PIV experiment is captured by the large-scale mesh.

An improved approach to simulate seismic events based on cellular automata

Earthquake is a very complex process of geological fracture. Seismic events simulation is significant in the study on the underlying physical laws of earthquakes. Since the computation ability of regular computers is limited, traditional approaches to simulate seismic events are oversimplified without considering enough physical factors, so that the simulation results are too far from the truth. To overcome the shortcoming, an improved approach is proposed in this paper to simulate seismic events more physically, utilizing supercomputers. Material characteristics and geometrical characteristics are introduced to build a more realistic model. Taking the advantage of TianHe-2 supercomputer in Chinese National Supercomputer Center in Guangzhou, this approach produces simulation results with very reasonable microscopic and macroscopic behaviors. As expected, Gutenberg–Richter scaling is observed in agreement with real earthquakes. Besides, more physical details, such as the distribution of released energy in seismic events, are revealed. This approach helps to explore physical laws in seismic process and even predict earthquakes.

General-Purpose Quantum Circuit Simulator with Projected Entangled-Pair States and the Quantum Supremacy Frontier.

Recent advances on quantum computing hardware have pushed quantum computing to the verge of quantum supremacy. Here, we bring together many-body quantum physics and quantum computing by using a method for strongly interacting two-dimensional systems, the projected entangled-pair states, to realize an effective general-purpose simulator of quantum algorithms. The classical computing complexity of this simulator is directly related to the entanglement generation of the underlying quantum circuit rather than the number of qubits or gate operations. We apply our method to study random quantum circuits, which allows us to quantify precisely the memory usage and the time requirements of random quantum circuits. We demonstrate our method by computing one amplitude for a 7×7 lattice of qubits with depth (1+40+1) on the Tianhe-2 supercomputer.

Parallel and fully implicit simulations of the thermal convection in the Earth’s outer core

A parallel and fully implicit method is developed for numerical simulations of the thermal convection of an incompressible fluid in a rotating spherical shell. The method is based on a pseudo-compressibility approach with dual time stepping in order to tackle with the numerical difficulty of the solenoidal condition in the incompressible Navier-Stokes equations. The numerical solution of the nonlinear governing equations is based on the method of lines, whereby the partial differential equations are transformed into ordinary differential equations by a suitable spatial discretization. A second-order cell-centered finite volume discretization based on a collocated cubed-sphere grid is used here. In order to relax the time step limit and accurately integrate the semi-discrete nonlinear ordinary differential equations in time, we employ a second-order explicit-first-step, single-diagonal-coefficient, diagonally implicit Runge–Kutta (ESDIRK) method with adaptive time stepping. The nonlinear algebraic system arising at each pseudo-time step is solved by a Newton–Krylov–Schwarz algorithm to achieve good load balance on modern supercomputers. The numerical results are in good agreement with the benchmark solutions and more accurate than those obtained with a fractional step approach in our previous research. Large-scale tests on the Tianhe-2 supercomputer indicate that the fully implicit solver can achieve good parallel scalabilities in both strong and weak senses.

Parallel visualization of large-scale multifield scientific data

Following the recent rapid growth in supercomputer performance, many real-world problems in fields such as nuclear fusion energy and electromagnetic environments can be solved via multiphysics simulation, which outputs multifield datasets. However, current multifield visualization has difficulty handling multiphysics parallel simulation data. First, it is difficult to correctly visualize overlapping multifield data with semitransparent properties because of the complex distribution of partitioned data domains across multicore processors. Second, the interactive visualization performance of large-scale multifield data in serial processing mode on a personal computer is often slow because multiphysics simulations can produce large-scale datasets, i.e., of the order of gigabytes to terabytes. Considering the fidelity and efficiency of large-scale data visualization on supercomputer, a new parallel visualization method is required for multifield scientific data that do not change the original distribution of the mesh data generated by the multiphysics applications. This paper introduces a hybrid scheduling framework for the parallel visualization of large-scale multifield scientific data. This framework is used to overcome problems both in correct visual representation and in efficient visualization of large-scale multiphysics applications. We discuss the results of several typical multiphysics applications to verify the feasibility and reliability of our proposed framework. This framework currently supports scalable in situ visualization of up to 8.5 billion mesh cells on the 10 k cores of China’s Tianhe-2 supercomputer, which could help domain scientists understand multiphysics phenomena more clearly and accurately.

Parallel reservoir simulators for fully implicit complementarity formulation of multicomponent compressible flows

The numerical simulation of multicomponent compressible flow in porous media is an important research topic in reservoir modeling. Traditional semi-implicit methods for such problems are usually conditionally stable, suffer from large splitting errors, and may accompany with violations of the boundedness requirement of the numerical solution. In this study we reformulate the original multicomponent equations into a nonlinear complementarity problem and discretize it using a fully implicit finite element method. We solve the resultant nonsmooth nonlinear system of equations arising at each time step by a parallel, scalable, and nonlinearly preconditioned semismooth Newton algorithm, which is able to preserve the boundedness of the solution and meanwhile treats the possibly imbalanced nonlinearity of the system. Some numerical results are presented to demonstrate the robustness and efficiency of the proposed algorithm on the Tianhe-2 supercomputer for both standard benchmarks as well as realistic problems in highly heterogeneous media.

Development of the integrated parallelism strategy for large scale depletion calculation in the Monte Carlo code RMC

The Monte Carlo (MC) method is becoming more and more attractive for high fidelity simulations in reactor physics. The huge memory consumption is the bottleneck in the three dimensional (3D) full core detailed depletion calculations. Three strategies were proposed to overcome the memory problem, including domain decomposition, data decomposition and hybrid parallelism. These methods can solve this problem in some extent, but each of them has its own limitation. In this paper, the integrated parallelism platform which combines these three strategies was proposed and developed in the Reactor Monte Carlo (RMC) code. Based on this platform, the performances of different combinations of these three strategies were investigated on PWR assembly and PWR full-core models on the Tianhe-2 supercomputer. The results show that “domain decomposition + hybrid parallelism” performed better than “tally data decomposition + hybrid parallelism” at the scale of above 0.6 million burnup regions. Moreover, the batch method was adopted to reduce the frequency of collective communications and thus save computational time. Finally, the strategy of “domain decomposition + burnup data decomposition + hybrid parallelism” with the batch method was proposed and tested on Tianhe-2 supercomputer compute nodes, equipped with 64 Gigabytes (GB) of memory. The proposed strategy can handle about twenty millions of burnup regions with high efficiency, which is the important foundation of full core detailed depletion calculation and high fidelity simulation.

MCtandem: an efficient tool for large-scale peptide identification on many integrated core (MIC) architecture

BackgroundTandem mass spectrometry (MS/MS)-based database searching is a widely acknowledged and widely used method for peptide identification in shotgun proteomics. However, due to the rapid growth of spectra data produced by advanced mass spectrometry and the greatly increased number of modified and digested peptides identified in recent years, the current methods for peptide database searching cannot rapidly and thoroughly process large MS/MS spectra datasets. A breakthrough in efficient database search algorithms is crucial for peptide identification in computational proteomics.ResultsThis paper presents MCtandem, an efficient tool for large-scale peptide identification on Intel Many Integrated Core (MIC) architecture. To support big data processing capability, a novel parallel match scoring algorithm, named MIC-SDP (spectrum dot product), and its two-level parallelization are presented in MCtandem’s design. In addition, a series of optimization strategies on both the host CPU side and the MIC side, which includes pre-fetching, optimized communication overlapping scheme, multithreading and hyper-threading, are exploited to improve the execution performance.ConclusionsFor fair comparisons, we first set up experiments and verified the 28 fold times speedup on a single MIC against the original CPU-based implementation. We then execute the MCtandem for a very large dataset on an MIC cluster (a component of the Tianhe-2 supercomputer) and achieved much higher scalability than in a benchmark MapReduce-based programs, MR-Tandem. MCtandem is an open-source software tool implemented in C++. The source code and the parameter settings are available at https://github.com/LogicZY/MCtandem.

Analyzing time-dimension communication characterizations for representative scientific applications on supercomputer systems

Exascale computing is one of the major challenges of this decade, and several studies have shown that communications are becoming one of the bottlenecks for scaling parallel applications. The analysis on the characteristics of communications can effectively aid to improve the performance of scientific applications. In this paper, we focus on the statistical regularity in time-dimension communication characteristics for representative scientific applications on supercomputer systems, and then prove that the distribution of communication-event intervals has a power-law decay, which is common in scientific interests and human activities. We verify the distribution of communication-event intervals has really a power-law decay on the Tianhe-2 supercomputer, and also on the other six parallel systems with three different network topologies and two routing policies. In order to do a quantitative study on the power-law distribution, we exploit two groups of statistics: bursty vs. memory and periodicity vs. dispersion. Our results indicate that the communication events show a “strong-bursty and weak-memory” characteristic and the communication event intervals show the periodicity and the dispersion. Finally, our research provides an insight into the relationship between communication optimizations and time-dimension communication characteristics.

Numerical simulation of O<sub>2</sub>/CO<sub>2</sub> combustion in large cement precalciner

The cement industry is the second largest source of CO2 emissions after the power industry. The greenhouse effect caused by CO2 is increasing nowadays, so CO2 emission reduction is urgent, the cement precalciner kiln O2/CO2 combustion technology has great application value in CO2 emission reduction. This paper selects 3200 t/d TTF cement precalciner as the research object, and Tianhe-2 supercomputer is used to carry out massive parallel CFD numerical simulation. Numerical simulation was used to study on the effects of pulverized coal combustion mixed air with pulverized coal combustion mixed O2/CO2 on velocity field, temperature field, material composition distribution and NO x concentration distribution. The results show that the O2/CO2 combustion technology has similar velocity field distribution and raw material decomposition ratio when compared with the conventional air combustion, and it will not affect the operation of the precalciner. Besides, the over-temperature zone in the precalciner is reduced, the temperature-brushing phenomenon is effectively alleviated, and the NO x concentration at the outlet reduced greatly from 995 to 96 mg/m3, so that the post-treatment De-NO x process can be removed to reduce investments. Furthermore, the nearly pure high-concentration CO2 in the flue gas is easy to be captured and reused, alleviating the greenhouse effect.

An Improved Algorithm for Drift Diffusion Transport and Its Application on Large Scale Parallel Simulation of Resistive Random Access Memory Arrays

A hybrid finite volume–finite element method which can avoid overestimation of volume shared by each vertex in the meshing grid is proposed to solve the diffusive transport governed continuity equation. The simulation results demonstrate that the improved algorithm can eliminate unphysical distortions as compared to the conventional Scharfetter Gummel method. Based on the proposed algorithm, a parallel-computation simulator is developed for large-scale electrothermal simulation of resistive random access memory (RRAM) arrays, in which the domain decomposition method and J parallel adaptive unstructured mesh applications infrastructure are adopted. The validity, speedup, and scalability of the parallel simulator are investigated on the TianHe-2 supercomputer. Based on the simulated results, the electrothermal characteristics and reliability analysis of large-scale RRAM arrays are investigated in detail.