Hybrid Supercomputer Research Articles

Visualization Methods of Vortex Structures Acting on the Wing in High-Speed Flows

This paper considers the use of vortex structures scientific identification and visualization methods in analyzing the effects of these structures on the wing at an angle of attack in supersonic flows. The results of applying the methods of scientific identification and visualization of vortex flows for the analysis and processing of numerical data of supersonic flow are presented. In the considered problem, the generator wing and the main wing are located at an angle of attack of 20° degrees to the incoming flow. Two configurations with different length of the vortex generator are considered. The simulation was carried out using URANS equations. Numerical simulations were performed on the hybrid supercomputer system K-60 at the Keldysh Institute of Applied Mathematics of the Russian Academy of Sciences. The scientific visualization of the simulation results was carried out primarily using the Liutex vortex structures identification criterion, which belongs to the third generation of such methods. And this method was effectively complemented and enhanced by the use of other methods.

Canonical Monte Carlo multispin cluster method

We present a new Canonical Multispin-flip Cluster Monte Carlo algorithm for Ising model and describe efficient implementations for hybrid supercomputer. Our method takes advantage of the computing architecture for parallel and multi-threaded operations and uses a sequential cluster state update scheme. Due to the peculiarity of the implementation, the method is more effective for models with a restricted radius of interaction. It is based on combining a random selection of spin cluster by the Monte Carlo method with a complete enumeration of the all states of the selected cluster. To show how it works we applied our method to models of interacting magnetic Ising-moments: ferromagnetic Ising model, the Edwards–Anderson spin glass model, dipolar spin ice on hexagonal and pentagonal Cairo lattices.

Влияние концевого вихря крыла-генератора на обтекание основного крыла под углом атаки

The effect of a tip vortex from an upstream generator wing on the flow around the main wing in a supersonic flow is considered. Numerical results were obtained on the multiprocessor hybrid supercomputer system K-60 at the Keldysh Institute of Applied Mathematics RAS.

Modeling of the Natural Objects’ Temperature Field Distribution Using a Supercomputer

There are some regions where the unique type of mineral water mining is compounded by the complex structure conditions of the field. Due to the high emergency risks, the automation of this type of mineral water fields’ exploitation process is a necessity. Kislovodskoe mineral water field was chosen as the experimental object, because it has a number of features that make it barely possible to use the conventional methods of mineral water extraction. In the process of extraction, the random vector impacts the mineral water pumping systems. This is mainly due to the complex hydrogeological structure. For the experiment, the laboratory installation and mathematical model were presented by the temperature field changing, due to the similitude of the mathematical apparatus and the hydrodynamic processes behavior,. The main object of this article is the analysis of the reservoir’s temperature field behavior using a hybrid supercomputer, and the differences between the supercomputer and a conventional personal computer modeling quality and implementation costs are also given..

Сравнение моделей турбулентности для расчета сверхзвукового концевого вихря

In the paper the turbulence models SA, SST and DES based on SA are compared for simulation of supersonic tip vortex at incoming flow Mach number M? = 3 and wing generator attack angles of 10 and 20 degrees. The simulations were carried out on the hybrid supercomputer system K-60 at the Keldysh Institute of Applied Mathematics RAS.

Computer modeling of air flows purification using sorption filters

The problem of computer modeling the processes of air purification from nanoparticles using sorption filters is considered. The paper proposes a complex model for describing the flow of contaminated air in the sorbent layer. The model is based on a system of quasigasdynamic equations defining the main characteristics of the air medium flow containing the mixture of gases and finely dispersed pollutants. Micro-and nanoparticles of heavy metals, coal and household dust can be considered as contaminants. The model takes into account the diffusion of pollutant particles, their adsorption on the surface of the sorbent granules and absorption inside the sorbent granules. The model is implemented numerically using splitting by physical processes and the mesh method. The basic algorithm is the finite volume method on irregular grids. An important part of the model is its parallel implementation adapted to computations on hybrid supercomputer systems. The preliminary calculations confirmed the robustness of the developed modeling technology.

Algorithm for replica redistribution in an implementation of the population annealing method on a hybrid supercomputer architecture

The population annealing method is a promising approach for large-scale simulations because it is potentially scalable on any parallel architecture. We present an implementation of the algorithm on a hybrid program architecture combining CUDA and MPI. The problem is to keep all general-purpose graphics processing unit devices as busy as possible by efficiently redistributing replicas. We provide details of testing on hardware based the Intel Skylake/Nvidia V100 running more than two million replicas of the Ising model sample in parallel. The results are quite encouraging because the acceleration grows toward the perfect line as the complexity of the simulated system increases.

Methodology of Effective Task Planning and Algorithm for Multivariate Computation of the Characteristics of a Quantum Rotation Sensor on a Hybrid Supercomputer Cluster

The problem of planning supercomputer computations is considered using the example of computing the characteristics of a quantum rotation sensor model. A methodology for applying the task scheduling algorithm is proposed. For this purpose, time estimates were obtained for computations at cluster nodes. The possibility of dividing settlements by cluster nodes without additional synchronization costs is shown. A start-up algorithm for improving the efficiency of multivariate computations is presented, and the efficiency of the planning algorithm in its use is estimated.

TopoLens: Building a CyberGIS community data service for enhancing the usability of high‐resolution national topographic datasets

SummaryIn recent years, geospatial data have exploded to massive volume and diversity and subsequently cause serious usability issues for researchers in various scientific areas. This paper describes a cyberGIS community data service framework to facilitate geospatial big data access, processing, and sharing based on a hybrid supercomputer architecture. Specifically, the framework aims to enhance the usability of national elevation dataset released by the U.S. Geological Survey in the contiguous United States at the resolution of arc‐second. A community data service, namely TopoLens, is created to demonstrate the workflow integration of national elevation dataset and the associated computation and analysis. Two user‐friendly environments, including a publicly available web application and a private workspace based on the Jupyter notebook, are provided for users to access both precomputed and on‐demand computed high‐resolution elevation data. The system architecture of TopoLens is implemented by exploiting the ROGER supercomputer, the first cyberGIS supercomputer dedicated to geospatial problem‐solving. The usability of TopoLens has been acknowledged in the topographic user community evaluation.

The “CPU+GPU+MIC” Hybrid Supercomputer π

The “CPU+GPU+MIC” Hybrid Supercomputer π

Multilevel parallelization for simulating compressible turbulent flows on most kinds of hybrid supercomputers

The paper describes a multilevel MPI+OpenMP+OpenCL parallelization approach that provides complete portability across a wide range of hybrid supercomputer architectures. A parallel CFD algorithm for heterogeneous computing of turbulent flows is presented. It simulates the compressible Navier–Stokes equations using a cell-centered finite-volume method with polynomial reconstruction on unstructured hybrid meshes. A two-level partitioning is used for the workload distribution among computing devices of hybrid nodes. The overlap of communications and computations hides the data transfer expenses. The scalability is tested on various HPC systems including a fat node with 8 GPUs and supercomputers using up to 320 GPUs. Comparison of performance is presented for multicore CPUs, Intel Xeon Phi, various GPUs of AMD and NVIDIA. The heterogeneous execution using CPUs and GPUs is studied in detail.

A data‐driven approach to modeling power consumption for a hybrid supercomputer

SummaryPower consumption of current High Performance Computing systems has to be reduced by at least one order of magnitude before they can be scaled up towards ExaFLOP performance. While we can expect novel hardware technologies and architectures to contribute towards this goal, significant advances have to come also from software technologies such as proactive and power‐aware scheduling, resource allocation, and fault‐tolerant computing. Development of these software technologies in turn relies heavily on our ability to model and accurately predict power consumption in large computing systems. In this paper, we present a data‐driven model of power consumption for a hybrid supercomputer (which held the top spot in the Green500 ranking in June 2013) that combines CPU, GPU, and MIC technologies to achieve high levels of energy efficiency. Our model takes as input workload characteristics—the number and location of resources that are used by each job at a certain time—and calculates a predicted power consumption at the system level. The model is application‐code‐agnostic and is based solely on a data‐driven predictive approach, where log data describing the past jobs in the system are employed to estimate future power consumption. For this, three different model components are developed and integrated. The first employs support vector regression to predict power usage for jobs before these are started. The second uses a simple heuristic to predict the length of jobs, again before they start. The two predictions are then combined to estimate power consumption due to the job at all computational elements in the system. The third component is a linear model that takes as input the power consumption at the computing units and predicts system‐wide power consumption. Our method achieves highly‐accurate predictions starting solely from workload information and user histories. The model can be applied to power‐aware scheduling and power capping: alternative workload dispatching configurations can be evaluated from a power perspective and more efficient ones can be selected. The methodology outlined here can be easily adapted to other HPC systems where the same types of log data are available.

Algorithms for planning Resource-Intensive computing tasks in a hybrid supercomputer environment for simulating the characteristics of a quantum rotation sensor and performing engineering calculations

The article considers the problem of planning computing tasks in hybrid supercomputer systems using the example of the Polytechnic Supercomputer Center. We have explored the classes of tasks using the distribution of calculations planning and shown their specific character when using hybrid supercomputer resources. With this in mind, we have developed the algorithms to optimize the use of resources of supercomputer systems. Furthermore, we have proposed an algorithm predicting task execution time taking into account the characteristics of the computation node and task parameters. Finally, after conducting the research we have compared the developed algorithms.

HPCG and HPGMG benchmark tests on multiple program, multiple data (MPMD) mode on Blue Waters—A Cray XE6/XK7 hybrid system

SummaryThe high‐performance conjugate gradients (HPCG) and high‐performance geometric multi‐grid (HPGMG) benchmarks are alternatives to the traditional LINPACK benchmark (HPL) in measuring the performance of modern HPC platforms. We performed HPCG and HPGMG benchmark tests on a Cray XE6/XK7 hybrid supercomputer, Blue Waters at National Center for Supercomputing Applications (NCSA). The benchmarks were tested on CPU‐based and GPU‐enabled nodes separately, and then we analyzed characteristic parameters that affect their performance. Based on our analyses, we performed HPCG and HPGMG runs in multiple program, multiple data (MPMD) mode in Cray Linux Environment in order to measure their hybrid performance on both CPU‐based and GPU‐enabled nodes. We observed and analyzed several performance issues during those tests. Based on lessons learned from this study, we provide recommendations about how to optimize science applications on modern hybrid HPC platforms.

Efficient CFD code implementation for the ARM-based Mont-Blanc architecture

Since 2011, the European project Mont-Blanc has been focused on enabling ARM-based technology for HPC, developing both hardware platforms and system software. The latest Mont-Blanc prototypes use system-on-chip (SoC) devices that combine a CPU and a GPU sharing a common main memory. Specific developments of parallel computing software and well-suited implementation approaches are of crucial importance for such a heterogeneous architecture in order to efficiently exploit its potential.This paper is devoted to the optimizations carried out in the TermoFluids CFD code to efficiently run it on the Mont-Blanc system. The underlying numerical method is based on an unstructured finite-volume discretization of the Navier–Stokes equations for the numerical simulation of incompressible turbulent flows. It is implemented using a portable and modular operational approach based on a minimal set of linear algebra operations. An architecture-specific heterogeneous multilevel MPI+OpenMP+OpenCL implementation of such kernels is proposed. It includes optimizations of the storage formats, dynamic load balancing between the CPU and GPU devices and hiding of communication overheads by overlapping computations and data transfers. A detailed performance study shows time reductions of up to 2.1× on the kernels’ execution with the new heterogeneous implementation, its scalability on up to 128 Mont-Blanc nodes and the energy savings (around 40%) achieved with the Mont-Blanc system versus the high-end hybrid supercomputer MinoTauro.

Evaluation of External Memory Access Performance on a High-End FPGA Hybrid Computer

The motivation of this research was to evaluate the main memory performance of a hybrid super computer such as the Convey HC-x, and ascertain how the controller performs in several access scenarios, vis-à-vis hand-coded memory prefetches. Such memory patterns are very useful in stencil computations. The theoretical bandwidth of the memory of the Convey is compared with the results of our measurements. The accurate study of the memory subsystem is particularly useful for users when they are developing their application-specific personality. Experiments were performed to measure the bandwidth between the coprocessor and the memory subsystem. The experiments aimed mainly at measuring the reading access speed of the memory from Application Engines (FPGAs). Different ways of accessing data were used in order to find the most efficient way to access memory. This way was proposed for future work in the Convey HC-x. When performing a series of accesses to memory, non-uniform latencies occur. The Memory Controller of the Convey HC-x in the coprocessor attempts to cover this latency. We measure memory efficiency as a ratio of the number of memory accesses and the number of execution cycles. The result of this measurement converges to one in most cases. In addition, we performed experiments with hand-coded memory accesses. The analysis of the experimental results shows how the memory subsystem and Memory Controllers work. From this work we conclude that the memory controllers do an excellent job, largely because (transparently to the user) they seem to cache large amounts of data, and hence hand-coding is not needed in most situations.

Massively parallel algorithm and implementation of RI-MP2 energy calculation for peta-scale many-core supercomputers

A new parallel algorithm and its implementation for the RI-MP2 energy calculation utilizing peta-flop-class many-core supercomputers are presented. Some improvements from the previous algorithm (J. Chem. Theory Comput. 2013, 9, 5373) have been performed: (1) a dual-level hierarchical parallelization scheme that enables the use of more than 10,000 Message Passing Interface (MPI) processes and (2) a new data communication scheme that reduces network communication overhead. A multi-node and multi-GPU implementation of the present algorithm is presented for calculations on a central processing unit (CPU)/graphics processing unit (GPU) hybrid supercomputer. Benchmark results of the new algorithm and its implementation using the K computer (CPU clustering system) and TSUBAME 2.5 (CPU/GPU hybrid system) demonstrate high efficiency. The peak performance of 3.1 PFLOPS is attained using 80,199 nodes of the K computer. The peak performance of the multi-node and multi-GPU implementation is 514 TFLOPS using 1349 nodes and 4047 GPUs of TSUBAME 2.5. © 2016 Wiley Periodicals, Inc.

Optimizing legacy molecular dynamics software with directive-based offload

Directive-based programming models are one solution for exploiting many-core coprocessors to increase simulation rates in molecular dynamics. They offer the potential to reduce code complexity with offload models that can selectively target computations to run on the CPU, the coprocessor, or both. In this paper, we describe modifications to the LAMMPS molecular dynamics code to enable concurrent calculations on a CPU and coprocessor. We demonstrate that standard molecular dynamics algorithms can run efficiently on both the CPU and an x86-based coprocessor using the same subroutines. As a consequence, we demonstrate that code optimizations for the coprocessor also result in speedups on the CPU; in extreme cases up to 4.7X. We provide results for LAMMPS benchmarks and for production molecular dynamics simulations using the Stampede hybrid supercomputer with both Intel® Xeon Phi™ coprocessors and NVIDIA GPUs. The optimizations presented have increased simulation rates by over 2X for organic molecules and over 7X for liquid crystals on Stampede. The optimizations are available as part of the “Intel package” supplied with LAMMPS.

AstroPhi: A code for complex simulation of the dynamics of astrophysical objects using hybrid supercomputers

We propose a new code named AstroPhi for simulation of the dynamics of astrophysical objects on hybrid supercomputers equipped with Intel Xenon Phi computation accelerators. The details of parallel implementation are described, as well as changes to the computational algorithm that facilitate efficient parallel implementation. A single Xeon Phi accelerator yielded 27-fold acceleration. The use of 32 Xeon Phi accelerators resulted in 94% parallel efficiency. Several collapse problems are simulated using the AstroPhi code. Program summaryProgram title: AstroPhiCatalogue identifier: AEUM_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEUM_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 99604No. of bytes in distributed program, including test data, etc.: 305433Distribution format: tar.gzProgramming language: C++.Computer: MVS-10P - RSC Tornado, Xeon E5-2690 8C 2.900 GHz, Infiniband FDR, Intel Xeon Phi SE10X.Operating system: Linux.Has the code been vectorized or parallelized?: Parallelized on MPI + OpenMP for Intel MIC architecture, 32 Intel Xeon Phi (60 cores per 1 Intel Xeon Phi = 1920 cores of Intel Xeon Phi).RAM: 137438953472 bytes (128 GB) bytesClassification: 1.9.External routines: MPI, OpenMP for Intel Xeon Phi, FFTW 2.1.5Nature of problem:Complex numerical simulation of dynamics of astrophysical objects plays an important role due to significant growth of observational astronomic data. The new astrophysical models and codes need to be developed for detailed simulation of different physical effects in astrophysics with the use of modern supercomputers with hybrid architecture.Solution method:AstroPhi code consisting of particle-in-cell and Godunov methods combination adapted for hybrid supercomputer architecture.Restrictions:For this version maximum grid size is restricted to 10243.Running time:Typical running on MVS-10P is 24 h. The test provided only takes a few minutes.

Отображение на кластеры с графическими процессорами DVMH-программ с регулярными зависимостями по данным.

In the 2011 year DVMH programming model for new heterogeneous and hybrid supercomputer systems (or DVM for Heterogeneous systems) was introduced in the Keldysh Institute for Applied Mathematics of RAS. The developed high-level programming languages were based on standard Fortran and C programming languages, but extended with the directives for mapping the program onto a parallel computer. The directives are represented as special comments (or pragmas). The paper describes problems and methods for mapping loops, which have dependencies to the GPU. Efficiency of the developed Fortran DVMH parallel programs with regular dependencies is demonstrated.