Vector-parallel Supercomputer Research Articles

Potential of a modern vector supercomputer for practical applications: performance evaluation of SX-ACE

Achieving a high sustained simulation performance is the most important concern in the HPC community. To this end, many kinds of HPC system architectures have been proposed, and the diversity of the HPC systems grows rapidly. Under this circumstance, a vector-parallel supercomputer SX-ACE has been designed to achieve a high sustained performance of memory-intensive applications by providing a high memory bandwidth commensurate with its high computational capability. This paper examines the potential of the modern vector-parallel supercomputer through the performance evaluation of SX-ACE using practical engineering and scientific applications. To improve the sustained simulation performances of practical applications, SX-ACE adopts an advanced memory subsystem with several new architectural features. This paper discusses how these features, such as MSHR, a large on-chip memory, and novel vector processing mechanisms, are beneficial to achieve a high sustained performance for large-scale engineering and scientific simulations. Evaluation results clearly indicate that the high sustained memory performance per core enables the modern vector supercomputer to achieve outstanding performances that are unreachable by simply increasing the number of fine-grain scalar processor cores. This paper also discusses the performance of the HPCG benchmark to evaluate the potentials of supercomputers with balanced memory and computational performance against heterogeneous and cutting-edge scalar parallel systems.

Numerical modeling of three dimensional self-gravitating Stokes flow problem with free surface

A solution code of the Stokes flow system in three dimensions under a self-gravitating field was developed with a free surface in order to simulate the long-time scale evolution of the solid earth system on the massively parallel-vector super computer Earth Simulator 2 (ES2). The largely deformable surface of a planetary body is represented by a sharp boundary of scalar color function, which is advected by the CIP-CLSR method. The self-gravitating field is calculated by the Poisson equation of the gravity potential. A low viscous material (sticky air) surrounding the planet mimics its free surface motion. In order to solve the ill conditioned Stokes problem of the finite difference discretization on a staggered grid, I employ the iterative Stokes flow solver, robust to large viscosity jumps, using a strong Schur complement preconditioner and mixed precision arithmetic utilizing the double-double method. In this paper, I present an overview of the numerical algorithms and the current parallel programming strategy for the efficient performance on ES2. In addition, preliminary results of the core formation simulation are demonstrated in three dimensions as an example of the target Stokes flow system of this simulation code.

Multibillion-atom molecular dynamics simulation: Design considerations for vector-parallel processing

Progress in adapting molecular dynamics algorithms for systems with short-range interactions to utilize the features of modern supercomputers is described. Efficient utilization of the latest generation of processor architectures requires algorithms that can be both vectorized and parallelized. The approach adopted for vectorization involves combining the layer and neighbor-list methods, while parallelization employs spatial subdivision with explicit communication. The techniques presented here have been used in performance tests on the Cray X1 vector-parallel supercomputer with systems containing over 12 billion atoms.

Development of the Efficient Electromagnetic Particle Simulation Code with High Performance Fortran on a Vector-Parallel Supercomputer

Development of the Efficient Electromagnetic Particle Simulation Code with High Performance Fortran on a Vector-Parallel Supercomputer

20503 稠密炉心内3次元気泡流構造に関する大規模シミュレーション(大規模数値解析(1),OS11 大規模数値解析)

In order to predict the water-vapor two-phase flow structure in a fuel bundle of an advanced light water reactor, a large-scale numerical simulation was carried out using a newly developed two-phase flow analysis code and a highly parallel vector supercomputer. Conventional analysis methods with a two-fluid model need composition equations and empirical correlations based on the experimental data. Therefore, it is difficult to obtain high prediction accuracy when experimental data are nothing. Then, a new two-phase flow analysis method based on the latest computational science was proposed. This paper describes the predicted bubbly flow dynamics in three-dimensional minichannels and the predicted void fractions in the simulated tight-lattice fuel channels. Regarding the coalescence and fragmentation of bubbles the useful knowledge was obtained, and also the high prospect was acquired on the possibility of development of the thermal design procedure of the advanced nuclear reactors with large-scale simulations.

EARTH SIMULATOR AND SIMULATION RESEARCH PROGRESS

Earth simulator is a parallel vector super-computer which developed by National Space Development Agency of Japan(NASDA)and Japan Atomic Energy Research Institute(JAERI)and Japan Agency for Marine-Earth Science and Technology(JAMSTEC).Understanding and forecasting high-impact phenomena in the atmosphere and ocean,global warm,crust.The performance of Earth Simulator,birth of the Earth Simulator,research contents of five groups which are Atmosphere and Ocean Simulation Research Group,Solid Earth Simulation Research Group,Multiscale Simulation Research Group,Advanced Perception Research Group and Holistic Simulation Research Group,research projects and collaborations are introduced.At last,usage details details of the Earth Simulator and sustained performance of actual application are introduced.

Vectorization and Parallelization of the Adaptive Mesh Refinement N-Body Code

Abstract We describe our vectorized and parallelized adaptive mesh refinement (AMR) $N$-body code with shared time steps, and report its performance on a Fujitsu VPP5000 vector-parallel supercomputer. Our AMR $N$-body code puts hierarchical meshes recursively where higher resolution is required and the time step of all particles are the same. The parts that are the most difficult to vectorize are loops that access the mesh data and particle data. We vectorized such parts by changing the loop structure, so that the innermost loop steps through the cells instead of the particles in each cell, in other words, by changing the loop order from the depth-first order to the breadth-first order. We also parallelized our code for distributed memory machines. The important part of parallelization is data decomposition. We sorted the hierarchical mesh data by the Morton order, or the recursive N-shaped order, level by level and split and allocated the mesh data to the processors. Particles are allocated to the processor to which the finest refined cells including the particles are also assigned. Our timing analysis using $\Lambda$-dominated cold dark matter simulations shows that our parallel code speeds up almost ideally up to 32 processors, the largest number of processors in our test.

The RIKEN Super Combined Cluster as a Grid-Enabled System (Anwendung von Grid-Technologien beim Super Combined Cluster von RIKEN)

Summary We developed the RIKEN Super Combined Cluster (hereafter “RSCC”) System, and it started operation in March 2004. It consists of a vector parallel supercomputer, five PC clusters, and a PC cluster with MDGRAPE-2 special purpose computers. We use grid computing technology so that the user can regard the system as one system, although actually it is composed of seven different computer systems.

Development of the Efficient Electromagnetic Particle Simulation Code with High Performance Fortran on a Vector-Parallel Supercomputer

A three-dimensional, relativistic, electromagnetic particle simulation code is developed using the “Exact Charge Conservation Scheme”on a vector-parallel supercomputer. This scheme is a method for calculating current densities. Applying this method, correction of electric fields is not needed. In this paper, some techniques to optimize the above scheme for a vector-parallel supercomputer are shown. The method for vectorization and parallelization in shared memory and in distributed memories are discussed. The code is written in Fortran90 and High Performance Fortran (HPF). Examination of this code is also made.

Hardware system of the Earth Simulator

The Earth Simulator (ES), developed under the Japanese government’s initiative “Earth Simulator project”, is a highly parallel vector supercomputer system. In this paper, an overview of ES, its architectural features, hardware technology and the result of performance evaluation are described. In May 2002, the ES was acknowledged to be the most powerful computer in the world: 35.86 teraflop/s for the LINPACK HPC benchmark and 26.58 teraflop/s for an atmospheric general circulation code (AFES). Such a remarkable performance may be attributed to the following three architectural features; vector processor, shared-memory and high-bandwidth non-blocking interconnection crossbar network. The ES consists of 640 processor nodes (PN) and an interconnection network (IN), which are housed in 320 PN cabinets and 65 IN cabinets. The ES is installed in a specially designed building, 65 m long, 50 m wide and 17 m high. In order to accomplish this advanced system, many kinds of hardware technologies have been developed, such as a high-density and high-frequency LSI, a high-frequency signal transmission, a high-density packaging, and a high-efficiency cooling and power supply system with low noise so as to reduce whole volume of the ES and total power consumption. For highly parallel processing, a special synchronization means connecting all nodes, Global Barrier Counter (GBC), has been introduced.

D-26 超ベクトル並列計算機による都市域の気温および風のCFD解析 : 東京の5km四方を事例として

D-26 超ベクトル並列計算機による都市域の気温および風のCFD解析 : 東京の5km四方を事例として

Waveform inversion for 3-D earth structure using the Direct Solution Method implemented on vector-parallel supercomputer

We implement the Direct Solution Method (DSM) on a vector-parallel supercomputer and show that it is possible to significantly improve its computational efficiency through parallel computing. We apply the parallel DSM calculation to waveform inversion of long period (250–500 s) surface wave data for three-dimensional (3-D) S-wave velocity structure in the upper and uppermost lower mantle. We use a spherical harmonic expansion to represent lateral variation with the maximum angular degree 16. We find significant low velocities under south Pacific hot spots in the transition zone. This is consistent with other seismological studies conducted in the Superplume project, which suggests deep roots of these hot spots. We also perform simultaneous waveform inversion for 3-D S-wave velocity and Q structure. Since resolution for Q is not good, we develop a new technique in which power spectra are used as data for inversion. We find good correlation between long wavelength patterns of Vs and Q in the transition zone such as high Vs and high Q under the western Pacific.

狭隘流路内の大規模気泡流挙動に関する数値シミュレーション(S19-3 混相流の多次元ビジュアリゼーション(気液・気泡), S19 混相流の多次元ビジュアリゼーション)

In order to predict the water-vapor two-phase flow structure in a fuel bundle of an advanced light water reactor, a large-scale numerical simulation was carried out using a newly developed two-phase flow analysis code and a highly parallel vector supercomputer. Conventional analysis methods such as subchannel codes need composition equations based on the experimental data. In case that there are no experimental data for the advanced light water reactor, therefore, it is difficult to obtain high prediction accuracy. Then, a new analysis method using the large-scale direct numerical simulation of water-vapor two-phase flow was proposed. This paper, describes the predicted results of a large-scale bubbly flow in a narrow square duct. Regarding the mechanism of bubble coalescence and fragmentation the useful knowledge was obtained.

Approaches that exploit vector-parallelism for three rendering and volume visualization techniques

Direct volume rendering using volume ray-casting and indirect volume rendering using the Marching Cubes isosurface extraction are popular techniques for volume visualization. Surface ray-tracing is a popular graphics technique for rendering scenes composed of well-defined surface primitives. However, these techniques are relatively computationally intensive. Thus, near-real-time computational performance is a difficult goal. This paper presents approaches to these rendering and volume visualization techniques that are tuned for efficient performance on a vector-parallel supercomputer. The approaches decompose and reconstruct the techniques to exploit inherent data parallelism and the specific characteristics of the CPU. Experimental results for several datasets are also exhibited.

Computational chemistry on Fujitsu vector–parallel processors: Hardware and programming environment

In this and the following paper, we provide an introduction to the Fujitsu VPP range of vector–parallel supercomputers and to some of the computational chemistry software available for the VPP. Here, we consider the hardware and the design of software to exploit its capabilities. The VPP employs proprietary vector processors connected via a crossbar switch in a distributed-memory architecture. High single-node performance requires consideration of vector operand lengths, arithmetic pipe utilisation and memory-to-CPU bandwidth. Most parallel chemistry applications use either explicit `message-passing' or a ‘global-memory’ paradigm, and benchmark results are presented for the communications performance of MPI, Linda and the Global Arrays.

Computational chemistry on Fujitsu vector–parallel processors: Development and performance of applications software

In this and a preceding paper, we provide an introduction to the Fujitsu VPP range of vector–parallel supercomputers and to some of the computational chemistry software available for the VPP. Here, we consider the implementation and performance of seven popular chemistry application packages. The codes discussed range from classical molecular dynamics to semiempirical and ab initio quantum chemistry. All have evolved from sequential codes, and have typically been parallelised using a replicated data approach. As such they are well suited to the large-memory/fast-processor architecture of the VPP. For one code, CASTEP, a distributed-memory data-driven parallelisation scheme is presented.

304大規模非定常数値シミュレーションのためのリアルタイム可視化システム

We have developed a concurrent visualization system RVSLIB (Real-time Visual Simulation Library). This paper shows the effectiveness of the system when it is applied to large-scale unsteady simulations, for which the conventional post-processing approach may no longer work, on high-performance parallel vector supercomputers. The system performs almost all of the visualization tasks on a computation server and uses compressed visualized image data for efficient communication between the server and the user terminal. We have introduced several techniques, including vectorization and parallelization, into the system to minimize the computational costs of the visualization tools. The performance of RVSLIB was evaluated by using an actual CFD code on an NEC SX-4. The computational time increase due to the concurrent visualization was at most 3% for a smaller (1.6 million) grid and less than 1% for a larger (6.2 million) one.

Ordering strategies and related techniques to overcome the trade-off between parallelism and convergence in incomplete factorizations

This paper is concerned with the parallel implementation of the incomplete factorization preconditioned iterative method. Although the use of such parallel ordering as multicolor ordering may increase parallelism in factorization, it often slows convergence when used in the preconditioned method, and thus may offset the gain in speed obtained with parallelization. Further, the higher the parallelism of an ordering, the slower the convergence; the lower the parallelism, the faster the convergence. This well-known trade-off between parallelism and convergence is well explained by the property of compatibility, the level of which can be clearly seen when ordering is presented in graph form (S. Doi, A. Lichnewsky, A graph-theory approach for analyzing the effects of ordering on ILU preconditioning, INRIA report 1452, 1991). In any given method, the fewer the incompatible local graphs in an ordering (i.e., the lower the parallelism), the faster the convergence (S. Doi, Appl. Numer. Math. 7 (1991) 417–436; S. Doi, in: T. Nodera (Ed.), Advances in Numerical Methods for Large Sparse Sets of Linear Systems, 7 Keio University, 1991). An ordering with no incompatible local graphs, for example, such as that implemented on vector multiprocessors by using the nested dissection technique, will have excellent convergence, but its parallelism will be limited (S. Doi, A. Lichnewsky, Int. J. High Speed Comput. 2 (1990) 143–179). To attain a better balance, a certain degree of incompatibility is necessary. In this regard, increasing the number of colors in multicolor ordering can be a useful approach (S. Fujino, S. Doi, in: R. Beauwens (Ed.), Proceeding of the IMACS Internation Symposium on Iterative Methods in Linear Algebra, March 1991; S. Doi, A. Hoshi, Int. J. Comput. Meth. 44 (1992) 143–152). Two related techniques also presented here are the overlapped multicolor ordering (T. Washio, K. Hayami, SIAM J. Sci. Comput. 16 (1995) 631–650), and a fill-in strategy selectively applied to incompatible local graphs. Experiments conducted with an SX-5/16A vector parallel supercomputer show the relative effectiveness of increasing the number of colors and also of using this approach in combination with overlapping and with fill-ins.

Equation-of-Motion O(N) Electronic Structure Studies of Very Large Systems (N ~ 10 7 )

Extremely fast parallel implementation of the equation-of-motion method for electronic structure computations is presented. The method can be applied to non-periodic, disordered nanocrystalline samples, transition metal oxides and other systems. It scales linearly, O(N), runs with a speed of up to 43 GFLOPS on a NEC SX-4 vector-parallel supercomputer with 32 processors and computes electronic densities of states (DOS) for multi-million atom samples in mere minutes. The largest test computation performed was for the electronic DOS for a TiO2 sample consisting of 7,623,000 atoms. Mathematically, this is equivalent to obtaining the spectrum of an n × n Hermitian operator (Hamiltonian) where n = 38;115; 000. We briefly discuss the practical implications of being able to perform electronic structure computations of this great speed and scale.

The Recent Revolution in High Performance Computing

Recent events in the high-performance computing industry have concerned scientists and the general public regarding a crisis or a lack of leadership in the field. That concern is understandable considering the industry's history from 1993 to 1996. Cray Research, the historic leader in supercomputing technology, was unable to survive financially as an independent company and was acquired by Silicon Graphics. Two ambitious new companies that introduced new technologies in the late 1980s and early 1990s—Thinking Machines and Kendall Square Research—were commercial failures and went out of business. And Intel, which introduced its Paragon supercomputer in 1994, discontinued production only two years later.During the same time frame, scientists who had finished the laborious task of writing scientific codes to run on vector parallel supercomputers learned that those codes would have to be rewritten if they were to run on the next-generation, highly parallel architecture. Scientists who are not yet involved in high-performance computing are understandably hesitant about committing their time and energy to such an apparently unstable enterprise.However, beneath the commercial chaos of the last several years, a technological revolution has been occurring. The good news is that the revolution is over, leading to five to ten years of predictable stability, steady improvements in system performance, and increased productivity for scientific applications. It is time for scientists who were sitting on the fence to jump in and reap the benefits of the new technology.