Parallel Vector Processor Research Articles

PIMCaffe: Functional Evaluation of a Machine Learning Framework for In-Memory Neural Processing Unit

The large amount of memory usage in recent machine learning applications imposes a significant system burden with respect to power and processing speed. To cope with such a problem, Processing-In-Memory (PIM) techniques can be applied as an alternative solution. Especially, the recommendation system, which is one of the major machine learning applications used in data centers, requires a large memory capacity and therefore represents a suitable candidate application that could be helped by the PIM technique. In this paper, we introduce a machine learning framework, PIMCaffe, designed for in-memory neural processing units and its evaluation environment. PIMCaffe consists of two components: a Caffe2-based deep learning framework that supports PIM acceleration and a PIM-emulating hardware platform. We develop a suite of functions, libraries, application programming interfaces, and a device driver to support the framework. In addition, we implement a prototype Neural Processing Unit (NPU) in PIMCaffe to evaluate the performance of our platform with machine learning applications. Our prototype NPU design includes a vector processor for parallel vector processing and a systolic array unit for matrix multiplication. Using the proposed software framework, we perform a detailed analysis of the in-memory neural processing unit. PIMCaffe supports evaluations of recommendation systems and various convolutional neural network models on the in-memory neural processing unit. PIMCaffe with the NPU shows up to 2.26×, 5.99×, and 1.71× speedup, compared to the ARM Cortex-A53 CPU, for the recommendation system, AlexNet, and ResNet-50, respectively.

A Fast Algorithm to Estimate the Deepest Points of Lakes for Regional Lake Registration.

When conducting image registration in the U.S. state of Alaska, it is very difficult to locate satisfactory ground control points because ice, snow, and lakes cover much of the ground. However, GCPs can be located by seeking stable points from the extracted lake data. This paper defines a process to estimate the deepest points of lakes as the most stable ground control points for registration. We estimate the deepest point of a lake by computing the center point of the largest inner circle (LIC) of the polygon representing the lake. An LIC-seeking method based on Voronoi diagrams is proposed, and an algorithm based on medial axis simplification (MAS) is introduced. The proposed design also incorporates parallel data computing. A key issue of selecting a policy for partitioning vector data is carefully studied, the selected policy that equalize the algorithm complexity is proved the most optimized policy for vector parallel processing. Using several experimental applications, we conclude that the presented approach accurately estimates the deepest points in Alaskan lakes; furthermore, we gain perfect efficiency using MAS and a policy of algorithm complexity equalization.

Operational Wind Wave Prediction System at KMA

An overview of the current operational wind wave prediction system and of new developments at the Korea Meteorological Administration (KMA) is presented. KMA has operated a numerical ocean wave prediction system since 1992. The first major upgrade was done in 1999 with the adaptation of a third-generation wave model (WAM) for both regional (ReWAM-0.25 deg resolution, covering Northeast Asia) and global (GoWAM-1.25 deg resolution) domains. In 2005, the KMA replaced its NEC SX5 computer with a 1024-CPU Cray X1E system, a Parallel Vector Processor (PVP) machine with 128 node modules. A coastal ocean wave prediction system (CoWAM) has been designed and is currently under test mode. Six CoWAM domains, each of 3 deg longitude by 2 deg latitude in size, with a mesh size of 1 km, are nested inside the regional ocean wave prediction system. The directional wave spectra at the boundaries of the CoWAM are provided from a 1/12° resolution upgraded version of the operational ReWAM. The WAVEWATCH-III code (developed at NOAA) is used for the upgraded ReWAM and new CoWAM system. To ensure the required model performance in the Massively Parallel Processor (MPP) architecture of the new supercomputer, a Message Passing Interface (MPI) has been implemented in the model source code. The sea surface wind and significant wave height are verified routinely on a monthly basis. The global moored buoy data (including the coastal buoys operated by KMA) and remote sensing data from Topex/Poseidon, and Jason (retrieved wave heights) and QuikSCAT (retrieved wind data) are used for verification of the wave prediction system. This validation and a comparison of results from the new ReWAM and CoWAM against buoy data off the coast of Korea are discussed.

Large scale FMO-MP2 calculations on a massively parallel-vector computer

The fragment molecular orbital (FMO) calculations have been successfully applied to a variety of realistic biochemical problems, by using our original A binit-mp program. In these applications, the inclusion of electron correlation through the second-order Møller-Plesset perturbation (MP2) was demonstrated to be essential to obtain qualitatively correct descriptions. Recently, the FMO calculations in A binit-mp were tuned for a massively parallel-vector processing. A series of FMO-MP2/6-31G calculations were performed on the Earth Simulator by which up to 4096 vector processors are available. The largest FMO-MP2 computation was carried out for an influenza hemagglutinin antigen–antibody system consisting of 921 residues, which was completed within one hour with 4096 processors.

Allostery of the two‐state model of hemoglobin studied by ECEPP energy minimization

Recently, a powerful parallel-vector processor became available for molecular science. A new FORTRUN program was coded to treat the whole hemoglobin molecule with twofold symmetry. Using the X-ray coordinates of deoxyhemoglobin and oxyhemoglobin, minimum energy conformations were obtained for both the T-state and the R-state on the two-state model of allostery. From them, further energy minimization was performed with simple perturbation to bring the proximal region close to the heme group instead of oxygen binding, and the structural changes and energy changes were investigated. The difference of calculated energy changes between T and R was semiquantitatively in agreement with the experimental value 2.7 kcal/mol for one oxygen binding. When the perturbation was exerted on the alpha-subunits, the energy change within the perturbed alpha-subunits in the T-state gave a main contribution, and in the R-state, the structural changes of the alpha-subunits were specifically large. When the perturbation was exerted on the beta-subunits, the intersubunit interaction energy between alpha1 and beta2 (alpha2 and beta1) was dominant in the difference of the energy changes between T and R.

Multibillion-atom molecular dynamics simulation: Design considerations for vector-parallel processing

Progress in adapting molecular dynamics algorithms for systems with short-range interactions to utilize the features of modern supercomputers is described. Efficient utilization of the latest generation of processor architectures requires algorithms that can be both vectorized and parallelized. The approach adopted for vectorization involves combining the layer and neighbor-list methods, while parallelization employs spatial subdivision with explicit communication. The techniques presented here have been used in performance tests on the Cray X1 vector-parallel supercomputer with systems containing over 12 billion atoms.

Assessment of computational performance for a vector parallel implementation: 3D probabilistic model discrete cracking in concrete

This work presents an assessment of the computational performance of a vector-parallel implementation of probabilistic model for concrete cracking in 3D. This paper shows the continuing efforts towards code optimization as reported in earlier works Paz, et al. (2002a,b and 2003). The probabilistic crack approach is based on the direct Monte Carlo method. Cracking is accounted by means of 3D interface elements. This approach considers that all nonlinearities are restricted to interface elements modeling cracks. The heterogeneity governs the overall cracking behavior and related size effects on concrete fracture. Computational kernels in the implementation are the inexact Newton iterative driver to solve the non-linear problem and a preconditioned conjugate gradient (PCG) driver to solve linearized equations, using an element by element (EBE) strategy to compute matrix-vector products. In particular the paper analyzes code behavior using OpenMP directives in parallel vector processors (PVP), such as the CRAY SV1 and CRAY T94. The impact of the memory architecture on code performance, and also some strategies devised to circumvent this issue are addressed by numerical experiment.

Application of Double-Fourier-series Spectral Method to a Large Size Problem: Two-dimensional Simulations of the Shear Instability on the Sphere

Double-Fourier-Series (DFS) spectral method is applied to a large-size problem of barotropic instability of double-shear flow on the sphere. The computing source is the NEC SX-5 parallel vector processors, with the maximum vector length of 512. It is demonstrated that the DFS spectral model is robust and stable even for such a large-sized intensively nonlinear problem, and can simulate well the multiple scale phenomenon without losing accuracy. In addition to the efficiency on serial computing, represented with O(N² log2 N) operations as opposed to O(N³) for the spherical harmonics spectral method, with N the truncation, the DFS spectral model also preserves the efficiency on parallel computing on vector architecture, due to its nature of two dimensional transform. The parallel performance increased slightly with the resolution, and nearly 33.5 percent (26.8 GFLOPS) of the theoretical peak performance (80 GFLOPS) was achieved in the highest-resolution experiment. The zonal-mean absolute vorticity, of which initial condition is characterized as two peaks in both hemispheres, evolves with time into a nearly constant value over the hemisphere. On the other hand, the meridional gradient of the absolute vorticity increases around the equator. The kinetic energy per unit mass is calculated for each total wavenumber, where a disturbance field of a single total wavenumber is separated by an 8th-order spherical harmonics filter. Kinetic energy spectrum shows two distinct subranges, each with a constant slope. The subrange, other than the viscous subrange, shows a slightly increasing slope with time and approaches l-3 (l is the total wavenumber) in the matured stage, when a single large vortex is formed. As the resolution increases, the subrange other than the viscous subrange extends to the higher wavenumber domain, due to low viscosity. Numerical convergence of the solution with a fixed viscosity is discussed in terms of time averaged zonal-mean statistics of the zonal-flow.

Large scale atomistic polymer simulations using Monte Carlo methods for parallel vector processors

In this paper we discuss the implementation of advanced variable connectivity Monte Carlo (MC) simulation methods for studying large (>10 5 atom) polymer systems at the atomic level. Such codes are intrinsically difficult to optimize since they involve a mixture of many different elementary MC steps, such as reptation, flip, end rotation, concerted rotation and volume fluctuation moves. In particular, connectivity altering MC moves, such as the recently developed directed end bridging (DEB) algorithm, are required in order to vigorously sample the configuration space. Techniques for effective vector implementation of such moves are described. We also show how a simple domain decomposition method can provide a general and efficient means of parallelizing these complex MC protocols. Benchmarks are reported for a 192,000 atom simulation of polydisperse linear polyethylene with an average chain length C 6000, for simulations using 1 to 8 processors and a variety of MC protocols.

A parallel 3D unsteady incompressible flow solver on VPP700

In this study, MIMD parallel computation of coupled non-linear partial differential equations governing the mass and momentum conservation of a 3D viscous incompressible unsteady flow on distributed memory-based vector parallel processing system (VPP700) under UXP/VPPF90 has been discussed. The parallel numerical implementation is based on a time accurate cell centered finite volume method in conjunction with Roe's flux difference splitting of non-linear terms and pseudo-compressibility technique. Data flow graphs necessary for combined parallelization and vectorization have been generated to provide a logical basis for parallel code development. The performance of the parallel implementation measured in terms of speedup and efficiency factors is found to be good. The parallel code is tested on three test problems of practical significance.

Adapting EMAP3D to parallel processing [for EM launcher modelling

EMAP3D (Electromechanical Analysis Program in Three Dimensions) is a single-processor (serial) program that uses finite element (FE) methods to solve coupled electromagnetic, thermal and structural problems for high velocity conductors in transient electromagnetic fields. Its primary application has been the simulation of electromagnetic launchers and pulsed rotating machines. While ETVAP3D has been applied successfully to a wide range of problems, its serial execution time limits the achievement of finer detail in large problems, even on high performance vector machines. The present class of production simulations, involving 100,000 unknowns, can take a week to complete on time-sharing machines at computation centers. To reduce simulation time, a parallel implementation of the EMAPSD program has been undertaken. While vector parallel programming is straightforward and a reasonable approach, access to more than 16 vector processors is limited. The availability, low cost, and scalability of massively parallel processing (MPP) make MPP computing more attractive than vector-parallel processing. The number of MPP processors on PC Beowulf clusters usually ranges from 8 to 100 and can be as high as several thousand on IBM (SP) and Gray (T3E) systems. The authors have decoupled the matrix generation and components of the FE algorithm, thereby allowing us to use any of the new MPP-parallel solvers on the matrix equations. Since the number of zero matrix elements is high for EMAPSD problems, a sparse matrix solver is ideal. Hence, their parallel implementation uses the sparse iterative solvers of PETSc (Portable Extensible Toolkit for Scientific Computing). Here, they report the performance, scalability and use of PETSc preconditioners and solver algorithms as a solution engine for real EMAP3D simulations and test cases.

Multiprocessor design options and the Silicon Graphics S2MP architecture

Most multiprocessors can be classified as one of four types: message-passing or MPP systems, clusters, shared-memory multiprocessors, and parallel vector processors. This paper discusses a new model, the scalable shared memory multiprocessor (SSMP), which removes the drawbacks of the traditional SMP systems. The first part of this paper reviews current multiprocessor alternatives, and introduces the scalable SMP architecture. The second part of the paper focuses on a particular instance of a scalable SMP, the Silicon Graphics Origin multiprocessor and its S2MP memory architecture. We give an overview of the Origin system and then discuss some of the important design trade-offs in the architecture and implementation of the system. In the final section we examine technology trends and predict that clusters of scalable shared-memory multiprocessors will become the dominate multiprocessor architecture.

Automatic differentiation for design sensitivity analysis of structural systems using parallel-vector processors

Automatic differentiation tools, ADIFOR [1](Bischof, C. H. and Griewank, A., Proc. 4th AIAA/USAF/NASA/OAI Symposium on Multidisciplinary Analysis and Optimization, Cleveland, OH, pp. 433–441, 1992), are incorporated into a finite element-based structural analysis program for shape and non-shape sensitivity analysis of structural systems [2, 3](Arora, J. S. and Haug, E. J., Applied Optimal Design, John Wiley and Sons, Inc., 1991; Hafta, R. T., Gurdal, Z. and Kamat, M. P., Elements of Structural Optimization, Kluwer, 1990). The entire analysis and sensitivity procedures are parallelized and vectorized for high-performance computation [4, 5](Agarwal, T. K., Storaasli, O. O. and Nguyen, D. T., Proc. AIAA/ASME/ASCE/AHS 31st SDM Conference, Long Beach, CA, 1990; Qin, J. and Nguyen, D. T., Proc. AIAA/ASME/ASCE/AHS 34th SDM Conference, LaJolla, CA, 1993). Small-scale examples to verify the accuracy of the proposed program and a medium-scale example to demonstrate the parallel-vector performance on the Cray-C90 processors are included in the paper.

Difficulties in Vector-Parallel Processing of Monte Carlo Codes

Experiences with vectorization of production-level Monte Carlo codes such as KENO-IV, MCNP, VIM, and MORSE have shown that it is difficult to attain high speedup ratios on vector processors because of indirect addressing, nests of conditional branches, short vector length, cache misses, and operations for realization of robustness and generality. A previous work has already shown that the first, second, and third difficulties can be resolved by using special computer hardware for vector processing of Monte Carlo codes. Here, the fourth and fifth difficulties are discussed in detail using the results for a vectorized version of the MORSE code. As for the fourth difficulty, it is shown that the cache miss-hit ratio affects execution times of the vectorized Monte Carlo codes and the ratio strongly depends on the number of the particles simultaneously tracked. As for the fifth difficulty, it is shown that remarkable speedup ratios are obtained by removing operations that are not essential to the specific problem being solved. These experiences have shown that if a production-level Monte Carlo code system had a capability to selectively construct source coding that complements the input data, then the resulting code could achieve much higher performance.

NON-LINEAR STRUCTURAL RESPONSE USING ADAPTIVE DYNAMIC RELAXATION ON A MASSIVELY PARALLEL-PROCESSING SYSTEM

A parallel adaptive dynamic relaxation (ADR) algorithm has been developed for nonlinear structural analysis. This algorithm has minimal memory requirements, is easily parallelizable and scalable to many processors, and is generally very reliable and efficient for highly nonlinear problems. Performance evaluations on single-processor computers have shown that the ADR algorithm is reliable and highly vectorizable, and that it is competitive with direct solution methods for the highly nonlinear problems considered. The present algorithm is implemented on the 512-processor Intel Touchstone DELTA system at Caltech, and it is designed to minimize the extent and frequency of interprocessor communication. The algorithm has been used to solve for the nonlinear static response of two and three dimensional hyperelastic systems involving contact. Impressive relative speedups have been achieved and demonstrate the high scalability of the ADR algorithm. For the class of problems addressed, the ADR algorithm represents a very promising approach for parallel-vector processing.

NON‐LINEAR STRUCTURAL RESPONSE USING ADAPTIVE DYNAMIC RELAXATION ON A MASSIVELY PARALLEL‐PROCESSING SYSTEM

A parallel adaptive dynamic relaxation (ADR) algorithm has been developed for non-linear structural analysis. This algorithm has minimal memory requirements, is easily parallelizable and scalable to many processors, and is generally very reliable and efficient for highly non-linear problems. Performance evaluations on single-processor computers have shown that the ADR algorithm is reliable and highly vectorizable, and that it is competitive with direct solution methods for the highly non-linear problems considered. The present algorithm is implemented on the 512-processor Intel Touchstone DELTA system at Caltech, and it is designed to minimize the extent and frequency of interprocessor communication. The algorithm has been used to solve for the non-linear static response of two- and three-dimensional hyperelastic systems involving contact. Impressive relative speed-ups have been achieved and demonstrate the high scalability of the ADR algorithm. For the class of problems addressed, the ADR algorithm represents a very promising approach for parallel-vector processing.

Weather forecasting on parallel architectures

Global forecasting at the National Meteorological Center is performed using a spherical harmonics spectral model. The model is used in global data assimilation, and in the aviation and medium range forecast production suites. In this presentation, the major numerical and physical aspects of the model are highlighted, and parallelization considerations are analyzed and applied to three parallel computer architectures. The first parallelization is on a parallel vector processor shared memory platform such as Cray YMP or C90. The other two are on distributed memory platforms: a data parallel implementation on Connection Machines and a single program multiple data (SPMD) adaptation to a Cray T3D machine.

Adaptive dynamic relaxation algorithm for non-linear hyperelastic structures Part II. Single-processor implementation

An efficient single-processor implementation of the Adaptive Dynamic Relaxation (ADR) algorithm is developed. It is designed to exploit data locality and can exploit vectorization of the finite element computations. The ADR algorithm is used to solve for the non-linear static response of two- and three-dimensional hyperelastic systems involving frictionless contact. Performance is compared with an existing finite element code which utilizes a direct solution method. ADR is found to be reliable and highly vectorizable, and it outperforms the direct solution method for the highly non-linear problems considered. In addition, it permits the use of a very simple and efficient contact algorithm. In contrast to direct solution methods, ADR has minimal memory requirements and is easily parallelizable and scalable to more processors. For the class of problems addressed, it represents a very promising approach for parallel-vector processing.

Parallel processing in Japan: national and corporate trends

This paper provides an idea of recent developments in Japan's parallel processing technologies. Workstation clusters, vector parallel processing, the dataflow model, and certain core technologies stand out. National projects transfer technology to business well, but their focus on specific applications has had mixed results.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Implementation of a synthetic aperture processing scheme in a towed array sonar system

This experimental paper is concerned with the implementation of a passive synthetic aperture processing scheme in a towed array sonar system. It addresses theoretical and practical concerns, and the experimental results suggest that the concept of a synthetic aperture towed array system complements the conventional beamformer’s processing results. The acoustic synthetic aperture technique, which has been considered in this study, is the ETAM algorithm [i.e., extended towed array measurements, J. Acoust. Soc. Am. 87, 158–171 (1989)]. Theoretical and experimental studies have shown that the above technique compensates the random phase fluctuations in the data that are introduced by the tow path. This results is an effective extension of the physical aperture of a deployed towed array and consequently provides improved angular resolution performance equivalent to a fully populated long array. Also included in this presentation is a discussion of the development of the computing architecture requirements related to this kind of implementation. It is shown that the development of a next generation signal processor (NGSP) by the Defence Research Establishment Atlantic (DREA) overcomes the limitations normally associated with the implementation of nonlinear algorithms into parallel FFT-based vector processors.