Shared Memory Environment Research Articles

New fast divide‐and‐conquer algorithms for the symmetric tridiagonal eigenvalue problem

SummaryIn this paper, two accelerated divide‐and‐conquer (ADC) algorithms are proposed for the symmetric tridiagonal eigenvalue problem, which cost O(N2r) flops in the worst case, where N is the dimension of the matrix and r is a modest number depending on the distribution of eigenvalues. Both of these algorithms use hierarchically semiseparable (HSS) matrices to approximate some intermediate eigenvector matrices, which are Cauchy‐like matrices and are off‐diagonally low‐rank. The difference of these two versions lies in using different HSS construction algorithms, one (denoted by ADC1) uses a structured low‐rank approximation method and the other (ADC2) uses a randomized HSS construction algorithm. For the ADC2 algorithm, a method is proposed to estimate the off‐diagonal rank. Numerous experiments have been carried out to show their stability and efficiency. These algorithms are implemented in parallel in a shared memory environment, and some parallel implementation details are included. Comparing the ADCs with highly optimized multithreaded libraries such as Intel MKL, we find that ADCs could be more than six times faster for some large matrices with few deflations. Copyright © 2016 John Wiley & Sons, Ltd.

Use of Multiple GPUs to Speedup the Execution of a Three-Dimensional Computational Model of the Innate Immune System

The development of computational systems that mimics the physiological response of organs or even the entire body is a complex task. One of the issues that makes this task extremely complex is the huge computational resources needed to execute the simulations. For this reason, the use of parallel computing is mandatory. In this work, we focus on the simulation of temporal and spatial behaviour of some human innate immune system cells and molecules in a small three-dimensional section of a tissue. To perform this simulation, we use multiple Graphics Processing Units (GPUs) in a shared-memory environment. Despite of high initialization and communication costs imposed by the use of GPUs, the techniques used to implement the HIS simulator have shown to be very effective to achieve this purpose.

Shared-memory and shared-nothing stochastic gradient descent algorithms for matrix completion

We provide parallel algorithms for large-scale matrix completion on problems with millions of rows, millions of columns, and billions of revealed entries. We focus on in-memory algorithms that run either in a shared-memory environment on a powerful compute node or in a shared-nothing environment on a small cluster of commodity nodes; even very large problems can be handled effectively in these settings. Our ASGD, DSGD-MR, DSGD++, and CSGD algorithms are novel variants of the popular stochastic gradient descent (SGD) algorithm, with the latter three algorithms based on a new "stratified SGD" approach. All of the algorithms are cache-friendly and exploit thread-level parallelism, in-memory processing, and asynchronous communication. We investigate the performance of both new and existing algorithms via a theoretical complexity analysis and a set of large-scale experiments. The results show that CSGD is more scalable, and up to 60 % faster, than the best-performing alternative method in the shared-memory setting. DSGD++ is superior in terms of overall runtime, memory consumption, and scalability in the shared-nothing setting. For example, DSGD++ can solve a difficult matrix completion problem on a high-variance matrix with 10M rows, 1M columns, and 10B revealed entries in around 40 min on 16 compute nodes. In general, algorithms based on SGD appear to perform better than algorithms based on alternating minimizations, such as the PALS and DALS alternating least-squares algorithms.

A High Performance Computing Cloud Computing Environment for Machining Simulations

Machining is a pervasive manufacturing process used in industries such as automotive, aerospace, medical implants and oil and gas. Analysis of processes via physics-based modeling enables new and innovative designs for cutting tools, provides confidence of machined workpiece quality characteristics and allows reduction in machining cycle times and tooling costs. Progressively sophisticated analyses of machining processes have evolved with the inclusion of effects of full three-dimensional analysis of cutting tools and complex tool/workpiece kinematics. Detailed-level analysis of machined workpiece surfaces based on finite element method (FEM) allows prediction of residual stresses, work hardened layer depths and heat flow. However, with the increase in model sophistication has come with computational burden. This paper details a high performance computing (HPC) environment for finite element models used for machining analysis. First, the FEM model is reviewed and its extension to high core-count shared memory environments is described. Scaled performance improvements for up to forty cores are demonstrated and performance improvements documented. Next, an HPC cluster is designed and a specialized batch queuing software is implemented that allows submission, monitoring and management of large scale machining simulations. Finally, an infrastructure for delivering the HPC capability to customers through Software as a Service (SaaS) is introduced.

A Comparative Study on Different Parallel Solvers for Nonlinear Analysis of Complex Structures

The parallelization of 2D/3D software SAPTIS is discussed for nonlinear analysis of complex structures. A comparative study is made on different parallel solvers. The numerical models are presented, including hydration models, water cooling models, modulus models, creep model, and autogenous deformation models. A finite element simulation is made for the whole process of excavation and pouring of dams using these models. The numerical results show a good agreement with the measured ones. To achieve a better computing efficiency, four parallel solvers utilizing parallelization techniques are employed: (1) a parallel preconditioned conjugate gradient (PCG) solver based on OpenMP, (2) a parallel preconditioned Krylov subspace solver based on MPI, (3) a parallel sparse equation solver based on OpenMP, and (4) a parallel GPU equation solver. The parallel solvers run either in a shared memory environment OpenMP or in a distributed memory environment MPI. A comparative study on these parallel solvers is made, and the results show that the parallelization makes SAPTIS more efficient, powerful, and adaptable.

Parallel 3D-TLM algorithm for simulation of the Earth-ionosphere cavity

A parallel 3D algorithm for solving time-domain electromagnetic problems with arbitrary geometries is presented. The technique employed is the Transmission Line Modeling (TLM) method implemented in Shared Memory (SM) environments. The benchmarking performed reveals that the maximum speedup depends on the memory size of the problem as well as multiple hardware factors, like the disposition of CPUs, cache, or memory. A maximum speedup of 15 has been measured for the largest problem. In certain circumstances of low memory requirements, superlinear speedup is achieved using our algorithm. The model is employed to model the Earth-ionosphere cavity, thus enabling a study of the natural electromagnetic phenomena that occur in it. The algorithm allows complete 3D simulations of the cavity with a resolution of 10km, within a reasonable timescale.

Improved Arbitrary size Benes Network

interconnection networks (MINs) are used to connect N inputs to N outputs. They are mainly used to connect processor to processor and for processor to memory in distributed and shared memory environment. The MINs are broadly divided into three categories Blocking Non Blocking and Rearrangeable networks. A new improved Arbitrary size Benes network has been proposed in this to improve the permutation capabilities and to reduce the cost of existing Arbitrary Size Benes Network.

Frequency response computation of structures including non-proportional damping in a shared memory environment

With the increased size of the finite element model for improved accuracy, the modal frequency response analysis has been one of the common practices of evaluating the performance of vehicle dynamics. However, there is difficulty in predicting the vehicle dynamics response with non-proportional damping regarding performance. The fast frequency response analysis algorithm (FFRA) has been proved to be very effective for partially damped structural system in the modal frequency response analysis. In the fast frequency response analysis algorithm, performance depends mainly on the complex symmetric matrix eigenvalue problem. Therefore, an efficient complex symmetric matrix eigenvalue problem solver is developed in this article. This approach also uses parallel processing in a shared memory machine for more efficient analysis. Numerical examples show that the new complex symmetric matrix eigensolver provides good accuracy and high performance. Then, the fast frequency response analysis algorithm is applied to a full scale vehicle system that includes only a few viscous damping finite elements. The fast frequency response analysis algorithm significantly improves the performance of the modal frequency response analysis compared to conventional method. In addition, parallel processing improves the efficiency of the overall simulation.

Comparative study of distributed implementation using Tuple space technology of two t-way test suite generation strategies

This paper highlights a comparative study of two new distributed t-way test suite generation strategies, called test suite generator one parameter (TS_OP) and test suite generator one test (TS_OT). Both strategies adopt computational greedy algorithm and were implemented on distributed shared memory environment using Tuple space technology. To characterize the behavior of both strategies in term of test size growth and test generation time, both TS_OP and TS_OT were subjected to a few experimentations with varied parameter, parameter value and interaction strength. Furthermore, to determine their scalability performance in term of speedup gained and test size ratio, both strategies were subjected to scalability analysis on single machine and multiple machine environments. An encouraging result on speedup is obtained for both strategies thus indicating the effectiveness of using Tuple space technology in distributing the t-way test suite generation computing work. A comparison against existing strategies in terms of the generated test suite size also indicates that both strategies give sufficiently competitive results. Furthermore, in comparison between both strategies on test suite size, TS_OP gives more satisfactory and competitive results as compared to TS_OT strategy. Key words: T-way testing, distributed test suite generation, map and reduce, tuple space technology.

Memory-aware Thread and Data Mapping for Hierarchical Multi-core Platforms

In parallel programs, the threads of a given application must cooperate in order to accomplish the required computation. However, the communication time between the tasks may be different depending on which core they are executing and how the memory hierarchy and interconnection are used. The problem is even more important in multi-core machines with NUMA characteristics, since the remote access imposes high overhead, making them more sensitive to thread and data mapping. In this context, thread and data mapping are techniques that provide performance gains by improving the use of resources such as interconnections, main memory and cache memory. The problem of detecting the best mapping is considered NP-Hard. Furthermore, in shared memory environments, there is an additional difficulty of finding the communication pattern, which is implicit and occurs through memory accesses. Our mechanism provides static mapping on NUMA architectures which does not require any prior knowledge of the application by the programmer. To obtain the mapping, different metrics were adopted and an heuristic method based on the Edmonds matching algorithm was used. In order to evaluate our proposal, we use the NAS Parallel Benchmarks (NPB) running on two modern multi-core NUMA machines. Results show performance gains of up to 75% compared to the native Linux scheduler and memory allocator.

Multi-GPU Implementation of LU Factorization

LU factorization is the most computationally intensive step in solving systems of linear equations. By obtaining first the LU factorization of the coefficient matrix, we then may readily solve the system using backward substitution. The computational cost of LU factorization in terms fioating point operations is cubic. There are various efforts to improve the performance of LU factorization. We propose a multi-core multi-GPU hybrid LU factorization algorithm that leverages the strengths of both multiple CPUs and multiple GPUs. Our algorithm uses some of the CPU cores for panel factorization, and the rest of the CPU cores together with all the available GPUs for trailing submatrix updates. Our algorithm employs both dynamic scheduling and static scheduling. Experiments show that our approach reaches 1134 Gflop/s with 4 Fermi GPU boards when combined with the total of 48 CPU cores from AMD. This is the first time such level of performance have been reported in a shared memory environment. Execution trace shows that our code also achieves good load balance and high system utilization.

The MOLDY short-range molecular dynamics package

We describe a parallelised version of the MOLDY molecular dynamics program. This Fortran code is aimed at systems which may be described by short-range potentials and specifically those which may be addressed with the embedded atom method. This includes a wide range of transition metals and alloys. MOLDY provides a range of options in terms of the molecular dynamics ensemble used and the boundary conditions which may be applied. A number of standard potentials are provided, and the modular structure of the code allows new potentials to be added easily. The code is parallelised using OpenMP and can therefore be run on shared memory systems, including modern multicore processors. Particular attention is paid to the updates required in the main force loop, where synchronisation is often required in OpenMP implementations of molecular dynamics. We examine the performance of the parallel code in detail and give some examples of applications to realistic problems, including the dynamic compression of copper and carbon migration in an iron–carbon alloy. Program summary Program title: MOLDY Catalogue identifier: AEJU_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEJU_v1_0.html Program obtainable from: CPC Program Library, Queenʼs University, Belfast, N. Ireland Licensing provisions: GNU General Public License version 2 No. of lines in distributed program, including test data, etc.: 382 881 No. of bytes in distributed program, including test data, etc.: 6 705 242 Distribution format: tar.gz Programming language: Fortran 95/OpenMP Computer: Any Operating system: Any Has the code been vectorised or parallelized?: Yes. OpenMP is required for parallel execution RAM: 100 MB or more Classification: 7.7 Nature of problem: Moldy addresses the problem of many atoms (of order 10 6) interacting via a classical interatomic potential on a timescale of microseconds. It is designed for problems where statistics must be gathered over a number of equivalent runs, such as measuring thermodynamic properities, diffusion, radiation damage, fracture, twinning deformation, nucleation and growth of phase transitions, sputtering etc. In the vast majority of materials, the interactions are non-pairwise, and the code must be able to deal with many-body forces. Solution method: Molecular dynamics involves integrating Newtonʼs equations of motion. MOLDY uses verlet (for good energy conservation) or predictor–corrector (for accurate trajectories) algorithms. It is parallelised using open MP. It also includes a static minimisation routine to find the lowest energy structure. Boundary conditions for surfaces, clusters, grain boundaries, thermostat (Nose), barostat (Parrinello–Rahman), and externally applied strain are provided. The initial configuration can be either a repeated unit cell or have all atoms given explictly. Initial velocities are generated internally, but it is also possible to specify the velocity of a particular atom. A wide range of interatomic force models are implemented, including embedded atom, Morse or Lennard-Jones. Thus the program is especially well suited to calculations of metals. Restrictions: The code is designed for short-ranged potentials, and there is no Ewald sum. Thus for long range interactions where all particles interact with all others, the order- N scaling will fail. Different interatomic potential forms require recompilation of the code. Additional comments: There is a set of associated open-source analysis software for postprocessing and visualisation. This includes local crystal structure recognition and identification of topological defects. Running time: A set of test modules for running time are provided. The code scales as order N. The parallelisation shows near-linear scaling with number of processors in a shared memory environment. A typical run of a few tens of nanometers for a few nanoseconds will run on a timescale of days on a multiprocessor desktop.

A framework for parallel computational physics algorithms on multi-core: SPH in parallel

In this paper, a simulation framework that enables distributed numerical computing in multi-core shared-memory environments is presented. Using multiple threads allows a single memory image to be shared concurrently across cores but potentially introduces race conditions. Race conditions can be avoided by ensuring each core operates on an isolated memory block. This is usually achieved by running a different operating system process on each core, such as multiple MPI processes. However, we show that in many computational physics problems, memory isolation can also be enforced within a single process by leveraging spatial sub-division of the physical domain. A new spatial sub-division algorithm is presented that ensures threads operate on different memory blocks, allowing for in-place updates of state, with no message passing or creation of local variables during time stepping. Additionally, the developed framework controls task distribution dynamically ensuring an events based load balance. Results from fluid mechanics analysis using Smoothed Particle Hydrodynamics (SPH) are presented demonstrating linear performance with number of cores.

Java in the High Performance Computing arena: Research, practice and experience

The rising interest in Java for High Performance Computing (HPC) is based on the appealing features of this language for programming multi-core cluster architectures, particularly the built-in networking and multithreading support, and the continuous increase in Java Virtual Machine (JVM) performance. However, its adoption in this area is being delayed by the lack of analysis of the existing programming options in Java for HPC and thorough and up-to-date evaluations of their performance, as well as the unawareness on current research projects in this field, whose solutions are needed in order to boost the embracement of Java in HPC.This paper analyzes the current state of Java for HPC, both for shared and distributed memory programming, presents related research projects, and finally, evaluates the performance of current Java HPC solutions and research developments on two shared memory environments and two InfiniBand multi-core clusters. The main conclusions are that: (1) the significant interest in Java for HPC has led to the development of numerous projects, although usually quite modest, which may have prevented a higher development of Java in this field; (2) Java can achieve almost similar performance to natively compiled languages, both for sequential and parallel applications, being an alternative for HPC programming; (3) the recent advances in the efficient support of Java communications on shared memory and low-latency networks are bridging the gap between Java and natively compiled applications in HPC. Thus, the good prospects of Java in this area are attracting the attention of both industry and academia, which can take significant advantage of Java adoption in HPC.

OpenMP parallelization of the SCIARA Cellular Automata lava flow model: performance analysis on shared-memory computers

Parallel Computing represents a valid solution for reducing execution times in simulations of complex geological processes, such as lava flows, debris flows and, in general, of fluid-dynamic processes. In these cases, Cellular Automata (CA) models have proved to be effective when the behavior of the system to be modeled can be described in terms of local interactions among its constituent parts. Cellular Automata are parallel computing models, discrete in space and time; space is generally subdivided into cells of uniform size and the overall dynamics of the system emerges as the result of the simultaneous application, at discrete time steps, of proper local rules of evolution to each one of them. Due to their intrinsic parallelism, CA models are attractive since they are suitable to be effectively and naturally implemented on parallel computers achieving also high performance. In the recent past, CA models were efficiently executed on distributed memory architectures, such as Beowulf clusters and many-node Supercomputers, while fewer implementations are found regarding shared-memory computers, such as in multi-core machines. This paper shows performance results of the parallelization of a well-known CA model for simulating lava flows – the SCIARA model – in a shared memory environment, by means of OpenMP, an Application Programming Interface which supports multi-platform shared-memory parallel programming.

Adaptive K-Parallel Best-First Search: A Simple but Efficient Algorithm for Multi-Core Domain-Independent Planning

Motivated by the recent hardware evolution towards multi-core machines, we investigate parallel planning techniques in a shared-memory environment. We consider, more specifically, parallel versions of a best-first search algorithm that run K threads, each expanding the next best node from the open list. We show that the proposed technique has a number of advantages. First, it is (reasonably) simple: we show how the algorithm can be obtained from a sequential version mostly by adding parallel annotations. Second, we conduct an extensive empirical study that shows that this approach is quite effective. It is also dynamic in the sense that the number of nodes expanded in parallel is adapted during the search. Overall we show that the approach is promising for parallel domain-independent, suboptimal planning.

Tarmo: A Framework for Parallelized Bounded Model Checking

This paper investigates approaches to parallelizing Bounded Model Checking (BMC) for shared memory environments as well as for clusters of workstations. We present a generic framework for parallelized BMC named Tarmo. Our framework can be used with any incremental SAT encoding for BMC but for the results in this paper we use only the current state-of-the-art encoding for full PLTL. Using this encoding allows us to check both safety and liveness properties, contrary to an earlier work on distributing BMC that is limited to safety properties only. Despite our focus on BMC after it has been translated to SAT, existing distributed SAT solvers are not well suited for our application. This is because solving a BMC problem is not solving a set of independent SAT instances but rather involves solving multiple related SAT instances, encoded incrementally, where the satisfiability of each instance corresponds to the existence of a counterexample of a specific length. Our framework includes a generic architecture for a shared clause database that allows easy clause sharing between SAT solver threads solving various such instances. We present extensive experimental results obtained with multiple variants of our Tarmo implementation. Our shared memory variants have a significantly better performance than conventional single threaded approaches, which is a result that many users can benefit from as multi-core and multi-processor technology is widely available. Furthermore we demonstrate that our framework can be deployed in a typical cluster of workstations, where several multi-core machines are connected by a network.

Light-Weight Leases for Storage-Centric Coordination

Reaching agreement among processes sharing read/write memory is possible only in the presence of an eventual unique leader. A leader that fails must be recoverable, but on the other hand, a live and well-performing leader should never be decrowned. This paper presents the first leader algorithm in shared memory environments that guarantees an eventual leader following global stabilization time. The construction is built using light-weight lease and renew primitives. The implementation is simple, yet efficient. It is uniform, in the sense that the number of potentially contending processes for leadership is not a priori known.

OpenMP Issues Arising in the Development of Parallel BLAS and LAPACK Libraries

Dense linear algebra libraries need to cope efficiently with a range of input problem sizes and shapes. Inherently this means that parallel implementations have to exploit parallelism wherever it is present. While OpenMP allows relatively fine grain parallelism to be exploited in a shared memory environment it currently lacks features to make it easy to partition computation over multiple array indices or to overlap sequential and parallel computations. The inherent flexible nature of shared memory paradigms such as OpenMP poses other difficulties when it becomes necessary to optimise performance across successive parallel library calls. Notions borrowed from distributed memory paradigms, such as explicit data distributions help address some of these problems, but the focus on data rather than work distribution appears misplaced in an SMP context.

분산 공유메모리 환경의 다단계 버스망에서 트래픽에 적응하는 동적 라우팅 알고리즘

This paper proposes an efficient dynamic routing algorithm for Multistage Bus Networks(MBN`s) in distributed shared memory environment. Our algorithm utilizes extra paths available on MBN and determines routing paths adaptively according to switch traffic in order to distribute traffic among switches. Precisely, a packet is transmitted to the next switch on an extra path having a lighter traffic. As a consequence the proposed algorithm reduces the mean response time and the average number of waiting tasks. The results of simulations, carried out with varying numbers of processors and varying switch sizes, show that the proposed algorithm improves the mean response time by 9% and the average number of waiting tasks by 21.6%, compared to the existing routing algorithms which do not consider extra paths on MBN.