Shared Memory Machines Research Articles

Parallelization of particle-mass-transfer algorithms on shared-memory, multi-core CPUs

Simulating the transfer of mass between particles is not straightforwardly parallelized because it involves the calculation of the influence of many particles on each other. Engdahl et al. (2019) intuited that the number of matrix operations used for mass transfer grows quadratically with the number of particles, so that dividing the domain geometrically into sub-domains will give speed and memory advantages, even on a single processing thread. Those authors also showed the speed scalability of several one-dimensional examples on multiple cores. Here, we extend those results for more general cases, both in terms of spatial dimensions and algorithmic implementation. We show that there is an optimal subdivision scheme for naive, full-matrix calculations on a multi-processor, or multi-threading shared-memory machine. A similar sparse-matrix implementation that also uses row-and-column-sum normalization often greatly reduces the memory requirements. We also introduce a completely new mass transfer algorithm that uses a non-geometric domain decomposition and only matrix row-sum normalization. This allows the mass-transfer “matrix” to be constructed and solved one row at a time in parallel, so it is faster and vastly more memory efficient than previous methods, but requires more care for suitable accuracy.

CXL and the Return of Scale-Up Database Engines

The trend toward specialized processing devices such as TPUs, DPUs, GPUs, and FPGAs has exposed the weaknesses of PCIe in interconnecting these devices and their hosts. Several attempts have been proposed to improve, augment, or downright replace PCIe, and more recently, these efforts have converged into a standard called Compute Express Link (CXL). CXL is already on version 2.0 in terms of commercial availability, but its potential to radically change the conventional server architecture has only just started to surface. For example, CXL can increase the bandwidth and quantity of memory available to any single machine beyond what that machine can originally provide, most importantly, in a manner that is fully transparent to software applications. We argue, however, that CXL can have a broader impact beyond memory expansion and deeply affect the architecture of data-intensive systems. In a nutshell, while the cloud favored scale-out approaches that grew in capacity by adding full servers to a rack, CXL brings back scale-up architectures that can grow by fine-tuning individual resources, all while transforming the rack into a large shared-memory machine. In this paper, we describe why such architectural transformations are now possible, how they benefit emerging heterogeneous hardware platforms for data-intensive systems, and the associated research challenges.

A general approach for supporting nonblocking data structures on distributed-memory systems

Nonblocking data structures are an essential part of many parallel applications in that they can help to improve fault tolerance and performance. Although there are scores of nonblocking data structures, such as stacks, queues, double-ended queues (deques), lists, widely used in practice, most of them are designed to be used on shared-memory machines only, and cannot be used in a distributed-memory setting. Several recent studies focus on the development of novel tailor-made nonblocking distributed data structures and omit the potential for adapting a great wealth of existing nonblocking shared-memory ones for the distributed-memory case. Hence, we propose a general approach for bridging the gap between most existing nonblocking data structures and distributed-memory machines in this work. Several challenges, such as safe memory reclamation and solving the ABA problem, must be overcome. To address these issues, we present a global memory management scheme. The scheme takes advantage of hazard pointers which are widely used to tackle the problems in shared-memory environments. To demonstrate our general approach, we take stacks as a typical example of nonblocking data structures. This work also provides a survey of well-known nonblocking stack algorithms along with our analysis and evaluation in distributed-memory environments. Moreover, this paper depicts how to improve performance of a stack algorithm by making use of node locality. Besides, a cost model based on worst cases is devised to help gain a better understanding into experimental results of nonblocking distributed data structures, along with our analysis of the influence of two popular lock-free programming patterns on performance.

Parallel Weighted Random Sampling

Data structures for efficient sampling from a set of weighted items are an important building block of many applications. However, few parallel solutions are known. We close many of these gaps. We give efficient, fast, and practicable parallel and distributed algorithms for building data structures that support sampling single items (alias tables, compressed data structures). This also yields a simplified and more space-efficient sequential algorithm for alias table construction. Our approaches to sampling k out of n items with/without replacement and to subset (Poisson) sampling are output-sensitive , i.e., the sampling algorithms use work linear in the number of different samples. This is also interesting in the sequential case. Weighted random permutation can be done by sorting appropriate random deviates. We show that this is possible with linear work. Finally, we give a communication-efficient, highly scalable approach to (weighted and unweighted) reservoir sampling. This algorithm is based on a fully distributed model of streaming algorithms that might be of independent interest. Experiments for alias tables and sampling with replacement show near linear speedups using up to 158 threads of shared-memory machines. An experimental evaluation of distributed weighted reservoir sampling on up to 5,120 cores also shows good speedups.

Computer big data modeling system based on finite element mathematical equation simulation

Abstract In the traditional computer analysis system, the equipment is cumbersome and takes up space, and is gradually being eliminated by the market. In view of the characteristics of small size, large throughput and high precision of computer systems in the era of big data, in order to solve the phenomenon of swell disasters induced by high-speed moving landslides, debris flows and glaciers impacting water bodies in enclosed waters, a mathematical equation based on finite element was developed. The sparse linear multi-thread parallel solving system of the group, using the sparse linear equation solver implemented on the shared memory machine provided by Intel MKL, completed the design of the high-performance finite element analysis program for surge. The theoretical solution and numerical calculation results of vertical dam failure and laboratory experimental results show that the normalized error between the theoretical value calculated by the calculation system designed in this paper and the actual value obtained from the experiment is only 5.2%~6.8% In between, it is verified that the system has rapid reliability and practical engineering significance.

Fast and compact matching statistics analytics.

Fast, lightweight methods for comparing the sequence of ever larger assembled genomes from ever growing databases are increasingly needed in the era of accurate long reads and pan-genome initiatives. Matching statistics is a popular method for computing whole-genome phylogenies and for detecting structural rearrangements between two genomes, since it is amenable to fast implementations that require a minimal setup of data structures. However, current implementations use a single core, take too much memory to represent the result, and do not provide efficient ways to analyze the output in order to explore local similarities between the sequences. We develop practical tools for computing matching statistics between large-scale strings, and for analyzing its values, faster and using less memory than the state-of-the-art. Specifically, we design a parallel algorithm for shared-memory machines that computes matching statistics 30 times faster with 48 cores in the cases that are most difficult to parallelize. We design a lossy compression scheme that shrinks the matching statistics array to a bitvector that takes from 0.8 to 0.2 bits per character, depending on the dataset and on the value of a threshold, and that achieves 0.04 bits per character in some variants. And we provide efficient implementations of range-maximum and range-sum queries that take a few tens of milliseconds while operating on our compact representations, and that allow computing key local statistics about the similarity between two strings. Our toolkit makes construction, storage and analysis of matching statistics arrays practical for multiple pairs of the largest genomes available today, possibly enabling new applications in comparative genomics. Our C/C++ code is available at https://github.com/odenas/indexed_ms under GPL-3.0. The data underlying this article are available in NCBI Genome at https://www.ncbi.nlm.nih.gov/genome and in the International Genome Sample Resource (IGSR) at https://www.internationalgenome.org. Supplementary data are available at Bioinformatics online.

An Algorithm for the Sequence Alignment with Gap Penalty Problem using Multiway Divide-and-Conquer and Matrix Transposition

We present a cache-efficient parallel algorithm for the sequence alignment with gap penalty problem for shared-memory machines using multiway divide-and-conquer and not-in-place matrix transposition. Our r−way divide-and-conquer algorithm, for a fixed natural number r≥2, performs Θ(n3) work, achieves Θ(nlogr⁡(2r−1)) span, and incurs O(n3/(BM)+(n2/B)log⁡M) serial cache misses for n>γM, and incurs O((n2/B)log⁡(n/M)) serial cache misses for αM<n≤γM, where, M is the cache size, B is the cache line size, and α and γ are constants.

Cuttlefish: fast, parallel and low-memory compaction of de Bruijn graphs from large-scale genome collections.

MotivationThe construction of the compacted de Bruijn graph from collections of reference genomes is a task of increasing interest in genomic analyses. These graphs are increasingly used as sequence indices for short- and long-read alignment. Also, as we sequence and assemble a greater diversity of genomes, the colored compacted de Bruijn graph is being used more and more as the basis for efficient methods to perform comparative genomic analyses on these genomes. Therefore, time- and memory-efficient construction of the graph from reference sequences is an important problem.ResultsWe introduce a new algorithm, implemented in the tool Cuttlefish, to construct the (colored) compacted de Bruijn graph from a collection of one or more genome references. Cuttlefish introduces a novel approach of modeling de Bruijn graph vertices as finite-state automata, and constrains these automata’s state-space to enable tracking their transitioning states with very low memory usage. Cuttlefish is also fast and highly parallelizable. Experimental results demonstrate that it scales much better than existing approaches, especially as the number and the scale of the input references grow. On a typical shared-memory machine, Cuttlefish constructed the graph for 100 human genomes in under 9 h, using 29 GB of memory. On 11 diverse conifer plant genomes, the compacted graph was constructed by Cuttlefish in under 9 h, using 84 GB of memory. The only other tool completing these tasks on the hardware took over 23 h using 126 GB of memory, and over 16 h using 289 GB of memory, respectively.Availability and implementationCuttlefish is implemented in C++14, and is available under an open source license at https://github.com/COMBINE-lab/cuttlefish.Supplementary information Supplementary data are available at Bioinformatics online.

PBBFMM3D: A parallel black-box algorithm for kernel matrix-vector multiplication

Kernel matrix-vector product is ubiquitous in many science and engineering applications. However, a naive method requires O(N2) operations, which becomes prohibitive for large-scale problems. To reduce the computation cost, we introduce a parallel method that provably requires O(N) operations and delivers an approximate result within a prescribed tolerance. The distinct feature of our method is that it requires only the ability to evaluate the kernel function, offering a black-box interface to users. Our parallel approach targets multi-core shared-memory machines and is implemented using OpenMP. Numerical results demonstrate up to 19× speedup on 32 cores. We also present a real-world application in geo-statistics, where our parallel method was used to deliver fast principle component analysis of covariance matrices.

RCHOL: Randomized Cholesky Factorization for Solving SDD Linear Systems

We introduce a randomized algorithm, namely RCHOL, to construct an approximate Cholesky factorization for a given Laplacian matrix (a.k.a., graph Laplacian). From a graph perspective, the exact Cholesky factorization introduces a clique in the underlying graph after eliminating a row/column. By randomization, RCHOL only retains a sparse subset of the edges in the clique using a random sampling developed by Spielman and Kyng. We prove RCHOL is breakdown-free and apply it to solving large sparse linear systems with symmetric diagonally dominant matrices. In addition, we parallelize RCHOL based on the nested dissection ordering for shared-memory machines. We report numerical experiments that demonstrate the robustness and the scalability of RCHOL. For example, our parallel code scaled up to 64 threads on a single node for solving the 3D Poisson equation, discretized with the 7-point stencil on a $1024\times 1024 \times 1024$ grid, a problem that has one billion unknowns.

Parallelization of the inverse fast multipole method with an application to boundary element method

We present an algorithm to parallelize the inverse fast multipole method (IFMM), which is an approximate direct solver for dense linear systems. The parallel scheme is based on a greedy coloring algorithm, where two nodes in the hierarchy with the same color are separated by at least σ nodes. We proved that when σ≥6, the workload associated with one color is embarrassingly parallel. However, the number of nodes in a group (color) may be small when σ=6. Therefore, we also explored σ=3, where a small fraction of the algorithm needs to be serialized, and the overall parallel efficiency was improved. We implemented the parallel IFMM using OpenMP for shared-memory machines. Successively, we applied it to a fast-multipole accelerated boundary element method (FMBEM) as a preconditioner, and compared its efficiency with (a) the original IFMM parallelized by linking a multi-threaded linear algebra library and (b) the commonly used parallel block-diagonal preconditioner. Our results showed that our parallel IFMM achieved at most 4× and 11× speedups over the reference method (a) and (b), respectively, in realistic examples involving more than one million variables.

Faster and Better Nested Dissection Orders for Customizable Contraction Hierarchies

Graph partitioning has many applications. We consider the acceleration of shortest path queries in road networks using Customizable Contraction Hierarchies (CCH). It is based on computing a nested dissection order by recursively dividing the road network into parts. Recently, with FlowCutter and Inertial Flow, two flow-based graph bipartitioning algorithms have been proposed for road networks. While FlowCutter achieves high-quality results and thus fast query times, it is rather slow. Inertial Flow is particularly fast due to the use of geographical information while still achieving decent query times. We combine the techniques of both algorithms to achieve more than six times faster preprocessing times than FlowCutter and even faster queries on the Europe road network. We show that, using 16 cores of a shared-memory machine, this preprocessing needs four minutes.

Computations of permeability of large rock images by dual grid domain decomposition

Digital rock physics (DRP) is an eminent technology that facilitates and repeatable core analysis and multi-physics simulation of rock properties. One of the challenges in this field is the scalability of the problem size, whereby large micro-CT images over the order of 10003 voxels incur a high computational demand on performance. We estimate the of permeability in large digital samples of rocks imaged by micro-CT by using a fast and efficient Dual Grid Domain Decomposition technique based on the Schwarz Alternating Method (slow, low memory) with Algebraic Multigrid (AMG) solvers (fast, high memory) to solve on an otherwise unfeasible shared-memory machine. The comparisons and differences to other methods commonly used have been added in the introduction. The method applies a scalable parallel simulation algorithm to solve pressure and velocity fields using the Semi Analytical Pore Scale Finite Volume Solver (PFVS) within real 3D pore-scale micro-CT images. The domain is split into non-overlapping coarse partitions and also split into a set of dual coarse partitions of varying width. The governing equations are then solved iteratively between the partition sets by updating the pressure and flux at the relevant boundaries before each step. The method is validated and shown to converge to flux continuous conditions requisite of the governing equations. Permeability estimation is within 5–10% of the fine scale solution and significantly reduces memory limitations and computational time for solving problems in micro-CT images, allowing ordinary workstations to solve images over 10003 within the magnitude of 1–100 h of CPU time. The permeability of a 2520 × 2520 × 7250 sample was calculated with a workstation to within 9% error of LBM calculated with a supercomputer within a similar timeframe.

Parallel computation of Watershed Transform in weighted graphs on shared memory machines

Watershed Transform is a widely used image segmentation technique that is known to be very data intensive and time consuming. The M-border Kernel Algorithm computes watersheds in the framework of Edge-Weighted Graphs and allows to preserve the topology of the initial map. Parallelization represents an effective solution to accelerate it. However, this task remains challenging due to the nature of this technique. In this paper, we address this problem. We start by analyzing the data dependency issues that this algorithm raises when dealing with parallel execution. With respect to that, we propose a parallelization strategy that opts for vertex scanning instead of edges scanning of the graph while preserving the thinning paradigm on which the M-border Kernel Algorithm is based. We show that this strategy overcomes the problem of the simultaneous lowering of two adjacent M-border edges that may occur when edge scan is used. The implementation of the proposed algorithm on a shared memory multicore architecture proves its effectiveness in terms of speedup. In fact, the experimental results show that a speedup factor of 5.55 is achieved using eight processors for $$2048 \times 2048$$ images over the performance of the sequential algorithm using a single processor on the same architecture. Furthermore, the gain in terms of execution time and thus speedup is guaranteed whatever is the size of images on which the algorithm is applied. In fact, a speedup factor of 5.55 is obtained for $$2048 \times 2048$$ images, 5.11 for $$1024 \times 1024$$ images and 4.45 for $$512 \times 512$$ images using eight cores.

Slurm: Fluid particle-in-cell code for plasma modeling

With the approach of exascale computing era, particle-based models are becoming the focus of research due to their excellent scalability. We present a new code, Slurm, which implements the classic particle-in-cell algorithm for modeling magnetized fluids and plasmas. It features particle volume evolution which damps the numerical finite grid instability, and allows modeling of key physical instabilities such as Kelvin–Helmholtz and Rayleigh–Taylor. The magnetic field in Slurm is handled via the electromagnetic vector potential carried by particles. Numerical diffusion of the magnetic flux is extremely low, and the solenoidality of the magnetic field is preserved to machine precision. A double-linked list is used to carry particles, thus implementation of open boundary conditions is simple and efficient. The code is written in C++ with OpenMP multi-threading, and has no external dependencies except for Boost. It is easy to install and use on multi-core desktop computers as well as on large shared-memory machines. Slurm is an ideal tool for its primary goal, modeling of space weather events in the heliosphere. This article walks the reader through the physical model, the algorithm, and all important details of implementation. Ideally, after finishing this paper, the reader should be able to either use Slurm for solving the desired problem, or create a new fluid PIC code.

A two-scale generalized finite element method for parallel simulations of spot welds in large structures

The diameter of spot welds used in the automotive and aerospace industry is orders of magnitude smaller than the dimensions of the structural components. Automotive bodies typically have between three and five thousand spot welds which makes 3-D Direct Finite Element Analysis (DFEA) of this class problem not practical. To circumvent these limitations of the FEM, spot welds are idealized in industrial problems as elastic or rigid beams connecting nodes of the metal sheet meshes. However, parameters used in these spot weld models are often problem dependent. This paper presents a parallel Generalized Finite Element Method (GFEM) for the simulation of spot welds in large structures. The proposed GFEM can adopt structural-scale meshes that ignore the spot welds and thus can be generated much quicker than in a DFEA. The element size in the GFEM mesh is of the same order as those used in the industry for the class of problems considered here. Verification problems show that the proposed method can provide an accuracy comparable to a 3-D DFEA. Numerical experiments on a hat-stiffened panel with 168 spot welds show that the proposed GFEM scales much better than a DFEA on shared memory machines, in particular when nonlinear material behavior near the spot welds is considered.

SCALO

Shared memory machines continue to increase in scale by adding more parallelism through additional cores and complex memory hierarchies. Often, executing multiple applications concurrently, dividing among them hardware threads, provides greater efficiency rather than executing a single application with large thread counts. However, contention for shared resources can limit the improvement of concurrent application execution: orchestrating the number of threads used by each application and is essential. In this article, we contribute SCALO, a solution to orchestrate concurrent application execution to increase throughput. SCALO monitors co-executing applications at runtime to evaluate their scalability. Its optimizing thread allocator analyzes these scalability estimates to adapt the parallelism of each program. Unlike previous approaches, SCALO differs by including dynamic contention effects on scalability and by controlling the parallelism during the execution of parallel regions. Thus, it improves throughput when other state-of-the-art approaches fail and outperforms them by up to 40% when they succeed.

Mplrs: A scalable parallel vertex/facet enumeration code

We describe a new parallel implementation, mplrs, of the vertex enumeration code lrs that uses the MPI parallel environment and can be run on a network of computers. The implementation makes use of a C wrapper that essentially uses the existing lrs code with only minor modifications. mplrs was derived from the earlier parallel implementation plrs, written by G. Roumanis in C $${++}$$ which runs on a shared memory machine. By improving load balancing we are able to greatly improve performance for medium to large scale parallelization of lrs. We report computational results comparing parallel and sequential codes for vertex/facet enumeration problems for convex polyhedra. The problems chosen span the range from simple to highly degenerate polytopes. For most problems tested, the results clearly show the advantage of using the parallel implementation mplrs of the reverse search based code lrs, even when as few as 8 cores are available. For some problems almost linear speedup was observed up to 1200 cores, the largest number of cores tested. The software that was reviewed as part of this submission is included in lrslib-062.tar.gz which has MD5 hash be5da7b3b90cc2be628dcade90c5d1b9.

Performance and energy metrics for multi-threaded applications on DVFS processors

Due to their internal execution characteristics, application programs exploit the hardware very differently, which leads to a quite diverse behavior concerning their performance or the energy consumed for their execution. A change of the operational frequency of DVFS processors leads to further variations in performance and energy consumption, as does the exploitation of thread parallelism on multicores. This article combines frequency scaling and thread-parallelism and considers several new metrics for the evaluation of an application's performance and energy consumption. As application programs, the PARSEC benchmark suite and the SPLASH-2 benchmark suite are investigated. The PARSEC benchmark suite provides an up-to-date collection of applications with different workloads on chip-multiprocessors. The SPLASH-2 is a common suite for scientific studies on parallel shared memory machines. Intel Core i7 processors are used as hardware platforms for the evaluation.

Scalable training of 3D convolutional networks on multi- and many-cores

Convolutional networks (ConvNets) have become a popular approach to computer vision. Here we consider the parallelization of ConvNet training, which is computationally costly. Our novel parallel algorithm is based on decomposition into a set of tasks, most of which are convolutions or FFTs. Theoretical analysis suggests that linear speedup with the number of processors is attainable. To attain such performance on real shared-memory machines, our algorithm computes convolutions converging on the same node of the network with temporal locality to reduce cache misses, and sums the convergent convolution outputs via an almost wait-free concurrent method to reduce time spent in critical sections. Benchmarking with multi-core CPUs shows speedup roughly equal to the number of physical cores. We also demonstrate 90x speedup on a many-core CPU (Xeon Phi Knights Corner). Our algorithm can be either faster or slower than certain GPU implementations depending on specifics of the network architecture, kernel sizes, and density and size of the output patch.