Shared-memory Multiprocessors Research Articles

Performance Evaluation on Parallel Speculation-Based Construction of a Binary Search Tree

Binary search trees (BSTs) are one of the most important data structures in the field of computer science. We may easily write a parallel construction program of a BST by extending the sequential algorithm straightly. However, in such conventional approaches, the order of nodes inserted into a BST is determined dynamically, depending on the occasional state of the parallel thread execution. It results in a BST with a different structure (node position) generated on every execution of the parallel program. On the other hand, we have been developing parallel construction schemes of the BST with the same structure as a BST generated by the sequential algorithm. One is the speculatively parallel construction of a BST. And another is the purely (non-speculatively) parallel construction, but it was derived through the concept of thread-level speculation. This paper evaluates the performances of those construction schemes on several types of shared-memory multiprocessors. For the large enough size of BST, our new parallel programs can construct a BST with always the same structure on a little lower or sometimes higher performance than the program that makes a BST with a different structure on every execution. And in contrast with the general expectation that simply enlarging the size of parallel tasks increases misspeculation and damages the performance, we found that it sometimes enhances the performance of speculatively parallel execution.

Barracuda, an Open Source Framework for Parallelizing Divide and Conquer Algorithm

This paper presents a newly-created Barracuda open-source framework which aims to parallelize Java divide and conquer applications. This framework exploits implicit for-loop parallelism in dividing and merging operations. So, this makes it a mixture of parallel for-loop and task parallelism. It targets shared-memory multiprocessors and hybrid distributed shared-memory architectures. We highlight the effectiveness of the framework and focus on the performance gain and programming effort by using this framework. Barracuda aims at large public actors as well as various application domains. In terms of performance achievement, it is very close to Fork/Join framework while allowing end-users to only focus on refactoring code and experts to have the opportunity to improve it.

Shared Cache Based on Content Addressable Memory in a Multi-Core Architecture

Modern shared-memory multi-core processors typically have shared Level 2 (L2) or Level 3 (L3) caches. Cache bottlenecks and replacement strategies are the main problems of such architectures, where multiple cores try to access the shared cache simultaneously. The main problem in improving memory performance is the shared cache architecture and cache replacement. This paper documents the implementation of a Dual-Port Content Addressable Memory (DPCAM) and a modified Near-Far Access Replacement Algorithm (NFRA), which was previously proposed as a shared L2 cache layer in a multi-core processor. Standard Performance Evaluation Corporation (SPEC) Central Processing Unit (CPU) 2006 benchmark workloads are used to evaluate the benefit of the shared L2 cache layer. Results show improved performance of the multicore processor’s DPCAM and NFRA algorithms, corresponding to a higher number of concurrent accesses to shared memory. The new architecture significantly increases system throughput and records performance improvements of up to 8.7% on various types of SPEC 2006 benchmarks. The miss rate is also improved by about 13%, with some exceptions in the sphinx3 and bzip2 benchmarks. These results could open a new window for solving the long-standing problems with shared cache in multi-core processors.

Fast Method Based on Fuzzy Logic for Gaussian-Impulsive Noise Reduction in CT Medical Images

To remove Gaussian-impulsive mixed noise in CT medical images, a parallel filter based on fuzzy logic is applied. The used methodology is structured in two steps. A method based on a fuzzy metric is applied to remove the impulsive noise at the first step. To reduce Gaussian noise, at the second step, a fuzzy peer group filter is used on the filtered image obtained at the first step. A comparative analysis with state-of-the-art methods is performed on CT medical images using qualitative and quantitative measures evidencing the effectiveness of the proposed algorithm. The parallel method is parallelized on shared memory multiprocessors. After applying parallel computing strategies, the obtained computing times indicate that the introduced filter enables to reduce Gaussian-impulse mixed noise on CT medical images in real-time.

Asynchronous domain decomposition methods for nonlinear PDEs

One- and two-level parallel asynchronous methods for the numerical solution of nonlinear systems of equations, especially those arising from (nonlinear) partial differential equations, are studied. The proposed methods are based on domain decomposition techniques. Local convergence theorems are presented in several cases, with appropriate hypotheses. Computational results on a shared memory multiprocessor machine for various problems exhibiting nonlinearities are reported, illustrating the potential of these asynchronous methods, especially for heterogeneous clusters.

Deterministic Constant-Amortized-RMR Abortable Mutex for CC and DSM

The abortable mutual exclusion problem, proposed by Scott and Scherer in response to the needs in real-time systems and databases, is a variant of mutual exclusion that allows processes to abort from their attempt to acquire the lock. Worst-case constant remote memory reference algorithms for mutual exclusion using hardware instructions such as Fetch&Add or Fetch&Store have long existed for both cache coherent (CC) and distributed shared memory multiprocessors, but no such algorithms are known for abortable mutual exclusion. Even relaxing the worst-case requirement to amortized, algorithms are only known for the CC model. In this article, we improve this state of the art by designing a deterministic algorithm that uses Fetch&Store to achieve amortized O (1) remote memory reference in both the CC and distributed shared memory models. Our algorithm supports Fast Abort (a process aborts within six steps of receiving the abort signal) and has the following additional desirable properties: it supports an arbitrary number of processes of arbitrary names, requires only O (1) space per process, and satisfies a novel fairness condition that we call Airline FCFS . Our algorithm is short with fewer than a dozen lines of code.

Optimizing weighted gene co-expression network analysis with a multi-threaded calculation of the topological overlap matrix.

Biomolecular networks are often assumed to be scale-free hierarchical networks. The weighted gene co-expression network analysis (WGCNA) treats gene co-expression networks as undirected scale-free hierarchical weighted networks. The WGCNA R software package uses an Adjacency Matrix to store a network, next calculates the topological overlap matrix (TOM), and then identifies the modules (sub-networks), where each module is assumed to be associated with a certain biological function. The most time-consuming step of WGCNA is to calculate TOM from the Adjacency Matrix in a single thread. In this paper, the single-threaded algorithm of the TOM has been changed into a multi-threaded algorithm (the parameters are the default values of WGCNA). In the multi-threaded algorithm, Rcpp was used to make R call a C++ function, and then C++ used OpenMP to start multiple threads to calculate TOM from the Adjacency Matrix. On shared-memory MultiProcessor systems, the calculation time decreases as the number of CPU cores increases. The algorithm of this paper can promote the application of WGCNA on large data sets, and help other research fields to identify sub-networks in undirected scale-free hierarchical weighted networks. The source codes and usage are available at https://github.com/do-somethings-haha/multi-threaded_calculate_unsigned_TOM_from_unsigned_or_signed_Adjacency_Matrix_of_WGCNA.

3D nonlinear conjugate gradient inversion for frequency-domain airborne EM based on vector finite element method

The 3D inversion is a challenging task in the research of the frequency-domain airborne electromagnetic (AEM) method due to the massive amounts of data and limited computational efficiency. In this paper, we introduce the 3D FAEM inversion based on the vector finite element method (FEM). Compare with the traditional nodal FEM, the vector FEM can improve the computational efficiency by reducing the number of variables and the scale of linear equations. To tackle the computational demands, in the calculation process we apply the PARDISO solver with shared-memory distributed-memory multi-processors to solve the multi-source forward linear equations and calculate the gradient of the objective function by adjoint equations. Then typical resistivity model with the anomalous body is carried out to verify the validity of the proposed method. Meanwhile, in allusion to the multi-solution problem caused by the inversion problem, we study the influence of typical initial models on inversion results. Finally, we applied our algorithm to invert field data. The inversion results demonstrate that the proposed method can obtain reliable AEM resistivity inverted results.

Multithreading Based Parallel Processing for Image Geometric Coregistration in SAR Interferometry

Within the framework of multi-temporal Synthetic Aperture Radar (SAR) interferometric processing, image coregistration is a fundamental operation that might be extremely time-consuming. This paper explores the possibility of addressing fast and accurate SAR image geometric coregistration, with sub-pixel accuracy and in the presence of a complex 3-D object scene, by exploiting the parallelism offered by shared-memory architectures. An efficient and scalable processor is proposed by designing a parallel algorithm incorporating thread-level parallelism for solving the inherent computationally intensive problem. The adopted functional scheme is first mathematically framed and then investigated in detail in terms of its computational structures. Subsequently, a parallel version of the algorithm is designed, according to a fork-join model, by suitably taking into account the granularity of the decomposition, load-balancing, and different scheduling strategies. The developed parallel algorithm implements parallelism at the thread-level by using OpenMP (Open Multi-Processing) and it is specifically targeted at shared-memory multiprocessors. The parallel performance of the implemented multithreading-based SAR image coregistration prototype processor is experimentally investigated and quantitatively assessed by processing high-resolution X-band COSMO-SkyMed SAR data and using two different multicore architectures. The effectiveness of the developed multithreaded prototype solution in fully benefitting from the computing power offered by multicore processors has successfully been demonstrated via a suitable experimental performance analysis conducted in terms of parallel speedup and efficiency. The demonstrated scalable performance and portability of the developed parallel processor confirm its potential for operational use in the interferometric SAR data processing at large scales.

Static Scheduling with Load Balancing for Solving Triangular Band Linear Systems on Multicore Processors

A new approach for solving triangular band linear systems is established in this study to balance the load and obtain a high degree of parallelism. Our investigation consists to attribute both adequate start time and processor to each task and eliminate the useless dependencies which are not used in the parallel solve stage. Thereby, processors execute in parallel their related tasks taking account of the considered precedence constraints. The theoretical lower bounds for parallel execution time and the number of processors required to carry out the task graph in the shortest time are determined. Experimentations are realized on a shared-memory multicore processor. The experimental results are fitted to the values derived from the determined mathematical formulas. The comparison of results obtained by our contribution with those from triangular systems resolution routine belonging to the library PLASMA, Parallel Linear Algebra Software for Multicore Architectures, confirms the efficiency of the proposed approach.

An Open-Source Platform for High-Performance Non-Coherent On-Chip Communication

On-chip communication infrastructure is a central component of modern systems-on-chip (SoCs), and it continues to gain importance as the number of cores, the heterogeneity of components, and the on-chip and off-chip bandwidth continue to grow. Decades of research on on-chip networks enabled cache-coherent shared-memory multiprocessors. However, communication fabrics that meet the needs of heterogeneous many-cores and accelerator-rich SoCs, which are not, or only partially, coherent, are a much less mature research area. In this work, we present a modular, topology-agnostic, high-performance on-chip communication platform. The platform includes components to build and link subnetworks with customizable bandwidth and concurrency properties and adheres to a state-of-the-art, industry-standard protocol. We discuss microarchitectural trade-offs and timing/area characteristics of our modules and show that they can be composed to build high-bandwidth (e.g., 2.5 GHz and 1024 bit data width) end-to-end on-chip communication fabrics (not only network switches but also DMA engines and memory controllers) with high degrees of concurrency. We design and implement a state-of-the-art ML training accelerator, where our communication fabric scales to 1024 cores on a die, providing 32 TB/s cross-sectional bandwidth at only 24 ns round-trip latency between any two cores.

Optimizing Speedup on Multicore Platform with OpenMP Schedule Clause and Chunk Size

Despite the description of schedule type which can be easily found at any OpenMP reference material, little is known about the effect of different schedule type and chunk size on the parallel performance of shared memory multicore processor. Literature shows that performance analysis on different multicore platform overlooked the effect of different schedule type and chunk size, where often it was not explicitly specified. Hence, default assignment of the loop iterations among threads is assumed. By default, static schedule is used and size of chunk which is the ratio of total number of iteration to the number of threads is implemented. This research analyses the effect of different schedule type and chunk size on speedup achieved of different shared memory multicore platform under regular workload. Apart from that, the performance gain obtained after turning on/off certain multicore technologies and after turning on/off selected number of active cores per processor is also analysed. Results shows that different multicore technology exhibit different speedup value under different combination of schedule type and chunk size. Apart from that, it also observe that different multicore platform is better than the other in terms of speedup as the number of cores are increased.

Parallel Data Distribution Management on Shared-memory Multiprocessors

The problem of identifying intersections between two sets of d -dimensional axis-parallel rectangles appears frequently in the context of agent-based simulation studies. For this reason, the High Level Architecture (HLA) specification—a standard framework for interoperability among simulators—includes a Data Distribution Management (DDM) service whose responsibility is to report all intersections between a set of subscription and update regions. The algorithms at the core of the DDM service are CPU-intensive, and could greatly benefit from the large computing power of modern multi-core processors. In this article, we propose two parallel solutions to the DDM problem that can operate effectively on shared-memory multiprocessors. The first solution is based on a data structure (the interval tree) that allows concurrent computation of intersections between subscription and update regions. The second solution is based on a novel parallel extension of the Sort Based Matching algorithm, whose sequential version is considered among the most efficient solutions to the DDM problem. Extensive experimental evaluation of the proposed algorithms confirm their effectiveness on taking advantage of multiple execution units in a shared-memory architecture.

Legacy code and parallel computing: updating and parallelizing a numerical model

In this paper, we present several important details in the process of legacy code parallelization, mostly related to the problem of maintaining numerical output of a legacy code while obtaining a balanced workload for parallel processing. Since we maintained the non-uniform mesh imposed by the original finite element code, we have to develop a specially designed data distribution among processors so that data restrictions are met in the finite element method. In particular, we introduce a data distribution method that is initially used in shared memory parallel processing and obtain better performance than the previous parallel program version. Besides, this method can be extended to other parallel platforms such as distributed memory parallel computers. We present results including several problems related to performance profiling on different (development and production) parallel platforms. The use of new and old parallel computing architectures leads to different behavior of the same code, which in all cases provides better performance in multiprocessor hardware.

SIGEST

The SIGEST article in this issue, “Randomized Projection for Rank-Revealing Matrix Factorizations and Low-Rank Approximations,” by Jed A. Duersch and Ming Gu, concerns the use of randomization to reduce the bottleneck in a central numerical linear algebra kernel. The authors consider QR factorization, which is a key component in algorithms for constructing stable bases, for solving least-squares problems, and generally for summarizing or extracting insights from data. Column pivoting is the standard technique for dealing with cases where the target matrix is not of full rank, or where the rank is in doubt. However, for large-scale problems column pivoting requires substantial communication between different levels of computer memory, which severely limits the overall efficiency. The authors therefore propose a strategy for choosing pivots based on randomization rather than exhaustive search. This approach reduces communication complexity while increasing uncertainty in the quality of the factorization in a controlled manner. The article makes use of clear pseudocode listings in order to summarize both the original deterministic strategies and the newly developed randomized approaches; see Algorithms 1 to 6. The authors also explain how a Bayesian framework can be set up to quantify the uncertainty introduced by the use of random projections. Extensive, large-scale tests are conducted using Fortran with OpenMP, an application programming interface that supports shared-memory multiprocessing. The original article appeared in the SIAM Journal on Scientific Computing in 2017, and the main idea has subsequently proved to be effective for a number of related tasks. In creating this highlighted article for the SIGEST section, the authors have revised the introductory material, providing additional context; reorganized the document structure and added new visualizations; updated and extended the computational experiments, while making this section more compact and easier to interpret; and discussed recent developments in the field and provided references to the growing literature.

Performance analysis of synchronous and asynchronous distributed genetic algorithms on multiprocessors

Abstract Because of their effectiveness and flexibility in finding useful solutions, Genetic Algorithms (GAs) are very popular search techniques for solving complex optimization problems in scientific and industrial fields. Parallel GAs (PGAs), and especially distributed ones have been usually presented as the way to overcome the time-consuming shortcoming of sequential GAs. In the case of applying PGAs, we can expect better performance, the reason being the exchange of knowledge during the parallel search process. The resulting distributed search is different compared to what sequential panmictic GAs do, then deserving additional studies. This article presents a performance study of three different PGAs. Moreover, we investigate the effect of synchronizing communications over modern shared-memory multiprocessors. We consider the master-slave model along with synchronous and asynchronous distributed GAs (dGAs), presenting their different designs and expected similarities when running in a number of cores ranging from one to 32 cores. The master-slave model showed a competitive numerical effort versus the other dGAs and demonstrated to be able to scale-up well over multiprocessors. We describe how the speed-up and parallel performance of the dGAs is changing as the number of cores enlarges. Results of the island model show that synchronous and asynchronous dGAs have different numerical performances on a multiprocessor, the asynchronous algorithm having a faster execution, thus more attractive for time demanding applications. Our results and statistical analyses help in developing a novel body of knowledge on PGAs running in shared memory multiprocessors (versus overwhelming literature oriented to distributed memory clusters), something useful for researchers, beginners, and final users of these techniques.

Scaling out NUMA-Aware Applications with RDMA-Based Distributed Shared Memory

The multicore evolution has stimulated renewed interests in scaling up applications on shared-memory multiprocessors, significantly improving the scalability of many applications. But the scalability is limited within a single node; therefore programmers still have to redesign applications to scale out over multiple nodes. This paper revisits the design and implementation of distributed shared memory (DSM) as a way to scale out applications optimized for non-uniform memory access (NUMA) architecture over a well-connected cluster. This paper presents MAGI, an efficient DSM system that provides a transparent shared address space with scalable performance on a cluster with fast network interfaces. MAGI is unique in that it presents a NUMA abstraction to fully harness the multicore resources in each node through hierarchical synchronization and memory management. MAGI also exploits the memory access patterns of big-data applications and leverages a set of optimizations for remote direct memory access (RDMA) to reduce the number of page faults and the cost of the coherence protocol. MAGI has been implemented as a user-space library with pthread-compatible interfaces and can run existing multithreaded applications with minimized modifications. We deployed MAGI over an 8-node RDMAenabled cluster. Experimental evaluation shows that MAGI achieves up to 9.25x speedup compared with an unoptimized implementation, leading to a scalable performance for large-scale data-intensive applications.

Timed Petri Net Models of Shared-Memory Bus-Based Multiprocessors

In shared-memory bus-based multiprocessors, when the number of processors grows, the processors spend an increasing amount of time waiting for access to the bus (and shared memory). This contention reduces the performance of processors and imposes a limitation of the number of processors that can be used efficiently in bus-based systems. Since the multi-processor’s performance depends upon many parameters which affect the performance in different ways, timed Petri nets are used to model shared-memory bus-based multiprocessors at the instruction execution level, and the developed models are used to study how the performance of processors changes with the number of processors in the system. The results illustrate very well the restriction on the number of processors imposed by the shared bus. All performance characteristics presented in this paper are obtained by discrete-event simulation of Petri net models.

Decoupled MapReduce for Shared-Memory Multi-Core Architectures

Modern multi-core processors exhibit high integration densities, e.g., up to several tens of cores. Multiple programming frameworks have emerged to facilitate the development of highly parallel applications. The MapReduce programming model, after having demonstrated its usability in the area of distributed computing systems, has been adapted to the needs of shared-memory multi-processors showing promising results in comparison with conventional multi-threaded libraries, e.g., pthreads. In this paper we enhance the traditional MapReduce architecture by decoupling the map and combine phases in order to boost parallel execution. We show that combiners’ memory intensive features limit the system’s degree of parallelism, thus resulting in sub-optimal hardware utilization, leaving space for further performance improvements. The proposed decoupled MapReduce architecture is evaluated into a NUMA server platform, showing that the adoption of the De-MapR runtime enables more efficient hardware utilization and competent run-time improvements. We demonstrate that the proposed solution achieves execution speedups of up to 2.46x compared to a state-of-the-art, shared-memory MapReduce library.

Parallel Processing Transport Model MT3DMS by Using OpenMP.

Solute transport modeling resolves advection, dispersion, and chemical reactions in groundwater systems with its accuracy depending on the resolution of domain at all scales, thus the computational efficiency of a simulator becomes a bottleneck for the wide application of numerical simulations. However, the traditional serial numerical simulators have reached their limits for the prohibitive computational time and memory requirement in solving large-scale problems. These limitations have greatly hindered the wide application of groundwater solute transport modeling. Thus, the development of an efficient method for handling large-scale groundwater solute transport simulation is urgently required. In this study, we developed and assessed a parallelized MT3DMS (Modular Three-Dimensional Multispecies Transport Model) by using OpenMP (Open specifications for Multi-Processing) to accelerate the solute transport simulation process. The parallelization was achieved by adding OpenMP compile directives (i.e., defining various types of parallel regions) into the most time-consuming packages, including the Advection package (ADV), Dispersion package (DSP), and Generalized Conjugate Gradient Solver package (GCG). This allows parallel processing on shared-memory multiprocessors, i.e., both the memory requirement and computing efforts are automatically distributed among all processors. Moreover, we discussed two different parallelization strategies for handling numerical models with either many layers or few layers. The performance of parallelized MT3DMS was assessed by two benchmark numerical models with different model domain sizes via a workstation with two quad-core processors. Results showed that the running time of parallelized MT3DMS can be 4.15 times faster than that using sequential MT3DMS. The effects of using different preconditioners (procedures that transform a given problem into a form that is more suitable for numerical solving methods) in the GCG package were additionally evaluated. The modified strategy for handling numerical models with few layers also achieved satisfactory results with running time two times faster than that via sequential simulation. Thus, the proposed parallelization allows high-resolution groundwater transport simulation with higher efficiency for large-scale or multimillion-cell simulation problems.