General Purpose Parallel Research Articles

A general purpose parallel Fortran code for grid projected concentration reconstruction from multidimensional particle distributions

Particle-based transport models are widely used for simulating the movement of pollutants in environmental systems. Unlike grid-based methods, particles naturally address advective transport without numerical dispersion. However, these models require concentration reconstruction from the discrete particle information. This is computationally demanding in multidimensional problems, posing a challenge for field-scale models requiring frequent reconstruction. Grid Projected Kernel Density Estimation (GPKDE) is a cell-averaged reconstruction with improvements in computational performance compared to classical KDE. Currently, no programs implementing this method are readily integrated into particle simulators, compatible with three-dimensional domains, and particles with unequal weights. This article introduces a Fortran code for general-purpose GPKDE, with modular functionalities facilitating the integration into external software. The program implements locally adaptive kernel bandwidth selection and alternatives for the reconstruction from particles with non-uniform weights. The code is parallelized with the OpenMP library. Numerical test cases demonstrate the program’s applicability and scalability of the parallel implementation.

An easy and direct protocol based on planar molecular images to quantify 131I using thyroid phantom

A planar nuclear medicine image can be used to estimate dosimetry during iodine therapy. To this end, radionuclide activity distribution should be quantified in the patient’s body in terms of a calibration coefficient. This coefficient allows the net counts to correlate with the image’s activity. This study aims propose a simple and easy calibration protocol to quantify 131I activity in thyroid phantom by molecular planar images. Were acquired 13 planar images of different phantoms: thyroid phantom of symmetrical lobes; thyroid phantom of asymmetrical lobes; the Jacszack cylinder phantom with a syringe surrounded by air and water, and finally a plastic bottle containing a syringe with radionuclide. We applied the 131I radionuclide in a General Electric gamma camera, model Discovery NM/CT 670 with a high energy general purpose parallel hole collimator above the phantoms positioned at camera bed. The calibration coefficient of the gamma camera and the standard deviation were determined for each phantom; the average calibration coefficient obtained was 0.062±0.006 MBq/cps. The results suggested that the phantoms applied as too the calibration coefficient obtained by them can provides reasonable value for the gamma camera calibration factor for iodine 131, therefore an accurate evaluation of the scattering media as the source detector distance could impose higher variability and uncertainties on results.

McRPL: a general purpose parallel raster processing library on distributed heterogeneous architectures

Parallel computing on distributed heterogeneous architectures (e.g. computing clusters with multiple CPUs and GPUs) can significantly improve the computational efficiency and scalability of complicated algorithms, but it is theoretically and technically complex. Parallel raster processing libraries reduce the development complexity of parallel raster algorithms by hiding parallel computing details; however, no existing library sufficiently utilizes distributed heterogeneous computing resources. A general-purpose raster processing library (mcRPL) combining multi-process parallelism and multi-thread parallelism is proposed to enable parallel raster processing on distributed heterogeneous architectures with multiple CPUs and GPUs. Additionally, an adaptive hardware assignment strategy is proposed to fully utilize available processors in various hardware environments. A series of task-processing strategies are adopted to aim toward maximizing the utilization of the computing capacity of involved processors. Experiments revealed that two raster algorithms parallelized using mcRPL for spatiotemporal data fusion and land-use change simulation were 170.7- and 143.2-fold faster than original serial algorithms using 8 and 16 GPUs, respectively. While hiding the details of mixed parallelism and reducing the development complexity, mcRPL provides user-friendly interfaces for the development of parallel raster algorithms to enhance computational performance and enable large-scale raster computing tasks with extensive data volumes.

A SCALABLE HYBRID CPU-GPU COMPUTATIONAL FRAMEWORK FOR A FINITE ELEMENT-BASED AIR QUALITY MODEL

We propose a scalable computational framework for the hybrid CPU-GPU implementation ofa traffic-induced and finite element-based air quality model. The hybrid computing paradigm we investigate consists in combining the CPU-based distributed-memory programming approach using Message Passing Interface (MPI) and a GPU programming model for the finite element numerical integration using Compute Unified Device Architecture (CUDA), a general purpose parallel computing platform released by NVIDIA Corporation and featured on its own GPUs. The scalability results obtained from numerical experiments on two major road traffic-induced air pollutants, namely the fine and inhalable particulate matter PM$_{2.5}$ and PM$_{10}$, are illustrated. These achievements, including speedup and efficiency analyses, support that this framework scales well up to 256 CPU cores used concurrently with GPUs from a hybrid computing system.

Fractal Parallel Computing

As machine learning (ML) becomes the prominent technology for many emerging problems, dedicated ML computers are being developed at a variety of scales, from clouds to edge devices. However, the heterogeneous, parallel, and multilayer characteristics of conventional ML computers concentrate the cost of development on the software stack, namely, ML frameworks, compute libraries, and compilers, which limits the productivity of new ML computers. Fractal von Neumann architecture (FvNA) is proposed to address the programming productivity issue for ML computers. FvNA is scale-invariant to program, thus making the development of a family of scaled ML computers as easy as a single node. In this study, we generalize FvNA to the field of general-purpose parallel computing. We model FvNA as an abstract parallel computer, referred to as the fractal parallel machine (FPM), to demonstrate several representative general-purpose tasks that are efficiently programmable. FPM limits the entropy of programming by applying constraints on the control pattern of the parallel computing systems. However, FPM is still general-purpose and cost-optimal. We settle some preliminary results showing that FPM is as powerful as many fundamental parallel computing models such as BSP and alternating Turing machine. Therefore, FvNA is also generally applicable to various fields other than ML.

MiCS-P:Parallel mutual-information computation of big categorical data on spark

Mutual information can effectively measure the correlation between categorical attributes. However, it is found to be quite computationally intensive and time consuming process for enormous size and different distribution data sets. It involves steps for computation of marginal entropies, probability distribution and so on. Spark is a fast, general-purpose parallel framework designed specifically for large-scale data processing. Main motivation of this paper is to provide an intelligent method for parallel mutual information calculation based on Spark computing environment with maintaining the synchronization between different computing nodes. Proposed method named MiCS-P has been able to execute with different number of computing nodes, and gives significant speedup working with different dimensions and sizes of data sets. The MiCS-P algorithm adopts column-wise transformation scheme, which is conducive to the calculation of mutual information between a large number of feature pairs. And to alleviate imbalanced load causing long execution times, we implement a two-phase virtual partitioning scheme running on Spark.

ZipLine: an optimized algorithm for the elastic bulk synchronous parallel model

The bulk synchronous parallel (BSP) is a celebrated synchronization model for general-purpose parallel computing that has successfully been employed for distributed training of deep learning models. A shortcoming of the BSP is that it requires workers to wait for the straggler at every iteration. Therefore, employing BSP increases the waiting time of the faster workers of a cluster and results in an overall prolonged training time. To ameliorate this shortcoming of BSP, we propose ElasticBSP, a model that aims to relax its strict synchronization requirement with an elastic synchronization by allowing delayed synchronization to minimize the waiting time. ElasticBSP offers more flexibility and adaptability during the training phase, without sacrificing the accuracy of the trained model. ElasticBSP is realized by the algorithm named ZipLine, which consists of two phases. First, it estimates for each worker the end time points of its future iterations at run time, and then a one-pass algorithm over the estimated time points of all workers is employed to fast compute an optimal future time point for synchronization. We provide theoretical results about the correctness and performance of the ZipLine algorithm. Furthermore, we propose algorithmic and implementation optimizations of ZipLine, namely ZipLineOpt and ZipLineOptBS, which reduce the time complexity of ZipLine to linearithmic time. A thorough experimental evaluation demonstrates that our proposed ElasticBSP model, materialized by the proposed optimized ZipLine variants, converges faster and to a higher accuracy than the predominant BSP. The focus of the paper is on optimizing the synchronization scheduling over a parameter server architecture. It is orthogonal to other types of optimizations, such as the learning rate optimization.

Efficient parallelization of SPH algorithm on modern multi-core CPUs and massively parallel GPUs

Smoothed Particle Hydrodynamics (SPH) is fast emerging as a practically useful computational simulation tool for a wide variety of engineering problems. SPH is also gaining popularity as the back bone for fast and realistic animations in graphics and video games. The Lagrangian and mesh-free nature of the method facilitates fast and accurate simulation of material deformation, interface capture, etc. Typically, particle-based methods would necessitate particle search and locate algorithms to be implemented efficiently, as continuous creation of neighbor particle lists is a computationally expensive step. Hence, it is advantageous to implement SPH, on modern multi-core platforms with the help of High-Performance Computing (HPC) tools. In this work, the computational performance of an SPH algorithm is assessed on multi-core Central Processing Unit (CPU) as well as massively parallel General Purpose Graphical Processing Units (GP-GPU). Parallelizing SPH faces several challenges such as, scalability of the neighbor search process, force calculations, minimizing thread divergence, achieving coalesced memory access patterns, balancing workload, ensuring optimum use of computational resources, etc. While addressing some of these challenges, detailed analysis of performance metrics such as speedup, global load efficiency, global store efficiency, warp execution efficiency, occupancy, etc. is evaluated. The OpenMP and Compute Unified Device Architecture[Formula: see text] parallel programming models have been used for parallel computing on Intel Xeon[Formula: see text] E5-[Formula: see text] multi-core CPU and NVIDIA Quadro M[Formula: see text] and NVIDIA Tesla p[Formula: see text] massively parallel GPU architectures. Standard benchmark problems from the Computational Fluid Dynamics (CFD) literature are chosen for the validation. The key concern of how to identify a suitable architecture for mesh-less methods which essentially require heavy workload of neighbor search and evaluation of local force fields from neighbor interactions is addressed.

Taskflow: A General-Purpose Parallel and Heterogeneous Task Programming System

Taskflow tackles the long-standing question: How can we make it easier for developers to program parallel and heterogeneous computer-aided design (CAD) applications with high performance and simultaneous high productivity? Taskflow introduces a new powerful task graph programming model to assist developers in the implementation of parallel and heterogeneous algorithms with complex control flow. We develop an efficient system runtime to solve many of the new scheduling challenges arising out of our models and optimize the performance across latency, energy efficiency, and throughput. Taskflow has demonstrated promising performance on both micro-benchmarks and real-world applications. As an example, Taskflow solved a large-scale circuit placement problem up to 17% faster, with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.3\times $ </tex-math></inline-formula> fewer memory, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.1\times $ </tex-math></inline-formula> less power consumption, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.9\times $ </tex-math></inline-formula> higher throughput than two industrial-strength systems, oneTBB and StarPU, on a machine of 40 CPUs and 4 GPUs.

CPU AND GPU PERFORMANCE ANALYSIS ON 2D MATRIX OPERATION

GPU or Graphic Processing Unit can be used on many platforms in general GPUs are used for rendering graphics but now GPUs are general purpose parallel processors with support for easily accessible programming interfaces and industry standard languages such as C, Python and Fortran. In this study, the authors will compare CPU and GPU for completing some matrix calculation. To compare between CPU and GPU, the authors have done some testing to observe the use of Processing Unit, memory and computing time to complete matrix calculations by changing matrix sizes and dimensions. The results of tests that have been done shows asynchronous GPU is faster than sequential. Furthermore, thread for GPU needs to be adjusted to achieve efficiency in GPU load.

A Comparative Study of the Computation Efficiency of a GPU-Based Ray Launching Algorithm for UAV-Assisted Wireless Communications

Graphics Processing Units (GPU), have opened up new opportunities for speeding up general-purpose parallel computing applications. In this paper, we present the computation efficiency in terms of time performances of a novel ray launching field prediction algorithm which relies on NVIDIA GPUs and its Compute Unified Device Architecture (CUDA). The software tool assesses the propagation losses between a wireless transmitter - carried by an Unmanned Air Vehicle (UAV) - over a 3D urban environment. Together with other effective features, the software tool is shown to reduce by several orders of magnitude the computation time of simulations. Performances and cost-benefit analysis of three different NVIDIA GPU configurations are thus investigated over three different urban scenarios, taken as test-cases for Air-to-Ground (A2G) communications for 5G applications and beyond.

Efficient Join Algorithms For Large Database Tables in a Multi-GPU Environment.

Relational join processing is one of the core functionalities in database management systems. It has been demonstrated that GPUs as a general-purpose parallel computing platform is very promising in processing relational joins. However, join algorithms often need to handle very large input data, which is an issue that was not sufficiently addressed in existing work. Besides, as more and more desktop and workstation platforms support multi-GPU environment, the combined computing capability of multiple GPUs can easily achieve that of a computing cluster. It is worth exploring how join processing would benefit from the adaptation of multiple GPUs. We identify the low rate and complex patterns of data transfer among the CPU and GPUs as the main challenges in designing efficient algorithms for large table joins. To overcome such challenges, we propose three distinctive designs of multi-GPU join algorithms, namely, the nested loop, global sort-merge and hybrid joins for large table joins with different join conditions. Extensive experiments running on multiple databases and two different hardware configurations demonstrate high scalability of our algorithms over data size and significant performance boost brought by the use of multiple GPUs. Furthermore, our algorithms achieve much better performance as compared to existing join algorithms, with a speedup up to 25X and 2.8X over best known code developed for multi-core CPUs and GPUs respectively.

Efficient parallel surrogate optimization algorithm and framework with application to parameter calibration of computationally expensive three-dimensional hydrodynamic lake PDE models

Parameter calibration for computationally expensive environmental models (e.g., hydrodynamic models) is challenging because of limits on computing budget and on human time for analysis and because the optimization problem can have multiple local minima and no available derivatives. We present a new general-purpose parallel surrogate global optimization method Parallel Optimization with Dynamic coordinate search using Surrogates (PODS) that reduces the number of model simulations as well as the human time needed for proper calibration of these multimodal problems without derivatives. PODS outperforms state-of-art parallel surrogate algorithms and a heuristic method, Parallel Differential Evolution (P-DE), on all eight well-known test problems. We further apply PODS to the parameter calibration of two expensive (5 h per simulation), three-dimensional hydrodynamic models with the assistant of High-Performance Computing (HPC). Results indicate that PODS outperforms the popularly used P-DE algorithm in speed (about twice faster) and accuracy with 24 parallel processors.

Migrating Engineering Windows HPC applications to Linux HTCondor and Slurm Clusters

The CERN IT department has been maintaining different High Performance Computing (HPC) services over the past five years. While the bulk of computing facilities at CERN are running under Linux, a Windows cluster was dedicated for engineering simulations and analysis related to accelerator technology development. The Windows cluster consisted of machines with powerful CPUs, big memory, and a low-latency interconnect. The Linux cluster resources are accessible through HTCondor, and are used for general purpose parallel but single-node type jobs, providing computing power to the CERN experiments and departments for tasks such as physics event reconstruction, data analysis, and simulation. For HPC workloads that require multi-node parallel environments for Message Passing Interface (MPI) based programs, there is another Linux-based HPC service that is comprised of several clusters running under the Slurm batch system, and consist of powerful hardware with low-latency interconnects.In 2018, it was decided to consolidate compute intensive jobs in Linux to make a better use of the existing resources. Moreover, this was also in line with CERN IT strategy to reduce its dependencies on Microsoft products. This paper focuses on the migration of Ansys [1], COMSOL [2] and CST [3] users from Windows HPC to Linux clusters. Ansys, COMSOL and CST are three engineering applications used at CERN for different domains, like multiphysics simulations and electromagnetic field problems. Users of these applications are in different departments, with different needs and levels of expertise. In most cases, the users have no prior knowledge of Linux. The paper will present the technical strategy to allow the engineering users to submit their simulations to the appropriate Linux cluster, depending on their simulation requirements. We also describe the technical solution to integrate their Windows workstations in order from them to be able to submit to Linux clusters. Finally, we discuss the challenges and lessons learnt during the migration.

Benchmarking a Many-Core Neuromorphic Platform With an MPI-Based DNA Sequence Matching Algorithm

SpiNNaker is a neuromorphic globally asynchronous locally synchronous (GALS) multi-core architecture designed for simulating a spiking neural network (SNN) in real-time. Several studies have shown that neuromorphic platforms allow flexible and efficient simulations of SNN by exploiting the efficient communication infrastructure optimised for transmitting small packets across the many cores of the platform. However, the effectiveness of neuromorphic platforms in executing massively parallel general-purpose algorithms, while promising, is still to be explored. In this paper, we present an implementation of a parallel DNA sequence matching algorithm implemented by using the MPI programming paradigm ported to the SpiNNaker platform. In our implementation, all cores available in the board are configured for executing in parallel an optimised version of the Boyer-Moore (BM) algorithm. Exploiting this application, we benchmarked the SpiNNaker platform in terms of scalability and synchronisation latency. Experimental results indicate that the SpiNNaker parallel architecture allows a linear performance increase with the number of used cores and shows better scalability compared to a general-purpose multi-core computing platform.

JSensor: A Parallel Simulator for Huge Wireless Sensor Networks Applications

This paper presents JSensor, a parallel general purpose simulator which enables huge simulations of Wireless Sensor Networks applications. Its main advantages are: i) to have a simple API with few classes to be extended, allowing easy prototyping and validation of WSNs applications and protocols; ii) to enable transparent and reproducible simulations, regardless of the number of threads of the parallel kernel; and iii) to scale over multi-core computer architectures, allowing simulations of more realistic applications. JSensor is a parallel event-driven simulator which executes according to event timers. The simulation elements, nodes, application, and events, can send messages, process task or move around the simulated environment. The mentioned environment follows a grid structure of extensible spatial cells. The results demonstrated that JSensor scales well, precisely it achieved a speedup of 7.45 with 16 threads in a machine with 16 cores (eight physical and eight virtual cores), and comparative evaluations versus OMNeT++ showed that the presented solution could be 43 times faster.

Parallel general purpose multiobjective optimization framework with application to electron beam dynamics

Particle accelerators are invaluable tools for research in the basic and applied sciences, such as materials science, chemistry, the biosciences, particle physics, nuclear physics and medicine. The design, commissioning, and operation of accelerator facilities is a nontrivial task, due to the large number of control parameters and the complex interplay of several conflicting design goals. The Argonne Wakefield Accelerator facility has some unique challenges resulting from its purpose to carry out advanced accelerator R Individual experiments often have challenging beam requirements, and the physical configuration of the beam lines is often changed to accommodate the variety of supported experiments. The need for rapid deployment of different operational settings further complicates the optimization work that must be done for multiple constraints and challenging operational regimes. One example of this is an independently staged two-beam acceleration experiment which requires the construction of an additional beam line (this is now in progress). The high charge drive beam, well into the space charge regime, must be threaded through small aperture (17.6 mm) decelerating structures. In addition, the bunch length must be sufficiently short to maximize power generation in the decelerator. We propose to tackle this problem by means of multiobjective optimization algorithms which also facilitate a parallel deployment. In order to compute solutions in a meaningful time frame, a fast and scalable software framework is required. In this paper, we present a general-purpose framework for simulation-based multiobjective optimization methods that allows the automatic investigation of optimal sets of machine parameters. Using evolutionary algorithms as the optimizer and opal as the forward solver, validation experiments and results of multiobjective optimization problems in the domain of beam dynamics are presented. Optimized solutions for the new high charge drive beam line found by the framework were used to finish the design of a two beam acceleration experiment. The selected solution along with the associated beam parameters is presented.

Statically Optimal Binary Search Tree Computation Using Non-Serial Polyadic Dynamic Programming on GPU's

Modern GPUs perform computation at a very high rate when compared to CPUs; as a result, they are increasingly used for general purpose parallel computation. Determining if a statically optimal binary search tree is an optimization problem to find the optimal arrangement of nodes in a binary search tree so that average search time is minimized. Knuth's modification to the dynamic programming algorithm improves the time complexity to O(n2). We develop a multiple GPU-based implementation of this algorithm using different approaches. Using suitable GPU implementation for a given workload provides a speedup of up to four times over other GPU based implementations. We are able to achieve a speedup factor of 409 on older GTX 570 and a speedup factor of 745 is achieved on a more modern GTX 1060 when compared to a conventional single threaded CPU based implementation.

New BSP/CGM algorithms for spanning trees

Computing a spanning tree (ST) and a minimum ST (MST) of a graph are fundamental problems in graph theory and arise as a subproblem in many applications. In this article, we propose parallel algorithms to these problems. One of the steps of previous parallel MST algorithms relies on the heavy use of parallel list ranking which, though efficient in theory, is very time-consuming in practice. Using a different approach with a graph decomposition, we devised new parallel algorithms that do not make use of the list ranking procedure. We proved that our algorithms are correct, and for a graph [Formula: see text], [Formula: see text], and [Formula: see text], the algorithms can be executed on a Bulk Synchronous Parallel/Coarse Grained Multicomputer (BSP/CGM) model using [Formula: see text] communications rounds with [Formula: see text] computation time for each round. To show that our algorithms have good performance on real parallel machines, we have implemented them on graphics processing unit. The obtained speedups are competitive and showed that the BSP/CGM model is suitable for designing general purpose parallel algorithms.

CuFFS: A GPU-accelerated code for Fast Faraday rotation measure Synthesis

Rotation measure (RM) synthesis is a widely used polarization processing algorithm for reconstructing polarized structures along the line of sight. Performing RM synthesis on large datasets produced by telescopes like LOFAR can be computationally intensive as the computational cost is proportional to the product of the number of input frequency channels, the number of output Faraday depth values to be evaluated and the number of lines of sight present in the data cube. The required computational cost is likely to get worse due to the planned large area sky surveys with telescopes like the Low Frequency Array (LOFAR), the Murchison Widefield Array (MWA), and eventually the Square Kilometre Array (SKA). The massively parallel General Purpose Graphical Processing Units (GPGPUs) can be used to execute some of the computationally intensive astronomical image processing algorithms including RM synthesis. In this paper, we present a GPU-accelerated code, called cuFFS or CUDA-accelerated Fast Faraday Synthesis, to perform Faraday rotation measure synthesis. Compared to a fast single-threaded and vectorized CPU implementation, depending on the structure and format of the data cubes, our code achieves an increase in speed of up to two orders of magnitude. During testing, we noticed that the disk I/O when using the Flexible Image Transport System (FITS) data format is a major bottleneck and to reduce the time spent on disk I/O, our code supports the faster HDFITS format in addition to the standard FITS format. The code is written in C with GPU-acceleration achieved using Nvidia's CUDA parallel computing platform. The code is available at https://github.com/sarrvesh/cuFFS.