High-level Parallel Research Articles

High-throughput Ant Colony Optimization on graphics processing units

Nowadays, computer researchers can face ever-complex scientific problems by using a hardware and software co-design. One successful approach is exploring novel massively-parallel Natural-inspired algorithms, such as the Ant Colony Optimization (ACO) algorithm, through the exploitation of high-throughput accelerators such as GPUs, which are designed to provide high levels of parallelism and low Energy per instruction (EP) cost through heavy vectorization. In this paper, we demonstrate how to take advantage of contemporary hardware-based CUDA vectorization to optimize the ACO algorithm when applied to the Traveling Salesman Problem (TSP). Several parallel designs are proposed and analyzed on two different CUDA architectures. Our results reveal that our vectorization approaches can obtain good performance on these architectures. Moreover, atomic operations are under study showing good benefits on latest generations of CUDA architectures. This work lays the groundwork for future developments of ACO algorithm on high-performance platforms.

Solving the inverse heat conduction problem using NVLink capable Power architecture

The accurate knowledge of Heat Transfer Coefficients is essential for the design of precise heat transfer operations. The determination of these values requires Inverse Heat Transfer Calculations, which are usually based on heuristic optimisation techniques, like Genetic Algorithms or Particle Swarm Optimisation. The main bottleneck of these heuristics is the high computational demand of the cost function calculation, which is usually based on heat transfer simulations producing the thermal history of the workpiece at given locations. This Direct Heat Transfer Calculation is a well parallelisable process, making it feasible to implement an efficient GPU kernel for this purpose. This paper presents a novel step forward: based on the special requirements of the heuristics solving the inverse problem (executing hundreds of simulations in a parallel fashion at the end of each iteration), it is possible to gain a higher level of parallelism using multiple graphics accelerators. The results show that this implementation (running on 4 GPUs) is about 120 times faster than a traditional CPU implementation using 20 cores. The latest developments of the GPU-based High Power Computations area were also analysed, like the new NVLink connection between the host and the devices, which tries to solve the long time existing data transfer handicap of GPU programming.

Bringing Parallel Patterns Out of the Corner

High-level parallel programming is an active research topic aimed at promoting parallel programming methodologies that provide the programmer with high-level abstractions to develop complex parallel software with reduced time to solution. Pattern-based parallel programming is based on a set of composable and customizable parallel patterns used as basic building blocks in parallel applications. In recent years, a considerable effort has been made in empowering this programming model with features able to overcome shortcomings of early approaches concerning flexibility and performance. In this article, we demonstrate that the approach is flexible and efficient enough by applying it on 12 out of 13 PARSEC applications. Our analysis, conducted on three different multicore architectures, demonstrates that pattern-based parallel programming has reached a good level of maturity, providing comparable results in terms of performance with respect to both other parallel programming methodologies based on pragma-based annotations (i.e., Open mp and O mp S s ) and native implementations (i.e., P threads ). Regarding the programming effort, we also demonstrate a considerable reduction in lines of code and code churn compared to P threads and comparable results with respect to other existing implementations.

Verification of Electromagnetic Physics Models for Parallel Computing Architectures in the GeantV Project

An intensive R&D and programming effort is required to accomplish new challenges posed by future experimental high-energy particle physics (HEP) programs. The GeantV project aims to narrow the gap between the performance of the existing HEP detector simulation software and the ideal performance achievable, exploiting latest advances in computing technology. The project has developed a particle detector simulation prototype capable of transporting in parallel particles in complex geometries exploiting instruction level microparallelism (SIMD and SIMT), task-level parallelism (multithreading) and high-level parallelism (MPI), leveraging both the multi-core and the many-core opportunities. We present preliminary verification results concerning the electromagnetic (EM) physics models developed for parallel computing architectures within the GeantV project. In order to exploit the potential of vectorization and accelerators and to make the physics model effectively parallelizable, advanced sampling techniques have been implemented and tested. In this paper we introduce a set of automated statistical tests in order to verify the vectorized models by checking their consistency with the corresponding Geant4 models and to validate them against experimental data.

A technique to automatically determine Ad-hoc communication patterns at runtime

Abstract Current High Performance Computing (HPC) systems are typically built as interconnected clusters of shared-memory multicore computers. Several techniques to automatically generate parallel programs from high-level parallel languages or sequential codes have been proposed. To properly exploit the scalability of HPC clusters, these techniques should take into account the combination of data communication across distributed memory, and the exploitation of shared-memory models. In this paper, we present a new communication calculation technique to be applied across different SPMD (Single Program Multiple Data) code blocks, containing several uniform data access expressions. We have implemented this technique in Trasgo, a programming model and compilation framework that transforms parallel programs from a high-level parallel specification that deals with parallelism in a unified, abstract, and portable way. The proposed technique computes at runtime exact coarse-grained communications for distributed message-passing processes. Applying this technique at runtime has the advantage of being independent of compile-time decisions, such as the tile size chosen for each process. Our approach allows the automatic generation of pre-compiled multi-level parallel routines, libraries, or programs that can adapt their communication, synchronization, and optimization structures to the target system, even when computing nodes have different capabilities. Our experimental results show that, despite our runtime calculation, our approach can automatically produce efficient programs compared with MPI reference codes, and with codes generated with auto-parallelizing compilers.

Efficient calculation of degenerate atomic rates by numerical quadrature on GPUs

The rates of atomic processes in cold, dense plasma are governed strongly by effects of quantum degeneracy. The electrons follow Fermi–Dirac statistics and their high density limits the number of quantum states available for occupation after a collision. These factors preclude a direct solution to the usual rate coefficient integrals. We summarize the formulation of this problem and present a simple, but efficient method of evaluating collisional rate coefficients via direct numerical integration. Numerical quadrature has an intrinsically high level of parallelism, ideally suited for graphics processor units. GPUs are particularly suited to this problem because of the large number of integrals which must be carried out simultaneously for a given atomic model. A CUDA code to calculate the rates of significant atomic processes as part of a collisional-radiative model is presented and discussed. This approach may be readily extended to other applications where rapid and repeated evaluation of many integrals is required.

Solving combinatorial problems using a parallel framework

This paper presents a new IBobpp framework which is an improvement of a high level parallel programming framework called Bobpp to optimize the performance of solving combinatorial problems. The Bobpp parallel computation model is as the majority of parallel models proposed in the context of tree search algorithms with node oriented parallelization, meaning that at every step of the algorithm, each thread gets one node from a unique global pool of non-explored nodes, generates the child nodes, and reinserts them into the pool to be explored later. This classical model has two drawbacks. First, the use of many threads creates a bottleneck problem. Second, grabbing a node causes memory contention problem when many nodes are generated and inserted into the same pool. To solve these problems, IBobpp framework proposes three solutions. The first consists of using multiple pools of nodes shared between all threads. The second solution consists of using a new computation model. The third solution consists of hybridization of the two previous solutions. Preliminary result shows that IBobpp gives a good result using the third solution.

Coarray-based load balancing on heterogeneous and many-core architectures

Abstract In order to reach challenging performance goals, computer architecture is expected to change significantly in the near future. Heterogeneous chips, equipped with different types of cores and memory, will force application developers to deal with irregular communication patterns, high levels of parallelism, and unexpected behavior. Load balancing among the heterogeneous compute units will be a critical task in order to achieve an effective usage of the computational power provided by such new architectures. In this highly dynamic scenario, Partitioned Global Address Space (PGAS) languages, like Coarray Fortran, appear a promising alternative to standard MPI programming that uses two-sided communications, in particular because of PGAS one-sided semantic and ease of programmability. In this paper, we show how Coarray Fortran can be used for implementing dynamic load balancing algorithms on an exascale compute node and how these algorithms can produce performance benefits for an Asian option pricing problem, running in symmetric mode on Intel Xeon Phi Knights Corner and Knights Landing architectures.

Computing with networks of nonlinear mechanical oscillators.

As it is getting increasingly difficult to achieve gains in the density and power efficiency of microelectronic computing devices because of lithographic techniques reaching fundamental physical limits, new approaches are required to maximize the benefits of distributed sensors, micro-robots or smart materials. Biologically-inspired devices, such as artificial neural networks, can process information with a high level of parallelism to efficiently solve difficult problems, even when implemented using conventional microelectronic technologies. We describe a mechanical device, which operates in a manner similar to artificial neural networks, to solve efficiently two difficult benchmark problems (computing the parity of a bit stream, and classifying spoken words). The device consists in a network of masses coupled by linear springs and attached to a substrate by non-linear springs, thus forming a network of anharmonic oscillators. As the masses can directly couple to forces applied on the device, this approach combines sensing and computing functions in a single power-efficient device with compact dimensions.

Formalised Composition and Interaction for Heterogeneous Structured Parallelism

Deployed through skeleton frameworks, structured parallelism yields a clear and consistent structure across platforms by distinctly decoupling computations from the structure in a parallel programme. Structured programming is a viable and effective means of providing the separation of concerns, as it subdivides a system into building blocks (modules, skids or components) that can be independently created, and then used in different systems to drive multiple functionalities. Depending on its defined semantic, each building block wraps a unit of computing function, where the valid assembly of these building blocks forms a high-level structural parallel programming model. This paper proposes a grammar to build block components to execute computational functions in heterogeneous multi-core architectures. The grammar is validated against three different families of computing models: skeleton-based, general purpose, and domain-specific. In conjunction with the protocol, the grammar produces fully instrumented code for an application suite using the skeletal framework FastFlow.

Guest Editorial: High-Level Parallel Programming with Algorithmic Skeletons

Guest Editorial: High-Level Parallel Programming with Algorithmic Skeletons

The Dynamo package for tomography and subtomogram averaging: components for MATLAB, GPU computing and EC2 Amazon Web Services.

Dynamo is a package for the processing of tomographic data. As a tool for subtomogram averaging, it includes different alignment and classification strategies. Furthermore, its data-management module allows experiments to be organized in groups of tomograms, while offering specialized three-dimensional tomographic browsers that facilitate visualization, location of regions of interest, modelling and particle extraction in complex geometries. Here, a technical description of the package is presented, focusing on its diverse strategies for optimizing computing performance. Dynamo is built upon mbtools (middle layer toolbox), a general-purpose MATLAB library for object-oriented scientific programming specifically developed to underpin Dynamo but usable as an independent tool. Its structure intertwines a flexible MATLAB codebase with precompiled C++ functions that carry the burden of numerically intensive operations. The package can be delivered as a precompiled standalone ready for execution without a MATLAB license. Multicore parallelization on a single node is directly inherited from the high-level parallelization engine provided for MATLAB, automatically imparting a balanced workload among the threads in computationally intense tasks such as alignment and classification, but also in logistic-oriented tasks such as tomogram binning and particle extraction. Dynamo supports the use of graphical processing units (GPUs), yielding considerable speedup factors both for native Dynamo procedures (such as the numerically intensive subtomogram alignment) and procedures defined by the user through its MATLAB-based GPU library for three-dimensional operations. Cloud-based virtual computing environments supplied with a pre-installed version of Dynamo can be publicly accessed through the Amazon Elastic Compute Cloud (EC2), enabling users to rent GPU computing time on a pay-as-you-go basis, thus avoiding upfront investments in hardware and longterm software maintenance.

High-throughput, low-loss, low-cost, and label-free cell separation using electrophysiology-activated cell enrichment

Currently, cell separation occurs almost exclusively by density gradient methods and by fluorescence- and magnetic-activated cell sorting (FACS/MACS). These variously suffer from lack of specificity, high cell loss, use of labels, and high capital/operating cost. We present a dielectrophoresis (DEP)-based cell-separation method, using 3D electrodes on a low-cost disposable chip; one cell type is allowed to pass through the chip whereas the other is retained and subsequently recovered. The method advances usability and throughput of DEP separation by orders of magnitude in throughput, efficiency, purity, recovery (cells arriving in the correct output fraction), cell losses (those which are unaccounted for at the end of the separation), and cost. The system was evaluated using three example separations: live and dead yeast; human cancer cells/red blood cells; and rodent fibroblasts/red blood cells. A single-pass protocol can enrich cells with cell recovery of up to 91.3% at over 300,000 cells per second with >3% cell loss. A two-pass protocol can process 300,000,000 cells in under 30 min, with cell recovery of up to 96.4% and cell losses below 5%, an effective processing rate >160,000 cells per second. A three-step protocol is shown to be effective for removal of 99.1% of RBCs spiked with 1% cancer cells while maintaining a processing rate of ∼170,000 cells per second. Furthermore, the self-contained and low-cost nature of the separator device means that it has potential application in low-contamination applications such as cell therapies, where good manufacturing practice compatibility is of paramount importance.

Partially collapsed parallel Gibbs sampler for Dirichlet process mixture models

Dirichlet Process (DP) is commonly used as a non-parametric prior on mixture models. It has adaptive model selection capability which is useful in clustering applications. Although exact inference is not tractable for this prior, Markov Chain Monte Carlo (MCMC) samplers have been used to approximate the target posterior distribution. These samplers often do not scale well. Thus, recent studies focused on improving run-time efficiency through parallelization. In this paper, we introduce a new sampling method for DP by combining Chinese Restaurant Process (CRP) with stick-breaking construction allowing for parallelization through conditional independence at the data point level. Stick breaking part uses an uncollapsed sampler providing a high level of parallelization while the CRP part uses collapsed sampler allowing more accurate clustering. We show that this partially collapsed Gibbs sampler has significant advantages over the collapsed-only version in terms of scalability. We also provide results on real-world data sets that favorably compares the proposed inference algorithm against a recently introduced parallel Dirichlet Process samplers in terms of F1 scores while maintaining a comparable run-time performance.

A Library for Portable and Composable Data Locality Optimizations for NUMA Systems

Many recent multiprocessor systems are realized with a nonuniform memory architecture (NUMA) and accesses to remote memory locations take more time than local memory accesses. Optimizing NUMA memory system performance is difficult and costly for three principal reasons: (1) Today’s programming languages/libraries have no explicit support for NUMA systems, (2) NUMA optimizations are not portable, and (3) optimizations are not composable (i.e., they can become ineffective or worsen performance in environments that support composable parallel software). This article presents TBB-NUMA, a parallel programming library based on Intel Threading Building Blocks (TBB) that supports portable and composable NUMA-aware programming. TBB-NUMA provides a model of task affinity that captures a programmer’s insights on mapping tasks to resources. NUMA-awareness affects all layers of the library (i.e., resource management, task scheduling, and high-level parallel algorithm templates) and requires close coupling between all these layers. Optimizations implemented with TBB-NUMA (for a set of standard benchmark programs) result in up to 44% performance improvement over standard TBB. But more important, optimized programs are portable across different NUMA architectures and preserve data locality also when composed with other parallel computations sharing the same resource management layer.

Comparative Analysis of Present and Future Space-Grade Processors with Device Metrics

Due to harsh and inaccessible operating environments, space computing presents many unique challenges and constraints that limit onboard computing performance. However, the increasing need for real-time sensor and autonomous processing, coupled with limited communication bandwidth with ground stations, is increasing onboard computing demands for next-generation space missions. Because currently available space-grade processors cannot satisfy this growing demand, research into various processors is conducted to ensure that potential new processors are based upon architectures that will best meet the computing needs of space missions. Device metrics are used to measure and compare the theoretical capabilities of processors based upon vendor-provided data and tools, enabling the study of large and diverse sets of architectures. Architectural tradeoffs are determined that can be considered when comparing or designing space-grade processors. Results demonstrate how onboard computing capabilities are increasing due to emerging architectures that support high levels of parallelism in terms of computational units, internal memories, and input/output resources; and that performance varies between applications, depending on the compute-intensive kernels used. Furthermore, the overheads incurred by radiation hardening are quantified and used to analyze low-power commercial-off-the-shelf processors for potential hardening and use in future space missions.

Design and implementation of communication patterns using parallel objects

Within an environment of parallel objects, an approach of structured parallel programming with the paradigm of object-orientation is presented here. The proposal includes a programming method based on high level parallel compositions or HLPCs (CPANs in Spanish). C++ classes and CPANs are syntactically alike and differ in concurrency mechanisms. Different parallel programming patterns, synchronisation operations and new constructs like futures have been discussed throughout the paper. To achieve software-reusability, a series of predefined patterns that use object-oriented programming concepts have been presented. Concurrency related constraints on process synchronisation are set by only resorting to maxpar, mutex, sync primitives in the application code. By means of the method application, the implementation of commonly used parallel communication patterns is explained to finally present a library of classes for C++ applications that use POSIX threads.

Speckle Reduction with Trained Nonlinear Diffusion Filtering

Speckle reduction is a prerequisite for many image processing tasks in synthetic aperture radar images, as well as all coherent images. In recent years, predominant state-of-the-art approaches for despeckling are usually based on nonlocal methods which mainly concentrate on achieving utmost image restoration quality, with relatively low computational efficiency. Therefore, in this study we aim to propose an efficient despeckling model with both high computational efficiency and high recovery quality. To this end, we exploit a newly developed trainable nonlinear reaction diffusion (TNRD) framework which has proven a simple and effective model for various image restoration problems. In the original TNRD applications, the diffusion network is usually derived based on the direct gradient descent scheme. However, this approach will encounter some problem for the task of multiplicative noise reduction exploited in this study. To solve this problem, we employed a new architecture derived from the proximal gradient descent method. Taking into account the speckle noise statistics, the diffusion process for the despeckling task is derived. We then retrain all the model parameters in the presence of speckle noise. Finally, optimized nonlinear diffusion filtering models are obtained, which are specialized for despeckling with various noise levels. Experimental results substantiate that the trained filtering models provide comparable or even better results than state-of-the-art nonlocal approaches. Meanwhile, our proposed model merely contains convolution of linear filters with an image, which offers high-level parallelism on GPUs. As a consequence, for images of size $$512 \times 512$$ , our GPU implementation takes less than 0.1 s to produce state-of-the-art despeckling performance.

A modified nanoelectronic spiking neuron model

Spiking neural networks (SNNs) first came to the attention of scientists due to the search for a structure capable of emulating more closely the behavior of the human brain. The biological nervous system has some characteristics that allow it to process a large amount of data very quickly. It is also a fault-tolerant system, with a high level of parallelism. Low power consumption is another feature of the human brain that is desirable for electronic circuits. In this context, several models of artificial spiking neurons were developed, aiming to construct networks able to combine the best characteristics of the human brain. Most of these models, however, lack validation in larger networks. This paper proposes the implementation of an SNN based on a nanoelectronic spiking neuron model developed in previous works. To validate the behavior of an isolated neuron in a network, logic gates (NOT, OR, AND, and XOR) are used as a benchmark. The goal of this paper is to present a feasibility study on the possibility of implementing such nanoelectronic spiking neuron networks based on this spiking neuron model. Nanoelectronics represents an appealing implementation due to the gains regarding occupied area and power consumption, which are inherent characteristics of this technology. The neuron model was modified for simulation at room temperature. An information code based on the amplitude of the pulses presented at the output of the neuron was developed. During deployment of this approach, some limitations regarding the neuron model were detected; some possible solutions are proposed as future work.

Guest Editorial for Special Section on Big Data Computing and Processing in Computational Biology and Bioinformatics

The papers in this special section focus on big data computing in the field of bioinformatics and biocomputing. Big data has emerged as an important application field which has shown its huge impact in different scientific research domains. In particular, the big data bioinformatics applications such as DNA sequence analysis have posed significant challenges to the state-of-the-art processing and computing systems. With the growing explosive data scale, the collection, storage, retrieval, processing, scheduling, and visualization are key big data issues to be tackled. Up to now, many researchers have been seeking high-level parallelism using novel big data computing architectures and processing mechanisms.