Fast Parallel Implementation Research Articles

GRAPE for fast and scalable graph processing and random-walk-based embedding

Graph representation learning methods opened new avenues for addressing complex, real-world problems represented by graphs. However, many graphs used in these applications comprise millions of nodes and billions of edges and are beyond the capabilities of current methods and software implementations. We present GRAPE (Graph Representation Learning, Prediction and Evaluation), a software resource for graph processing and embedding that is able to scale with big graphs by using specialized and smart data structures, algorithms, and a fast parallel implementation of random-walk-based methods. Compared with state-of-the-art software resources, GRAPE shows an improvement of orders of magnitude in empirical space and time complexity, as well as competitive edge- and node-label prediction performance. GRAPE comprises approximately 1.7 million well-documented lines of Python and Rust code and provides 69 node-embedding methods, 25 inference models, a collection of efficient graph-processing utilities, and over 80,000 graphs from the literature and other sources. Standardized interfaces allow a seamless integration of third-party libraries, while ready-to-use and modular pipelines permit an easy-to-use evaluation of graph-representation-learning methods, therefore also positioning GRAPE as a software resource that performs a fair comparison between methods and libraries for graph processing and embedding.

Mdapy: A flexible and efficient analysis software for molecular dynamics simulations

The mdapy is a library for pre- and postprocessing molecular dynamics simulation data. Benefitting from the just-in-time compile technology of TaiChi mdapy can be written in pure Python while possessing similar speed to those written in C++. mdapy is designed with highly paralleled and makes full advantages of modern computer resources on both multicore CPU and GPU architecture. The package implements a fast module to find the neighbors of particles in both free and periodic boundaries, based on which it offers a wide variety of methods to analyze atomic environments, such as standard centrosymmetry parameters, radial distribution function and newer methods, such as atomic entropy fingerprint. In addition, mdapy can be used to create the geometric structure of polycrystals with metallic or graphene grain boundaries by Voronoi diagram. mdapy can directly read the DUMP and DATA format defined in LAMMPS code, and, in practice, it accepts any other format by converting it into NumPy ndarray format. This design philosophy enables seamless integration with abundant scientific ecosystems in the Python community and easy cooperation with other analysis codes like OVITO or freud. Program summaryProgram Title: mdapyCPC Library link to program files:https://doi.org/10.17632/dtdkxvcsc9.1Developer's repository link:https://github.com/mushroomfire/mdapyCode Ocean capsule:https://codeocean.com/capsule/5271472Licensing provisions: BSD 3-clauseProgramming language: Python, C++Nature of problem: Atomic environment analysis and generation of the initial structure are important in molecular dynamics simulations. Many analysis methods relying on particle neighbors, such as radial distribution functions and atomic entropy, are computationally intensive and need to be carefully implemented to scale to large systems. Traditional code is often written in C++ or Fortran to guarantee performance while causing difficulty in installation and secondary development due to the complexity of the programming language.Solution method: mdapy is written in parallel to quickly perform neighbor finding, provides a set of analysis methods and creates an atomic geometry structure on multicore CPU and GPU. Over 95 percent of mdapy is written in Python with a uniform API to call. All data are stored in NumPy ndarray, making users easy to install, use and secondary develop.Additional comments including restrictions and unusual features:1. mdapy provides fast parallel implementations of neighbor finding in periodic and free boundary system.2. mdapy can generate a polycrystalline model with graphene grain boundaries.3. mdapy is helpful for analyzing the melting process by the mean squared displacement and Lindemann index at the atomic level.4. mdapy is very easy to install via pip install mdapy without any compile steps and can be run on Windows, Linux and Mac OS with Python 3.7-3.10.5. mdapy has detailed documentation to make it easier to use.

Fast parallel implementation for total variation constrained algebraic reconstruction technique.

In computed tomography (CT), the total variation (TV) constrained algebraic reconstruction technique (ART) can obtain better reconstruction quality when the projection data are sparse and noisy. However, the ART-TV algorithm remains time-consuming since it requires large numbers of iterations, especially for the reconstruction of high-resolution images. In this work, we propose a fast algorithm to calculate the system matrix for line intersection model and apply this algorithm to perform the forward-projection and back-projection operations of the ART. Then, we utilize the parallel computing techniques of multithreading and graphics processing units (GPU) to accelerate the ART iteration and the TV minimization, respectively. Numerical experiments show that our proposed parallel implementation approach is very efficient and accurate. For the reconstruction of a 2048 × 2048 image from 180 projection views of 2048 detector bins, it takes about 2.2 seconds to perform one iteration of the ART-TV algorithm using our proposed approach on a ten-core platform. Experimental results demonstrate that our new approach achieves a speedup of 23 times over the conventional single-threaded CPU implementation that using the Siddon algorithm.

A fast and concise parallel implementation of the 8x8 2D forward and inverse DCTs using halide

The Discrete Cosine Transform (DCT) is commonly used for image and video coding and very efficient implementations of the forward and inverse transforms are of great importance. The popular libjpeg-turbo library contains handwritten, highly-optimised assembly language DCT implementations utilizing SIMD instruction sets for a variety of architectures. We present an alternative approach, implementing the 8x8 IDCT and FDCT written in the functional image processing language Halide. We show how less than 200 lines of Halide can replace over 20,000 lines of code the libjpeg-turbo library to perform JPEG encoding and decoding. The Halide implementation is compared for ARMv8 NEON and x86-64 SIMD extensions and shows a 5–25 percent performance improvement over the SIMD code in libjpeg-turbo for decoding and a 10–40 percent improvement for encoding. The Halide code is significantly easier to maintain and port to new architectures than the existing code.

Local time stepping for the shallow water equations in MPAS

We assess the performance of a set of local time-stepping (LTS) schemes for the shallow water equations implemented in the Model for Prediction Across Scales (MPAS). The goal of LTS is to speed up the simulation by allowing different time-steps on different regions of the computational grid. The LTS schemes considered here were originally introduced by Hoang et al. (2019) [26], who laid out the mathematical foundation of the methods. Here, the authors take on the task of presenting a fast, efficient and scalable parallel implementation of these LTS methods on high performance computing machines, with the aim to provide a recipe for other climate modeling groups that may be interested in employing LTS algorithms in their codes. As a matter of fact, even if MPAS is our framework of choice, our approach is general enough and could be of interest to other groups beyond the MPAS community. Due to their nature, LTS methods possess an inherent load imbalance that needs to be carefully addressed in order to obtain efficient scalability. Even more important is the far from trivial task of computing the right-hand side terms only on specific LTS regions during the time-stepping procedure. An inefficient handling of this task causes a drastic decay of the CPU time performance, making the LTS algorithms practically of no use. The emphasis of the present work is therefore on the computational and parallel aspects of the LTS methods, whose proper treatment is crucial to make the methods run faster against existing strategies, such as for instance high-order explicit global time-stepping schemes. This is in fact the ultimate goal of using an LTS procedure and it is the one to which we direct all our optimization efforts.

Classifying Interictal Epileptiform Activities in Intracranial EEG Using Complex-Valued Convolutional Neural Network

The goal of our study is to develop a fast parallel implementation of group independent component analysis (ICA) for functional magnetic resonance imaging (fMRI) data using graphics processing units (GPU). Though ICA has become a standard method to identify brain functional connectivity of the fMRI data, it is computationally intensive, especially has a huge cost for the group data analysis. GPU with higher parallel computation power and lower cost are used for general purpose computing, which could contribute to fMRI data analysis significantly. In this study, a parallel group ICA (PGICA) on GPU, mainly consisting of GPU-based PCA using SVD and Infomax-ICA, is presented. In comparison to the serial group ICA, the proposed method demonstrated both significant speedup with 6–11 times and comparable accuracy of functional networks in our experiments. This proposed method is expected to perform the real-time post-processing for fMRI data analysis.

Freud: A software suite for high throughput analysis of particle simulation data

The freud Python package is a library for analyzing simulation data. Written with modern simulation and data analysis workflows in mind, freud provides a Python interface to fast, parallelized C++ routines that run efficiently on laptops, workstations, and supercomputing clusters. The package provides the core tools for finding particle neighbors in periodic systems, and offers a uniform API to a wide variety of methods implemented using these tools. As such, freud users can access standard methods such as the radial distribution function as well as newer, more specialized methods such as the potential of mean force and torque and local crystal environment analysis with equal ease. Rather than providing its own trajectory data structure, freud operates either directly on NumPy arrays or on trajectory data structures provided by other Python packages. This design allows freud to transparently interface with many trajectory file formats by leveraging the file parsing abilities of other trajectory management tools. By remaining agnostic to its data source, freud is suitable for analyzing any particle simulation, regardless of the original data representation or simulation method. When used for on-the-fly analysis in conjunction with scriptable simulation software such as HOOMD-blue, freud enables smart simulations that adapt to the current state of the system, allowing users to study phenomena such as nucleation and growth. Program summaryProgram Title:freudProgram Files doi:http://dx.doi.org/10.17632/v7wmv9xcct.1Licensing provisions: BSD 3-ClauseProgramming language: Python, C++Nature of problem: Simulations of coarse-grained, nano-scale, and colloidal particle systems typically require analyses specialized to a particular system. Certain more standardized techniques – including correlation functions, order parameters, and clustering – are computationally intensive tasks that must be carefully implemented to scale to the larger systems common in modern simulations.Solution method:freud performs a wide variety of particle system analyses, offering a Python API that interfaces with many other tools in computational molecular sciences via NumPy array inputs and outputs. The algorithms in freud leverage parallelized C++ to scale to large systems and enable real-time analysis. The library’s broad set of features encode few assumptions compared to other analysis packages, enabling analysis of a broader class of data ranging from biomolecular simulations to colloidal experiments.Additional comments including restrictions and unusual features:1. freud provides very fast parallel implementations of standard analysis methods like RDFs and correlation functions.2. freud includes the reference implementation for the potential of mean force and torque (PMFT).3. freud provides various novel methods for characterizing particle environments, including the calculation of descriptors useful for machine learning. The source code is hosted on GitHub (https://github.com/glotzerlab/freud), and documentation is available online (https://freud.readthedocs.io/). The package may be installed via pip install freud-analysis or conda install -c conda-forge freud.

High-Dimensional Metric Combining for Non-Coherent Molecular Signal Detection

In emerging Internet-of-Nano-Thing (IoNT), information will be embedded and conveyed in the form of molecules through complex and diffusive medias. One main challenge lies in the long-tail nature of the channel response causing inter-symbol-interference (ISI), which deteriorates the detection performance. If the channel is unknown, we cannot easily achieve traditional coherent channel estimation and cancellation, and the impact of ISI will be more severe. In this paper, we develop a novel high-dimensional non-coherent scheme for blind detection of molecular signals. We achieve this in a higher-dimensional metric space by combining different non-coherent metrics that exploit the transient features of the signals. By deducing the theoretical bit error rate (BER) for any constructed high-dimensional non-coherent metric, we prove that, higher dimensionality always achieves a lower BER in the same sample space. Then, we design a generalised blind detection algorithm that utilizes the Parzen approximation and its probabilistic neural network (Parzen-PNN) to detect information bits. Taking advantages of its fast convergence and parallel implementation, our proposed scheme can meet the needs of detection accuracy and real-time computing. Numerical simulations demonstrate that our proposed scheme can gain 10dB BER compared with other state of the art methods.

Fast Parallel Implementation of Dual-Camera Compressive Hyperspectral Imaging System

Coded aperture snapshot spectral imager (CASSI) provides a potential solution to recover the 3D hyperspectral image (HSI) from a single 2D measurement. The latest proposed design of the dual-camera compressive hyperspectral imager (DCCHI) can collect more information simultaneously with the CASSI to improve the reconstruction quality. The main bottleneck now lies in the high computation complexity of the reconstruction methods, which hinders the practical application. In this paper, we propose a fast parallel implementation based on DCCHI to reach a stable and efficient HSI reconstruction. Specifically, we develop a new optimization method for the reconstruction problem, which integrates the alternative direction multiplier method with the total variation-based regularization to boost the convergence rate. Then, to improve the time efficiency, a novel parallel implementation based on GPU is proposed. The performance of the proposed method is validated on both synthetic and real data. The experimental results demonstrate that our method has a significant advantage in time efficiency, while maintaining a comparable reconstruction fidelity.

CADISHI: Fast parallel calculation of particle-pair distance histograms on CPUs and GPUs

We report on the design, implementation, optimization, and performance of the CADISHI software package, which calculates histograms of pair-distances of ensembles of particles on CPUs and GPUs. These histograms represent 2-point spatial correlation functions and are routinely calculated from simulations of soft and condensed matter, where they are referred to as radial distribution functions, and in the analysis of the spatial distributions of galaxies and galaxy clusters. Although conceptually simple, the calculation of radial distribution functions via distance binning requires the evaluation of O(N2) particle-pair distances where N is the number of particles under consideration. CADISHI provides fast parallel implementations of the distance histogram algorithm for the CPU and the GPU, written in templated C++ and CUDA. Orthorhombic and general triclinic periodic boxes are supported, in addition to the non-periodic case. The CPU kernels feature cache blocking, vectorization and thread-parallelization to obtain high performance. The GPU kernels are tuned to exploit the memory and processor features of current GPUs, demonstrating histogramming rates of up to a factor 40 higher than on a high-end multi-core CPU. To enable high-throughput analyses of molecular dynamics trajectories, the compute kernels are driven by the Python-based CADISHI engine. It implements a producer–consumer data processing pattern and thereby enables the complete utilization of all the CPU and GPU resources available on a specific computer, independent of special libraries such as MPI, covering commodity systems up to high-end high-performance computing nodes. Data input and output are performed efficiently via HDF5. In addition, our CPU and GPU kernels can be compiled into a standard C library and used with any application, independent from the CADISHI engine or Python. The CADISHI software is freely available under the MIT license. Program summaryProgram Title: CADISHIProgram Files doi:http://dx.doi.org/10.17632/82b8sdft79.1Licensing provisions: MITProgramming language: C++, CUDA, PythonNature of problem: Radial distribution functions are of fundamental importance in soft and condensed matter physics and astrophysics. However, the calculation of distance histograms scales quadratically with the particle number. To be able to analyze large data sets, fast and efficient implementations of distance histogramming are crucial.Solution method: CADISHI provides parallel, highly optimized implementations of distance histogramming. On the CPU, high performance is achieved via an advanced cache blocking scheme in combination with vectorization and threading. On the GPU, the problem is decomposed via a tiling scheme to exploit the GPU’s massively parallel architecture and hierarchy of global, constant, and shared memory efficiently, resulting in significant speedups compared to the CPU. Moreover, CADISHI exploits all the resources (GPUs, CPUs) available on a compute node in parallel.Additional comments including restrictions and unusual features: Additionally to the non-periodic case CADISHI implements the minimum image convention for orthorhombic and general triclinic periodic boxes. We provide Python interfaces and the option to compile the kernels into a plain C library.

Parallel implementation of hyper-dimensional dynamical particle system on CUDA

The presented paper deals with possible approaches to parallel implementation of solution of a hyper-dimensional dynamical particle system. The proposed implementation approaches are generally applicable for similar particle systems of interest in various research and engineering fields. The original motivation for the present work was a simulation of particles that represent a space-filling design to be optimized for further use in design of experiments. Due to the underlying purpose of this particle system, the dimension of the particle system of interest is considered to be entirely arbitrary. Such a hyper-dimensional space is further folded into a periodically repeated domain.The theoretical background of the proposed particle system is provided along with the derivation of equations of motion of the dynamical system. As the complexity of the system is not limited by the number of particles nor the number of dimensions, the possibilities of utilizing the GPGPU platform are more restricted in comparison with today’s fast parallel implementations of common particle systems.Two distinct approaches to parallel implementation are presented, one aiming at a generalized usage of the fast on-chip resources, the other entirely relying on the GPU’s on-board global memory. Despite unambiguous mutual differences in their performance, both parallel implementations deliver major speedup compared to the single-thread CPU solution as well as a better scaling of execution time when increasing both the number of particles and dimensions.

Parallel Modularity Clustering

In this paper we develop a parallel approach for computing the modularity clustering often used to identify and analyse communities in social networks. We show that modularity can be approximated by looking at the largest eigenpairs of the weighted graph adjacency matrix that has been perturbed by a rank one update. Also, we generalize this formulation to identify multiple clusters at once. We develop a fast parallel implementation for it that takes advantage of the Lanczos eigenvalue solver and k-means algorithm on the GPU. Finally, we highlight the performance and quality of our approach versus existing state-of-the-art techniques.

Embarrassingly Parallel Acceleration of Global Tractography via Dynamic Domain Partitioning.

Global tractography estimates brain connectivity by organizing signal-generating fiber segments in an optimal configuration that best describes the measured diffusion-weighted data, promising better stability than local greedy methods with respect to imaging noise. However, global tractography is computationally very demanding and requires computation times that are often prohibitive for clinical applications. We present here a reformulation of the global tractography algorithm for fast parallel implementation amendable to acceleration using multi-core CPUs and general-purpose GPUs. Our method is motivated by the key observation that each fiber segment is affected by a limited spatial neighborhood. In other words, a fiber segment is influenced only by the fiber segments that are (or can potentially be) connected to its two ends and also by the diffusion-weighted signal in its proximity. This observation makes it possible to parallelize the Markov chain Monte Carlo (MCMC) algorithm used in the global tractography algorithm so that concurrent updating of independent fiber segments can be carried out. Experiments show that the proposed algorithm can significantly speed up global tractography, while at the same time maintain or even improve tractography performance.

Minimum divergence viscous flow simulation through finite difference and regularization techniques

We develop a new algorithm to simulate single- and two-phase viscous flow through a three-dimensional Cartesian representation of the porous space, such as those available through X-ray microtomography. We use the finite difference method to discretize the governing equations and also propose a new method to enforce the incompressible flow constraint under zero Neumann boundary conditions for the velocity components. Finite difference formulation leads to fast parallel implementation through linear solvers for sparse matrices, allowing relatively fast simulations, while regularization techniques used on solving inverse problems lead to the desired incompressible fluid flow.Tests performed using benchmark samples show good agreement with experimental/theoretical values. Additional tests are run on Bentheimer and Buff Berea sandstone samples with available laboratory measurements. We compare the results from our new method, based on finite differences, with an open source finite volume implementation as well as experimental results, specifically to evaluate the benefits and drawbacks of each method. Finally, we calculate relative permeability by using this modified finite difference technique together with a level set based algorithm for multi-phase fluid distribution in the pore space. To our knowledge this is the first time regularization techniques are used in combination with finite difference fluid flow simulations.

Accelerating Global Tractography Using Parallel Markov Chain Monte Carlo.

Global tractography estimates brain connectivity by determining the optimal configuration of signal-generating fiber segments that best describes the measured diffusion-weighted data, promising better stability than local greedy methods with respect to imaging noise. However, global tractography is computationally very demanding and requires computation times that are often prohibitive for clinical applications. We present here a reformulation of the global tractography algorithm for fast parallel implementation amendable to acceleration using multi-core CPUs and general-purpose GPUs. Our method is motivated by the key observation that each fiber segment is affected by a limited spatial neighborhood. That is, a fiber segment is influenced only by the fiber segments that are (or can potentially be) connected to its both ends and also by the diffusion-weighted signal in its proximity. This observation makes it possible to parallelize the Markov chain Monte Carlo (MCMC) algorithm used in the global tractography algorithm so that updating of independent fiber segments can be done concurrently. The experiments show that the proposed algorithm can significantly speed up global tractography, while at the same time maintain or improve tractography performance.

A Fast Parallel Implementation of a PTAS for Fractional Packing and Covering Linear Programs

We present a parallel implementation of the randomized $$(1+\varepsilon )$$(1+?) approximation algorithm for packing and covering linear programs presented by Koufogiannakis and Young (2007). Their approach builds on ideas of the sublinear time algorithm of Grigoriadis and Khachiyan's (Oper Res Lett 18(2):53---58, 1995) and Garg and Konemann's (SIAM J Comput 37(2):630---652, 2007) non-uniform-increment amortization scheme. With high probability it computes a feasible primal and dual solution whose costs are within a factor of $$1+\varepsilon $$1+? of the optimal cost. In order to make their algorithm more parallelizable we also implemented a deterministic version of the algorithm, i.e. instead of updating a single random entry at each iteration we updated deterministically many entries at once. This slowed down a single iteration of the algorithm but allowed for larger step-sizes which lead to fewer iterations. We use NVIDIA's parallel computing architecture CUDA for the parallel environment. We report a speedup between one and two orders of magnitude over the times reported by Koufogiannakis and Young (2007).

GPU-based parallel group ICA for functional magnetic resonance data

The goal of our study is to develop a fast parallel implementation of group independent component analysis (ICA) for functional magnetic resonance imaging (fMRI) data using graphics processing units (GPU). Though ICA has become a standard method to identify brain functional connectivity of the fMRI data, it is computationally intensive, especially has a huge cost for the group data analysis. GPU with higher parallel computation power and lower cost are used for general purpose computing, which could contribute to fMRI data analysis significantly. In this study, a parallel group ICA (PGICA) on GPU, mainly consisting of GPU-based PCA using SVD and Infomax-ICA, is presented. In comparison to the serial group ICA, the proposed method demonstrated both significant speedup with 6–11 times and comparable accuracy of functional networks in our experiments. This proposed method is expected to perform the real-time post-processing for fMRI data analysis.

Minerals detection for hyperspectral images using adapted linear unmixing: LinMin

Minerals detection over large volume of spectra is the challenge addressed by current hyperspectral imaging spectrometer in Planetary Science. Instruments such OMEGA (Mars Express), CRISM (Mars Reconnaissance Orbiter), M3 (Chandrayaan-1), VIRTIS (Rosetta) and many more, have been producing very large datasets since one decade. We propose here a fast supervised detection algorithm called LinMin, in the framework of linear unmixing, with innovative arrangement in order to treat non-linear cases due to radiative transfer in both atmosphere and surface. We use reference laboratory and synthetic spectral library. Additional spectra are used in order to mimic the effect of Martian aerosols, grain size, and observation geometry discrepancies between reference and observed spectra. The proposed algorithm estimates the uncertainty on “mixing coefficient” from the uncertainty of observed spectra. Both numerical and observational tests validate the approach. Fast parallel implementation of the best algorithm (IPLS) on Graphics Processing Units (GPU) allows to significantly reduce the computation cost by a factor of ∼40.

On Parallel Implementation of Horn and Schunck Motion Estimation Method

The aim of this study is to present a fast parallel implementation of the Horn and Schunck method using a new kind of recurrent neural network called discrete Zhang neural networks. This network is characterized by a few iterations to converge which make it very suitable for real-time motion estimation. To compute the optical flow, we propose to solve directly the system of equations utilizing the discrete Zhang neural networks for matrix inversion instead of the original iterative method. The simulation results for synthetic and real image sequences show that the proposed algorithm is faster than the Horn and Schunck method.

3-D Adaptive Nonlinear Complex-Diffusion Despeckling Filter

This work aims to improve the process of speckle noise reduction while preserving edges and other relevant features through filter expansion from 2-D to 3-D. Despeckling is very important for data visual inspection and as a preprocessing step for other algorithms, as they are usually notably influenced by speckle noise. To that intent, a 3-D approach is proposed for the adaptive complex-diffusion filter. This 3-D iterative filter was applied to spectral-domain optical coherence tomography medical imaging volumes of the human retina and a quantitative evaluation of the results was performed to allow a demonstration of the better performance of the 3-D over the 2-D filtering and to choose the best total diffusion time. In addition, we propose a fast graphical processing unit parallel implementation so that the filter can be used in a clinical setting.