Parallel GPU Implementation Research Articles

A parallel recursive framework for modelling time series

Abstract Time series modelling is of significance to several scientific fields. Several approaches based on statistics, machine learning or combinations have been utilized. In order to model and forecast time series a novel parallel framework based on recursive pseudoinverse matrices is proposed. This framework enables the design of arbitrary statistical and machine learning models, adaptively, from a set of potential basis functions. This unification enables compact definition of existing and new models as well as easy implementation for new massively parallel architectures. The choice of appropriate basis functions is analysed and the fitting accuracy, termination criteria and model update operations are presented. A block variant for multivariate time series is also proposed. Parallel GPU implementation and performance optimization of the framework are provided, based on mixed precision arithmetic and matrix operations. The use of different basis functions is showcased with respect to various model univariate and multivariate time series for applications such as regression, frequency estimation and automatic trend detection. Discussions on limitations and future directions of research are also provided.

CPU and GPU parallel efficiency of ARM based single board computing cluster for CFD applications

This paper examines the parallel efficiency of an ARM-based single board computing cluster made of 16 Raspberry Pi 4 and 8 Nvidia Jetson Nano Single Board Computer, considering both CPU and GPU parallel implementation of CFD applications. It is found that the parallelization scales up to 16 Raspberry Pi 4 and 8 Nvidia Jetson Nano (maximum available on the current cluster). Moreover, it is shown that regarding the computation time, about 12 SBC are as fast as a classical computing workstation. Finally, it is shown that such cluster are energy efficient considering CFD applications.

GPU Implementation of the Improved CEEMDAN Algorithm for Fast and Efficient EEG Time-Frequency Analysis.

Time-frequency analysis of EEG data is a key step in exploring the internal activities of the human brain. Studying oscillations is an important part of the analysis, as they are thought to provide the underlying mechanism for communication between neural assemblies. Traditional methods of analysis, such as Short-Time FFT and Wavelet Transforms, are not ideal for this task due to the time-frequency uncertainty principle and their reliance on predefined basis functions. Empirical Mode Decomposition and its variants are more suited to this task as they are able to extract the instantaneous frequency and phase information but are too time consuming for practical use. Our aim was to design and develop a massively parallel and performance-optimized GPU implementation of the Improved Complete Ensemble EMD with the Adaptive Noise (CEEMDAN) algorithm that significantly reduces the computational time (from hours to seconds) of such analysis. The resulting GPU program, which is publicly available, was validated against a MATLAB reference implementation and reached over a 260× speedup for actual EEG measurement data, and provided predicted speedups in the range of 3000-8300× for longer measurements when sufficient memory was available. The significance of our research is that this implementation can enable researchers to perform EMD-based EEG analysis routinely, even for high-density EEG measurements. The program is suitable for execution on desktop, cloud, and supercomputer systems and can be the starting point for future large-scale multi-GPU implementations.

LibEMM: A fictious wave domain 3D CSEM modelling library bridging sequential and parallel GPU implementation

This paper delivers a software - libEMM - for 3D controlled-source electromagnetics (CSEM) modelling in fictitious wave domain, based on the newly developed high-order finite-difference time-domain (FDTD) method on non-uniform grid. The numerical simulation can be carried out over a number of parallel processors using MPI-based high performance computing architecture. The FDTD kernel coded in C has been parallelized with OpenMP for speedup using local shared memory. In addition, the software features a GPU implementation of the same algorithm based on CUDA programming language, which can be cross-validated and compared in terms of efficiency. A perspective of libEMM on the horizon is its application to 3D CSEM inversion in land and marine environment. Program summaryProgram Title: libEMMCPC Library link to program files:https://doi.org/10.17632/p769t7c5bk.1Developer's repository link:https://github.com/yangpl/libEMMLicensing provisions: GNU General Public License v3.0Programming language: C, CUDA, Fortran, ShellExternal dependencies: MPI [1], FFTW [2], CUDA [3]Nature of problem: Controlled-source electromagnetics (CSEM)Solution method: High-order finite-difference time-domain (FDTD) on non-uniform grid by fictious wave domain transformation References[1]https://www.mpich.org/[2]http://fftw.org/[3]https://developer.nvidia.com/cuda-toolkit

PERFORMANCE REFINEMENT OF CONVOLUTIONAL NEURAL NETWORK ARCHITECTURES FOR SOLVING BIG DATA PROBLEMS

Two of the most well-liked neural network frameworks, Theano and TensorFlow, will be compared in this study for how well they perform on a given problem. The MNIST database will be used for this specific problem, which is the recognition of handwritten digits from one to nine. It is a good idea to use more examples than contrasted ones to compare these frameworks because this database is the subject of active research at the moment and has produced excellent results. However, in order to be trained and deliver accurate results, neural networks need a sizeable amount of sample data, as will be covered in more detail later. Because of this, big data experts frequently encounter problems of this nature. As the project description implies, we won't just present a standard comparison because of this; instead, we'll work to present a comparison of these networks' performance in a Big Data environment using distributed computing. The FMNIST or Fashion MNIST database and CIFAR10 will also be tested (using the same neural network design), extending the scope of the comparison beyond MNIST. The same code will be used with the same structure thanks to the use of a higher-level library called Keras, which makes use of the aforementioned support (in our case, Theano or TensorFlow). There has been a surge in open-source parallel GPU implementation research and development as a result of the high computational cost of training CNNs on large data sets. However, there aren't many studies that have been done to assess the performance traits of those implementations. In this study, we compare these implementations carefully across a wide range of parameter configurations, look into potential performance bottlenecks, and pinpoint a number of areas that could use more fine-tuning.

A GPU parallel randomized CUR compression method for the Method of Moments

In this work, we propose a GPU parallel implementation of the randomized CUR (or Pseudo Skeleton) Approximation to compress the H-matrices of linear systems that arise in the discretization of integral equations modeling electromagnetic scattering problems. This compression method is highly parallelizable, in contrast with other similar methods such as the Adaptive Cross Approximation. It involves dense linear algebra computations that can be efficiently implemented on a GPU device. Besides, a stochastic convergence criterion is introduced to minimize the communication between the host and the device. Testing the code with standard cases shows the efficiency and accuracy of the method.

Scalable Katz Ranking Computation in Large Static and Dynamic Graphs

Network analysis defines a number of centrality measures to identify the most central nodes in a network. Fast computation of those measures is a major challenge in algorithmic network analysis. Aside from closeness and betweenness, Katz centrality is one of the established centrality measures. In this article, we consider the problem of computing rankings for Katz centrality. In particular, we propose upper and lower bounds on the Katz score of a given node. Previous approaches relied on numerical approximation or heuristics to compute Katz centrality rankings; however, we construct an algorithm that iteratively improves those upper and lower bounds until a correct Katz ranking is obtained. We extend our algorithm to dynamic graphs while maintaining its correctness guarantees. Experiments demonstrate that our static graph algorithm outperforms both numerical approaches and heuristics with speedups between \( 1.5\times \) and \( 3.5\times \) , depending on the desired quality guarantees. Our dynamic graph algorithm improves upon the static algorithm for update batches of less than 10,000 edges. We provide efficient parallel CPU and GPU implementations of our algorithms that enable near real-time Katz centrality computation for graphs with hundreds of millions of edges in fractions of seconds.

An efficient and locality-oriented Hausdorff distance algorithm: Proposal and analysis of paradigms and implementations

Hausdorff distance (HD) is a popular similarity metric used in the comparison of images or 3D volumes. Although popular, its main weakness is computing power consumption, being one of the slowest set distances. In this work, a novel, parallel and locality-oriented Hausdorff distance implementation is proposed. Novel as it is the first time in the literature that an actual algorithmic implementation using morphological dilations is proposed and thoroughly evaluated. Parallel, as it is more robust in terms of parallelization than the state-of-the-art algorithm and local as it has an intrinsic sensitivity to voxels that are closer in space. This proposal can be faster than the state-of-the-art in several practical cases such as in medical imaging registrations (up to 8 times faster on average in one of the CPU experiments) and is faster in the worst-case (up to 22337 times faster in one of the CPU experiments). Worst-case scenarios and high resolution volumes also favor the proposed approach. Throughout the work, several sequential and parallel CPU and GPU implementations are evaluated and compared.

Recursive filter based GPU algorithms in a Data Assimilation scenario

Data Assimilation process is generally used to estimate the best initial state of a system in order to improve accuracy of future states prediction. This powerful technique has been widely applied in investigations of the atmosphere, ocean, and land surface. In this work, we deal with the Gaussian convolution operation which is a central step of the Data Assimilation approach, as well as in several data analysis procedures. In particular, we consider the use of recursive filters to approximate the Gaussian convolution. In [1] we presented an accelerated first-order recursive filter to compute the Gaussian convolution kernel, in a very fast way. We present theory and results, and we provide a new GPU-parallel implementation which is based on the third order recursive filter. To observe the benefits in terms of performance, tests and experiments complete our work.

Efficient GPU implementation of the Particle-in-Cell/Monte-Carlo collisions method for 1D simulation of low-pressure capacitively coupled plasmas

In this paper, we describe an efficient, massively parallel GPU implementation strategy for speeding up one-dimensional electrostatic plasma simulations based on the Particle-in-Cell method with Monte-Carlo collisions. Relying on the Roofline performance model, we identify performance-critical points of the program and provide optimised solutions. We use four benchmark cases to verify the correctness of the CUDA and OpenCL implementations and analyse their performance properties on a number of NVIDIA and AMD cards. Plasma parameters computed with both GPU implementations differ not more than 2% from each other and respective literature reference data. Our final implementations reach over 2.6 Tflop/s sustained performance on a single card, and show speed up factors of up to 200 (when using 10 million particles). We demonstrate that GPUs can be very efficiently used for simulating collisional plasmas and argue that their further use will enable performing more accurate simulations in shorter time, increase research productivity and help in advancing the science of plasma simulation.

A particle packing parallel geometric method using GPU

The purpose of this paper is to present a methodology for obtaining granular models from a GPU parallel implementation of the geometric separation particle packing strategy. This methodology is suitable for the generation of large-scale granular models used in discontinuous media simulations. The proposed approach uses disk-shaped particles (two-dimensional approach) and parallelization mechanisms that consider different computational environments, with a focus on GPUs. The methodology is divided into three macro-steps: (a) definition of an input set of particles; (b) geometric separation; and (c) removal of spurious particles. The set of input particles uses data related to the particle size distribution and the domain filling rate, defining arbitrary positions for the particles. The other steps are used to eliminate overlaps between particles. Parallel computing is performed using the OpenCL programming API on compatible devices. Examples are presented to show the effectiveness of the proposed methodology. They show good time improvement and better memory efficiency in comparison with the original serial version of the strategy. The method is also validated by comparing the results with experimental and numerical data from the literature. The proposed methodology allows generating granular models with a parallel GPU particle packing method. It turns possible the achievement of bigger models in a smaller amount of time, without compromising the strategy efficiency and accuracy. It also presents mechanisms to avoid information exchange between GPU and CPU.

A new calibration of the Heston Stochastic Local Volatility Model and its parallel implementation on GPUs

In this article we propose a new more general calibration of the Heston Stochastic-Local Volatility (HSLV) model. More precisely, the main contribution is to perform the direct calibration of the whole set of parameters at the same time instead of the usual two steps procedure. Moreover, the proposed approach allows to use exotic options to calibrate the HSLV model, thus making it more flexible and general. However, as there are no analytical formulas available to price exotic options to calibrate the model, the cost function (the HSLV pricer) involved in the calibration process must be computed using Monte Carlo methods, thus leading to a highly demanding computational problem. Therefore, we also propose efficient parallel GPU implementations of Monte Carlo techniques for the pricers. Furthermore, for solving the resulting global optimization problem, we develop customized parallel multi-CPU implementations of two of the most common stochastic metaheuristic global optimization algorithms: Differential Evolution and Simulated Annealing. A comparison between both algorithms has been made. This second level of parallelization has been carried out by the implementation of the cost function as a single GPU kernel and keeping the OpenMP parallelization for the optimization algorithm, thus leading to a hybrid multi-GPU implementation of the calibrator. All these implementations have been tested with real market data for European and barrier options in the context of foreign exchange markets.

Toward an ultrasonic inspecting method to detect and classify adhesive bonding defects in real time: a numeric study

Adhesive bonding is an efficient method to join different components in structural design. However, a reliable nondestructive inspecting method to attest the integrity of adhesive bonds is still an open task. In the last few decades, many researchers have put effort into addressing this demand, and the methods based on ultrasound have emerged as the most promising ones. It is consensual that the capability of modeling both mathematically and computationally the interaction between ultrasonic waves and adhesive bonds will play a crucial role in the development of any ultrasonic inspecting method. In that sense, in a previous work, an algorithm to compute the scattering of ultrasonic waves by defective adhesive bonds was developed and implemented. In the present work, we revisit the algorithm and develop a novel GPU parallel implementation, aiming to reduce considerably the execution time. As shown, our new implementation has reduced the execution time by a factor of around 25, opening the possibility for solving the correlated inverse problem in real time. To the best of our knowledge, this is the first time in the literature that GPU is employed to solve this particular ultrasonic scattering problem.

Parallel calibration based on modified trim strategy

PurposePartial alignment for 3 D point sets is a challenging problem for laser calibration and robot calibration due to the unbalance of data sets, especially when the overlap of data sets is low. Geometric features can promote the accuracy of alignment. However, the corresponding feature extraction methods are time consuming. The purpose of this paper is to find a framework for partial alignment by an adaptive trimmed strategy.Design/methodology/approachFirst, the authors propose an adaptive trimmed strategy based on point feature histograms (PFH) coding. Second, they obtain an initial transformation based on this partition, which improves the accuracy of the normal direction weighted trimmed iterative closest point (ICP) method. Third, they conduct a series of GPU parallel implementations for time efficiency.FindingsThe initial partition based on PFH feature improves the accuracy of the partial registration significantly. Moreover, the parallel GPU algorithms accelerate the alignment process.Research limitations/implicationsThis study is applicable to rigid transformation so far. It could be extended to non-rigid transformation.Practical implicationsIn practice, point set alignment for calibration is a technique widely used in the fields of aircraft assembly, industry examination, simultaneous localization and mapping and surgery navigation.Social implicationsPoint set calibration is a building block in the field of intelligent manufacturing.Originality/valueThe contributions are as follows: first, the authors introduce a novel coarse alignment as an initial calibration by PFH descriptor similarity, which can be viewed as a coarse trimmed process by partitioning the data to the almost overlap part and the rest part; second, they reduce the computation time by GPU parallel coding during the acquisition of feature descriptor; finally, they use the weighted trimmed ICP method to refine the transformation.

GPU parallel implementation and optimisation of SAR target recognition method

GPU parallel implementation and optimisation of SAR target recognition method

GPGPU-accelerated environmental modelling based on the 2D advection-reaction-diffusion equation

The advection-reaction-diffusion equation is a rather general model that describes a variety of environmental processes, such as air pollution transport, aquifer recharge, groundwater contaminant transport and forest growth, among others. The resulting models are known to be computationally demanding, especially when the model covers large areas where in high resolution. In this context, the exponential increase in computational power of massively parallel computing devices, such as the General Purpose Graphics Processing Units (GPGPUs), has triggered a paradigm shift in scientific computing and environmental modelling. In this paper we present an environmental modelling solution based on a parallel implementation of the 2D advection-reaction-diffusion equation, tailored for GPGPU devices. The software is flexible, free, open-source and allows the modelling of a wide range of environmental phenomena, including those taking place in heterogeneous and anisotropic media. Performance tests show that the parallel GPU implementation can provide up to a 70-fold speedup in relation to an analogous CPU implementation.

Massively Parallel GPU Implementation of the Lattice-Boltzmann Method for the Simulation of Coupled Aero-Thermodynamic Systems

The automotive industry is showing high demand for efficient design of cooling systems in electric vehicles. The development of complex aero-thermodynamic systems requires reliable, high-fidelity simulations of high Reynolds number flows. We implemented a numerical solver based on the double-distribution lattice-Boltzmann method (LBM). The main advantage of the LBM, compared to the Navier-Stokes-based solvers, is its computational efficiency and intrinsic parallelism, which allows for execution on massively parallel architectures (GPUs). We have validated our GPU implementation by simulating a natural convection flow and a heated cylinder in an enclosed cavity. Both cases show very good agreement with published literature. In the future, we aim to extend the usage of the LBM framework to industry-relevant cases like the simulation of various packaging concepts for electric vehicles.

GPU implementation of RX detection using spectral derivative features

Hyperspectral image (HSI), which can record abundance information of a pixel, has shown huge potential on many applications such as image classification, target and anomaly detection and so on. Nowadays, anomaly detection has attracted more attention because there is no limitation of spectral library. A standard approach for anomaly detection is the method developed by Reed and Xiaoli, called RX algorithm. However, the data volume is getting bigger with the developing of imaging technology. A problem that ensues is the rapid increase of computation complexity and this will lead a time-consumed application. In addition, there will be noise in HSI with the influence of illumination and atmospheric. In this paper, we realize an implementation of RX algorithm on NVIDIA GeForce 1060 GPU with the utilization of derivative features. On one hand, the GPU parallel implementation reach the purpose of real-time processing and it also eliminates the storage burden of on-board processing. On the other hand, the derivative features have better performance on salient features detection and noise restraint. Thus, it can further promote the detection performance of RXD. In our experiments, three real HSI datasets were tested to verify the effect of GPU parallel implementation. The experiment results had indicated that the utilization of derivative features can promote the detection performance. Compared with serial computation, the parallel implementation achieves a great reduction on processing time.

Parallel SIMD CPU and GPU Implementations of Berlekamp–Massey Algorithm and Its Error Correction Application

The Berlekamp–Massey algorithm finds the shortest linear feedback shift register for a binary input sequence. A wide range of applications like cryptography and digital signal processing use this algorithm. This research proposes novel parallel mechanisms offered by heterogeneous CPU and GPU hardwares in order to achieve the best possible performance for BMA. The proposed bitwise implementation of the BMA algorithm is almost 35 times faster than state of the art implementations. This further improvement is achieved by using SIMD instructions which provides data level parallelism. This new approach can be 4.6 and 35 times faster than a bitwise CPU and state of the art implementations, respectively. In order to achieve the highest possible speedup over a multi-core structure, a multi-threading implementation is introduced in this research. By leveraging on OpenMP we were able to obtain a speedup of 10 times over 12 cores server. The GPU device with thousands of processing cores can bring great speedup over the best CPU implementation. Two other parallel mechanisms offered by GPU are concurrent kernel execution and streaming. They achieve 14.5 and 2.2 times of speedup compared to CPU serial and typical CUDA implementations, respectively. Also, the performance of the openMP code with SIMD instructions is compared with GPU stream implementation. The effectiveness of the proposed method is evaluated in a real world error correction application and it achieves 6.8 times of speedup.

Parallel rule‐based selective sampling and on‐demand learning to rank

SummaryLearning to rank (L2R) works by constructing a ranking model from training data so that, given a new query, the model is able to generate an effective rank of the objects for the query. Almost all work in L2R focus on ranking accuracy leaving performance and scalability overlooked. However, performance is a critical factor, especially when dealing with on‐demand queries. In this scenario, Learning to Rank using association rules has been shown to be extremely effective but only at a high computational cost. In this work, we show how to exploit parallelism on rule‐based systems to: i) drastically reduce L2R training datasets using selective sampling and ii) to generate query customized ranking models on the fly. We present parallel algorithms and GPU implementations for these two tasks showing that dataset reduction takes only a few seconds with speedups up to 148x over a serial baseline, and that queries can be processed in only a few milliseconds with speedups of 1000x over a serial baseline and 29x over a parallel baseline for the best case. We also extend the implementations to work with multiple GPUs, further increasing the speedup over the baselines and showing the scalability of our proposed algorithms.