Data-parallel Techniques Research Articles

Comparing open-source DEM frameworks for simulations of common bulk processes

Multiple software frameworks based on the Discrete Element Method (DEM) are available for simulating granular materials. All of them employ the same principles of explicit time integration, with each time step consisting of three main steps: contact detection, calculation of interactions, and integration of the equations of motion. However, there exist significant algorithmic differences, such as the choice of contact models, particle and wall shapes, and data analysis methods. Further differences can be observed in the practical implementation, including data structures, architecture, parallelization and domain decomposition techniques, user interaction, and the documentation of resources.This study compares, verifies, and benchmarks nine widely-used software frameworks. Only open-source packages were considered, as these are freely available and their underlying algorithms can be reviewed, edited, and tested. The benchmark consists of three common bulk processes: silo emptying, drum mixing, and particle impact. To keep it simple and comparable, only standard features were used, such as spherical particles and the Hertz-Mindlin model for dry contacts. Scripts for running the benchmarks in each software are provided as a dataset.

Comparative Study on Distributed Lightweight Deep Learning Models for Road Pothole Detection.

This paper delves into image detection based on distributed deep-learning techniques for intelligent traffic systems or self-driving cars. The accuracy and precision of neural networks deployed on edge devices (e.g., CCTV (closed-circuit television) for road surveillance) with small datasets may be compromised, leading to the misjudgment of targets. To address this challenge, TensorFlow and PyTorch were used to initialize various distributed model parallel and data parallel techniques. Despite the success of these techniques, communication constraints were observed along with certain speed issues. As a result, a hybrid pipeline was proposed, combining both dataset and model distribution through an all-reduced algorithm and NVlinks to prevent miscommunication among gradients. The proposed approach was tested on both an edge cluster and Google cluster environment, demonstrating superior performance compared to other test settings, with the quality of the bounding box detection system meeting expectations with increased reliability. Performance metrics, including total training time, images/second, cross-entropy loss, and total loss against the number of the epoch, were evaluated, revealing a robust competition between TensorFlow and PyTorch. The PyTorch environment's hybrid pipeline outperformed other test settings.

Improving scalability of Earth system models through coarse-grained component concurrency – a case study with the ICON v2.6.5 modelling system

Abstract. In the era of exascale computing, machines with unprecedented computing power are available. Making efficient use of these massively parallel machines, with millions of cores, presents a new challenge. Multi-level and multi-dimensional parallelism will be needed to meet this challenge. Coarse-grained component concurrency provides an additional parallelism dimension that complements typically used parallelization methods such as domain decomposition and loop-level shared-memory approaches. While these parallelization methods are data-parallel techniques, and they decompose the data space, component concurrency is a function-parallel technique, and it decomposes the algorithmic space. This additional dimension of parallelism allows us to extend scalability beyond the limits set by established parallelization techniques. It also offers a way to maintain performance (by using more compute power) when the model complexity is increased by adding components, such as biogeochemistry or ice sheet models. Furthermore, concurrency allows each component to run on different hardware, thus leveraging the usage of heterogeneous hardware configurations. In this work we study the characteristics of component concurrency and analyse its behaviour in a general context. The analysis shows that component concurrency increases the “parallel workload”, improving the scalability under certain conditions. These generic considerations are complemented by an analysis of a specific case, namely the coarse-grained concurrency in the multi-level parallelism context of two components of the ICON modelling system: the ICON ocean model ICON-O and the marine biogeochemistry model HAMOCC. The additional computational cost incurred by the biogeochemistry module is about 3 times that of the ICON-O ocean stand alone model, and data parallelization techniques (domain decomposition and loop-level shared-memory parallelization) present a scaling limit that impedes the computational performance of the combined ICON-O–HAMOCC model. Scaling experiments, with and without concurrency, show that component concurrency extends the scaling, in cases doubling the parallel efficiency. The experiments' scaling results are in agreement with the theoretical analysis.

Standart Dışı Değerlerin Tespitinde DBSCAN Algoritması ile Seri ve Paralel Programlama Performansının Karşılaştırılması

With the introduction of computers into our lives, digital data sizes are increasing gradually. Non-standard values (outliers) which behave differently from the others can be found in these data produced in the digital world. Detection of these values, especially in big data sets; has great importance in fields such as security, insurance, finance, medicine and genetics. Clustering methods of data mining techniques are frequently used in outlier detection in big data sets. Density based DBSCAN (Density-based spatial clustering of applications with noise) algorithm from clustering algorithms which are sensitive to noisy and outlier values is one of the most important methods in outlier detection. In this study, an application was developed using DBSCAN algorithm in C# programming language for the detection of outliers. In the developed application, 2 data sets with different data numbers were examined and analyzed. For the shortest possible data analysis time, serial and parallel programming techniques were used separately. In order to shorten the analysis time of big data sets, parallel class members in TPL (Task Parallel Library) provided with .Net 4.0 were used. In series of analysis of data sets, it was seen that DBSCAN algorithm produces more accurate results and is more practicable than other selected algorithms in terms of outlier detection. When considered in terms of computing performance, parallel programming has become more efficient as the number of data increases.

A hardware approach to value function iteration

This paper proposes a novel approach for the computation of dynamic stochastic equilibrium models. We design an FPGA specialized in the computation of a Bellman equation via value function iteration (VFI). Our hardware approach exhibits significant speed gains vis-à-vis GPU-based data-parallelization techniques. The speed gains arise from two layers of parallelism, accessible to hardware developers: instruction-level and pipeline parallelism at the logical resources level. By and large, the paper documents significant computational speed gains from hardware specialization, so far unexplored by the macroeconomic literature.

Scalable and Adaptive Fuzzy Grid Partitioning for Enhanced Rule Generation in Complex Decision-Making Systems

This research focuses on addressing the challenges of rule generation in complex decision-making systems by proposing a scalable and adaptive fuzzy grid partitioning approach. Traditional rule generation methods often struggle to handle large datasets and dynamic environments, leading to decreased accuracy and computational inefficiencies. In this study, we present a novel approach that integrates scalable and adaptive techniques to enhance the accuracy, efficiency, and interpretability of rule-based frameworks. The scalable fuzzy grid partitioning algorithm efficiently partitions the attribute space, allowing for the generation of rules in decision-making systems with a large number of data points. By incorporating data parallelization and dimensionality reduction techniques, the approach mitigates computational complexity while maintaining rule generation accuracy. Furthermore, the adaptive fuzzy grid partitioning algorithm dynamically adjusts the partitioning structure based on changing conditions, capturing evolving patterns and ensuring the relevancy and reliability of the generated rules over time. The generated rules are evaluated using fuzzy rule evaluation functions, which consider the degree of membership in the corresponding fuzzy grid cells. This evaluation process ranks and selects the rules based on their firing strengths, providing an interpretable decision-making framework for complex systems. The approach enhances the interpretability of the generated rules by capturing the uncertainties and complexities inherent in decision-making processes. To validate the effectiveness of the proposed approach, we conducted experiments using a credit risk assessment case example. The results demonstrate improved accuracy and efficiency compared to traditional rule generation methods. The generated rules offer transparency and insight into the factors influencing credit risk assessments, enabling informed decision-making. However, this research has some limitations, including potential dataset dependencies, the choice of fuzzy membership functions, computational complexity, and the need for further evaluation metrics and real-world implementation considerations. Future research should focus on addressing these limitations and exploring the applicability of the proposed approach in diverse domains. In conclusion, the scalable and adaptive fuzzy grid partitioning approach presented in this research offers a promising solution to the challenges of rule generation in complex decision-making systems. By addressing scalability, adaptability, and interpretability, this approach enhances the accuracy and efficiency of rule-based frameworks, paving the way for more effective decision support systems in various domains.

A Hardware Approach to Value Function Iteration

We propose a novel approach for the computation of dynamic stochastic equilibrium models. We design an FPGA specialized in the computation of a bellman equation via value function iteration (VFI). Our hardware approach documents significant speed gains vis-a-vis GPU-based data-parallelization techniques. The speed gains arise from two layers of parallelism, accessible to hardware developers: instruction-level and pipeline parallelism at logical resources level. By and large, the paper highlights significant computational speed gains from hardware specialization.

Language-based vectorization and parallelization using intrinsics, OpenMP, TBB and Cilk Plus

The aim of this paper is to evaluate OpenMP, TBB and Cilk Plus as basic language-based tools for simple and efficient parallelization of recursively defined computational problems and other problems that need both task and data parallelization techniques. We show how to use these models of parallel programming to transform a source code of Adaptive Simpson’s Integration to programs that can utilize multiple cores of modern processors. Using the example of Belman–Ford algorithm for solving single-source shortest path problems, we advise how to improve performance of data parallel algorithms by tuning data structures for better utilization of vector extensions of modern processors. Manual vectorization techniques based on Cilk array notation and intrinsics are presented. We also show how to simplify such optimization using Intel SIMD Data Layout Template containers.

Data Parallel Implementation of Belief Propagation in Factor Graphs on Multi-core Platforms

We investigate data parallel techniques for belief propagation in acyclic factor graphs on multi-core systems. Belief propagation is a key inference algorithm in factor graph, a probabilistic graphical model that has found applications in many domains. In this paper, we explore data parallelism for basic operations over the potential tables in belief propagation. Data parallel techniques for these table operations are developed for shared memory platforms. We then propose a complete belief propagation algorithm using these table operations to perform exact inference in factor graphs. The proposed algorithms are implemented on state-of-the-art multi-socket multi-core systems with additional NUMA-aware optimizations. Our proposed algorithms exhibit good scalability using a representative set of factor graphs. On a four-socket Intel Westmere-EX system with 40 cores, we achieve 39.5 $$\times $$ × speedup for the table operations and 39 $$\times $$ × speedup for the complete algorithm using factor graphs with large potential tables.

Data-parallel techniques for simulating a mega-scale agent-based model of systemic inflammatory response syndrome on graphics processing units

Agent-based modeling is increasingly being used for computer simulation of complex biological systems. An agent-based model (ABM) is a bottom-up simulation where the bulk dynamics of the model result from the local interactions of its individual constituents or agents. However, due to emergent qualities of ABMs, bulk behaviors may be sensitive to the size of the model as determined by the population of individuals. Therefore, in certain circumstances it may be critical to closely match the simulation size with the actual system. This may be particularly true in biological systems, where multiple large-scale heterogeneous populations can range into millions or even billions of individual cells/agents. Most existing ABM simulation toolkits are designed for serial computing and canno*t effectively simulate such mega-scale systems from a run-time standpoint. In this paper, we investigate data-parallel ABM implementations on graphics processing units to address the scalability issue of ABMs. As an example, we have implemented an abstracted version of the Systemic Inflammatory Response Syndrome ABM. We also implemented a serial version to confirm statistical accuracy. Our results show that parallelization on graphics processing units offers a substantial gain in performance without a loss in accuracy.

Graphics processing unit based direct simulation Monte Carlo

The direct simulation Monte Carlo (DSMC) is a computational method for fluid mechanics simulation in the regime of rarefied gas flow. It is a numerical solution of the Boltzmann equation based on an individual particle basis. Accurate simulations typically require particle numbers in the range of hundreds of thousands to millions. Such large simulations require an inordinate amount of time for processing using serial computing on central processing units (CPUs). In this paper we investigate data-parallel techniques on graphics processing units (GPUs) to execute very large scale DSMC simulations. We have designed and implemented Bird’s method on a three-dimensional simulation domain that includes complex geometry interactions. We also have tested and verified the statistical and theoretical accuracy of our implementation. Our results show substantial performance improvements (nearly two orders of magnitude) over Bird’s serial implementation without loss of accuracy.

Regular Lattice and Small-World Spin Model Simulations Using CUDA and GPUs

Data-parallel accelerator devices such as Graphical Processing Units (GPUs) are providing dramatic performance improvements over even multi-core CPUs for lattice-oriented applications in computational physics. Models such as the Ising and Potts models continue to play a role in investigating phase transitions on small-world and scale-free graph structures. These models are particularly well-suited to the performance gains possible using GPUs and relatively high-level device programming languages such as NVIDIA’s Compute Unified Device Architecture (CUDA). We report on algorithms and CUDA data-parallel programming techniques for implementing Metropolis Monte Carlo updates for the Ising model using bit-packing storage, and adjacency neighbour lists for various graph structures in addition to regular hypercubic lattices. We report on parallel performance gains and also memory and performance tradeoffs using GPU/CPU and algorithmic combinations.

Interactive machinability analysis of free-form surfaces using multiple-view image space techniques on the GPU

In this paper we present a set of graphics hardware accelerated algorithms to interactively evaluate the machinability of complex free-form surfaces. These algorithms work in image space and easily interface with all common formats available on CAD systems. The running time of these algorithms is independent of the complexity of the surface to be analyzed and depends only on the size of the projected image of the surface and the largest available tool head. Interactive speed is achieved through clever use of data-parallel techniques that map nicely onto the programming model of modern programmable graphics processing units. We demonstrate a method for pre-calculating and storing the machinability of a surface within a texture to further reduce rendering costs. The algorithms are implemented and tested on a complex set of parts and their performance has been analyzed.

Detached-Eddy Simulation Procedure Targeted for Design

A new detached-eddy simulation procedure, MBFLO, is described for the simulation of turbulent flow over/within arbitrary geometry. This new procedure represents the first step toward developing a turbulent technique that reduces the reliance on traditional turbulence modeling by directly solving for the larger-scale turbulence effects in regions where the computational grid is sufficient to resolve those scales. A goal of this effort was to develop a detached-eddy simulation procedure-for which the time-averaged solution can replace current steady simulation results for design. Because detached-eddy simulation requires the solution of the unsteady Navier-Stokes equations on computational grids that are typically an order in magnitude more dense than their Reynolds-averaged counterparts, special care was taken to make this new procedure computationally efficient. This paper describes the data structure, parallelization, and automation techniques used to produce time-averaged detached-eddy simulation predictions. Results for the turbulent flow through a turbine and compressor cascade at design and offdesign conditions, respectively, are shown to illustrate the technique.

Rendering large scenes using parallel ray tracing

Ray tracing is a powerful technique to generate realistic images of 3D scenes. However, rendering complex scenes may easily exceed the processing and memory capabilities of a single workstation. Distributed processing offers a solution if the algorithm can be parallelized in an efficient way. In this paper a hybrid scheduling approach is presented that combines demand driven and data parallel techniques. Which tasks to process demand driven and which data parallel, is decided by the data intensity of the task and the amount of data locality (coherence) that will be present in the task. By combining demand driven and data driven tasks, a better load balance may be achieved, while at the same time the communication is spread evenly across the network. This leads to a scalable and efficient parallel implementation of the ray tracing algorithm with little restriction on the size of the model data base to be rendered.