Parallel Programming Paradigms Research Articles

StarPlat: A versatile DSL for graph analytics

Graphs model several real-world phenomena. With the growth of unstructured and semi-structured data, parallelization of graph algorithms is inevitable. Unfortunately, due to inherent irregularity of computation, memory access, and communication, graph algorithms are traditionally challenging to parallelize. To tame this challenge, several libraries, frameworks, and domain-specific languages (DSLs) have been proposed to reduce the parallel programming burden of the users, who are often domain experts. However, existing frameworks to model graph algorithms typically target a single architecture. In this paper, we present a graph DSL, named StarPlat, that allows programmers to specify graph algorithms in a high-level format, but generates code for three different backends from the same algorithmic specification. In particular, the DSL compiler generates OpenMP for multi-core systems, MPI for distributed systems, and CUDA for many-core GPUs. Since these three are completely different parallel programming paradigms, binding them together under the same language is challenging. We share our experience with the language design. Central to our compiler is an intermediate representation which allows a common representation of the high-level program, from which individual backend code generations begin. We demonstrate the expressiveness of StarPlat by specifying four graph algorithms: betweenness centrality computation, page rank computation, single-source shortest paths, and triangle counting. Using a suite of ten large graphs, we illustrate the effectiveness of our approach by comparing the performance of the generated codes with that obtained with hand-crafted library codes. We find that the generated code is competitive to library-based codes in many cases. More importantly, we show the feasibility to generate efficient codes for different target architectures from the same algorithmic specification of graph algorithms.

LLM4VV: Developing LLM-driven testsuite for compiler validation

Large language models (LLMs) are a new and powerful tool for a wide span of applications involving natural language and demonstrate impressive code generation abilities. The goal of this work is to automatically generate tests and use these tests to validate and verify compiler implementations of a directive-based parallel programming paradigm, OpenACC. To do so, in this paper, we explore the capabilities of state-of-the-art LLMs, including open-source LLMs - Meta’s Codellama, Phind’s fine-tuned version of Codellama, Deepseek’s Deepseek Coder and closed-source LLMs - OpenAI’s GPT-3.5-Turbo and GPT-4-Turbo. We further fine-tune the open-source LLMs and GPT-3.5-Turbo using our own testsuite dataset along with using the OpenACC specification. We also explored these LLMs using various prompt engineering techniques that include code template, template with retrieval-augmented generation (RAG), one-shot example, one-shot with RAG, expressive prompt with code template and RAG. This paper highlights our findings from over 5000 tests generated via all the above mentioned methods. Our contributions include: (a) exploring the capabilities of the latest and relevant LLMs for code generation, (b) investigating fine-tuning and prompt methods, and (c) analyzing the outcome of LLMs generated tests including manually analysis of representative set of tests. We found the LLM Deepseek-Coder-33b-Instruct produced the most passing tests followed by GPT-4-Turbo.

Introducing Multi-Threaded Programming in Parallel Programming Process for Optimal Performance Results

Within the world of parallel programming, the quest for optimal performance results is an ongoing challenge. As modern computer systems continue to evolve with an increasing number of processor cores, the need for efficient parallelization becomes paramount. Multi-threaded programming emerges as a powerful tool in this context, offering a means to harness the full potential of multi-core processors. This essay explores the integration of multi-threaded programming techniques within the parallel programming paradigm to achieve the most efficient performance results. It will examine the principles of multi-threaded programming, its advantages, and challenges, and provide insights into best practices for its effective utilization that illustrate the benefits and complexities of such an approach.

A new parallel tabu search algorithm for the optimization of the maximum vertex weight clique problem

SummaryThe efficiency of metaheuristic algorithms depends significantly on the number of fitness value evaluations performed on candidate solutions. In addition to various intelligent techniques used to obtain better results, parallelization of calculations can substantially improve the solutions in cases where the problem is NP‐hard and requires many evaluations. This study proposes a new parallel tabu search method for solving the Maximum Vertex Weight Clique Problem (MVWCP) on the Non‐Uniform Memory Access (NUMA) architectures using the OpenMP parallel programming paradigm. Achieving scalability in the NUMA architectures presents significant challenges due to the high complexity of their memory systems, which can lead to performance loss. However, our proposed Tabu‐NUMA algorithm provides up to speed‐up with 64 cores for ten basic problem instances in DIMACS‐W and BHOSLIB‐W benchmarks. And it improves the performance of the serial Multi Neighborhood Tabu Search (MN/TS) algorithm for 38 problem instances in DIMACS‐W and BHOSLIB‐W benchmarks. We further evaluate our algorithm on larger datasets with thousands of edges and vertices from Network Data Repository benchmark problem instances, and we report significant improvements in terms of speed up. Our results confirm that the Tabu‐NUMA algorithm is among the best recent algorithms for solving MVWCP on the NUMA architectures.

Traveler: Navigating Task Parallel Traces for Performance Analysis.

Understanding the behavior of software in execution is a key step in identifying and fixing performance issues. This is especially important in high performance computing contexts where even minor performance tweaks can translate into large savings in terms of computational resource use. To aid performance analysis, developers may collect an execution trace-a chronological log of program activity during execution. As traces represent the full history, developers can discover a wide array of possibly previously unknown performance issues, making them an important artifact for exploratory performance analysis. However, interactive trace visualization is difficult due to issues of data size and complexity of meaning. Traces represent nanosecond-level events across many parallel processes, meaning the collected data is often large and difficult to explore. The rise of asynchronous task parallel programming paradigms complicates the relation between events and their probable cause. To address these challenges, we conduct a continuing design study in collaboration with high performance computing researchers. We develop diverse and hierarchical ways to navigate and represent execution trace data in support of their trace analysis tasks. Through an iterative design process, we developed Traveler, an integrated visualization platform for task parallel traces. Traveler provides multiple linked interfaces to help navigate trace data from multiple contexts. We evaluate the utility of Traveler through feedback from users and a case study, finding that integrating multiple modes of navigation in our design supported performance analysis tasks and led to the discovery of previously unknown behavior in a distributed array library.

Programming big data analysis: principles and solutions

In the age of the Internet of Things and social media platforms, huge amounts of digital data are generated by and collected from many sources, including sensors, mobile devices, wearable trackers and security cameras. This data, commonly referred to as Big Data, is challenging current storage, processing, and analysis capabilities. New models, languages, systems and algorithms continue to be developed to effectively collect, store, analyze and learn from Big Data. Most of the recent surveys provide a global analysis of the tools that are used in the main phases of Big Data management (generation, acquisition, storage, querying and visualization of data). Differently, this work analyzes and reviews parallel and distributed paradigms, languages and systems used today to analyze and learn from Big Data on scalable computers. In particular, we provide an in-depth analysis of the properties of the main parallel programming paradigms (MapReduce, workflow, BSP, message passing, and SQL-like) and, through programming examples, we describe the most used systems for Big Data analysis (e.g., Hadoop, Spark, and Storm). Furthermore, we discuss and compare the different systems by highlighting the main features of each of them, their diffusion (community of developers and users) and the main advantages and disadvantages of using them to implement Big Data analysis applications. The final goal of this work is to help designers and developers in identifying and selecting the best/appropriate programming solution based on their skills, hardware availability, application domains and purposes, and also considering the support provided by the developer community.

EVOLUTION OF PROGRAMMING PARADIGMS

The subject of computer programming has witnessed a wonderful journey over a long time, characterized by the continuous evolution of programming paradigms. This overview paper provides an in-depth analysis of the dynamic and difficult method through which programming paradigms have advanced from their inception to their cutting-edge kingdom. The review commences with an ancient perspective, tracing the origins of programming paradigms from the early days of machine code and meeting language to the development of higher-degree languages. It examines the vital paradigm, which turned into most important throughout the early years of computing, and how it gave an upward push to dependent programming and modular programming. A pivotal shift occurred with the arrival of the object-orientated paradigm, which added a brand new way of considering software layout and improvement. This paradigm's impact on cutting-edge software engineering is explored, highlighting its lasting impact on industries and the open-source movement. As software complexity multiplied, so did the need for greater expressive and abstract programming paradigms, main to the emergence of practical programming and its mathematical foundations. The overview discusses how practical programming languages have received traction in recent years and how features drastically impacted parallel and dispensed computing. Furthermore, the paper explores concurrent and parallel programming paradigms, illustrating how they address the developing call for efficient use of multi-middle processors and disbursed structures. It delves into the challenges and possibilities presented through the increasing significance of parallelism in modern-day computing. The review also discusses rising paradigms such as reactive and eventdriven programming, highlighting their programs in actual-time systems, internet improvement, and the Internet of Things. These paradigms are analyzed in the context of modern software program improvement and the everincreasing technology panorama.

Prediction of multicore CPU performance through parallel data mining on public datasets

In the present scenario, high-performance computing needs more attention towards multicore computing. While designing the CPU, we need to consider hardware for processing speed, cache bandwidth, minimum memory requirements, etc. So for the selection of the best combination of CPU data mining tools may play an important role. In this era, data mining attracts more interest on parallel computing to enhance the performance of multicores. This paper demonstrates a parallel strategy similar to the traditional parallel programming paradigm to improve the performance of multicores by using a data mining approach. We have selected one approach EM with Gaussian and analyze the impact of its parallel execution on selected multicore clusters obtained through data mining. We have also evaluated our finding with a virtual environment of having 32 different families of CPUs, showing a speedup of up to ≈1.02x. This paper considers data clusters, cache mapping techniques and ranking of MPI programming techniques.

Message Passing Applications: A Review

Message Passing Applications: A Re It is known that message passing has become one of the most popular parallel programming paradigms because of its ease of use, so it was necessary to know or study the applications that were adopt message passing in their work. Programming models are for the most part classified by how memory is utilized. In the shared memory model, each cycle gets to a shared location space, yet in the message passing model, an application runs as an assortment of self-ruling cycles, each with its own local memory. The principle preferences of setting up a message-passing standard are convey ability and convenience. In a circulated memory correspondence climate in which the more significant level schedules as well as reflections are based upon lower level message-passing schedules the benefits of normalization are especially evident. Moreover, the usage of a message passing, for example, that proposed here, gives sellers a plainly dined base arrangement of schedules that they can actualize in days of yore, or at times for which they can give equipment uphold, consequently improving adaptability. view

On the implementation of flux limiters in algebraic frameworks

The use of flux limiters is widespread within the scientific computing community to capture shock discontinuities and are of paramount importance for the temporal integration of high-speed aerodynamics, multiphase flows and hyperbolic equations in general.Meanwhile, the breakthrough of new computing architectures and the hybridization of supercomputer systems pose a huge portability challenge, particularly for legacy codes, since the computing subroutines that form the algorithms, the so-called kernels, must be adapted to various complex parallel programming paradigms. From this perspective, the development of innovative implementations relying on a minimalist set of kernels simplifies the deployment of scientific computing software on state-of-the-art supercomputers, while it requires the reformulation of algorithms, such as the aforementioned flux limiters.Equipped with basic algebraic topology and graph theory underlying the classical mesh concept, a new flux limiter formulation is presented based on the adoption of algebraic data structures and kernels. As a result, traditional flux limiters are cast into a stream of only two types of computing kernels: sparse matrix-vector multiplication and generalized pointwise binary operators. The newly proposed formulation eases the deployment of such a numerical technique in massively parallel, potentially hybrid, computing systems and is demonstrated for a canonical advection problem.

Low-Power FPGA Architecture Based Monitoring Applications in Precision Agriculture

Today’s on-chip systems technology has grounded impressive advances in computing power and energy consumption. The choice of the right architecture depends on the application. In our case, we were studying vegetation monitoring algorithms in precision agriculture. This study presents a system based on a monitoring algorithm for agricultural fields, an electronic architecture based on a CPU-FPGA SoC system and the OpenCL parallel programming paradigm. We focused our study on our own dataset of agricultural fields to validate the results. The fields studied in our case are in the Guelmin-Oued noun region in the south of Morocco. These fields are divided into two areas, with a total surface of 3.44 Ha2 for the first field and 3.73 Ha2 for the second. The images were collected using a DJI-type unmanned aerial vehicle and an RGB camera. Performance evaluation showed that the system could process up to 86 fps versus 12 fps or 20 fps in C/C++ and OpenMP implementations, respectively. Software optimizations have increased the performance to 107 fps, which meets real-time constraints.

OpenSBLI: Automated code-generation for heterogeneous computing architectures applied to compressible fluid dynamics on structured grids

OpenSBLI is an open-source code-generation system for compressible fluid dynamics (CFD) on heterogeneous computing architectures. Written in Python, OpenSBLI is an explicit high-order finite-difference solver on structured curvilinear meshes. Shock-capturing is performed by a choice of high-order Weighted Essentially Non-Oscillatory (WENO) or Targeted Essentially Non-Oscillatory (TENO) schemes. OpenSBLI generates a complete CFD solver in the Oxford Parallel Structured (OPS) domain specific language. The OPS library is embedded in C code, enabling massively-parallel execution of the code on a variety of high-performance-computing architectures, including GPUs. The present paper presents a code base that has been completely rewritten from the earlier proof of concept Jacobs et al. (2017) [7], allowing shock capturing, coordinate transformations for complex geometries, and a wide range of boundary conditions, including solid walls with and without heat transfer. A suite of validation and verification cases are presented, plus demonstration of a large-scale Direct Numerical Simulation (DNS) of a transitional Shockwave Boundary Layer Interaction (SBLI). The code is shown to have good weak and strong scaling on multi-GPU clusters. We demonstrate that code-generation and domain specific languages are suitable for performing efficient large-scale simulations of complex fluid flows on emerging computing architectures. Program summaryProgram Title: OpenSBLI code-generation framework for compressible fluid dynamics on heterogeneous architecturesCPC Library link to program files:https://doi.org/10.17632/3sdb6hck2c.1Developer's repository link:https://github.com/opensbli/opensbliLicensing provisions: GPLv3Programming languages: Python, C/C++, OPS DSLNature of problem: The compressible 3D Navier-Stokes equations are solved via Implicit Large Eddy Simulation (ILES) or Direct Numerical Simulation (DNS).Solution method: OpenSBLI [1,2] is a Python-based code-generation system that uses symbolic algebra to generate a complete CFD solver in C/C++. The basic algorithm is a stencil-based finite-difference solver on structured curvilinear meshes. Shock-capturing is performed by a selection of high-order Weighted/Targeted Essentially Non-Oscillatory (WENO/TENO) schemes. Explicit low-storage Runge-Kutta schemes are used for time-advancement.Additional comments including restrictions and unusual features: The generated code is compliant with the Oxford Parallel Structured (OPS) [3] software library. OpenSBLI/OPS executables can be generated for the OpenMP, MPI, CUDA, OpenCL, and OpenACC parallel programming paradigms. Multi-GPU support is available via combinations of MPI with CUDA, OpenCL or OpenACC.

K-Athena: A Performance Portable Structured Grid Finite Volume Magnetohydrodynamics Code

Large scale simulations are a key pillar of modern research and require ever-increasing computational resources. Different novel manycore architectures have emerged in recent years on the way towards the exascale era. Performance portability is required to prevent repeated non-trivial refactoring of a code for different architectures. We combine ATHENA++, an existing magnetohydrodynamics (MHD) CPU code, with KOKKOS, a performance portable on-node parallel programming paradigm, into K-ATHENA to allow efficient simulations on multiple architectures using a single codebase. We present profiling and scaling results for different platforms including Intel Skylake CPUs, Intel Xeon Phis, and NVIDIA GPUs. K-ATHENA achieves > 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">8</sup> cell-updates/s on a single V100 GPU for second-order double precision MHD calculations, and a speedup of 30 on up to 24 576 GPUs on Summit (compared to 172,032 CPU cores), reaching 1:94 × 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">12</sup> total cell-updates/s at 76 percent parallel efficiency. Using a roofline analysis we demonstrate that the overall performance is currently limited by DRAM bandwidth and calculate a performance portability metric of 62.8 percent. Finally, we present the implementation strategies used and the challenges encountered in maximizing performance. This will provide other research groups with a straightforward approach to prepare their own codes for the exascale era. K-ATHENA is available at https://gitlab.com/pgrete/kathena.

Telecom fraud detection with big data analytics

The rapid development in telecom has also led to an increase in fraud activities, which causes both revenue and reputation losses. For this reason, this paper proposes a new telecom fraud detection model based on behaviour deviations of users expressed through time-varying signatures. In line with the similarity of these deviations to known frauds, a suspect list has been created and reported to fraud experts for the final decision. The proposed model was developed with the MapReduce parallel programming paradigm, which provides simplicity and flexibility for large-scale applications. Finally, the model was applied on call detail records of a telecom company. The obtained results have shown that the proposed approach detects the telecom frauds with 86% success and is suitable for application into a fraud management system for real-world implementation.

A Generalized and Parallelized SSIM-Based Multilevel Thresholding Algorithm

ABSTRACTMultilevel thresholding is a widely used technique to perform image segmentation. It consists of dividing an input image into several distinct regions by finding the optimal thresholds according to a certain objective function. In this work, we generalize the use of the SSIM quality measure as an objective function to solve the multilevel thresholding problem using empirically tuned swarm intelligence algorithms. The experimental study we have conducted shows that our approach, producing near-exact solutions, is more effective compared to the state-of-the-art methods. Moreover, we show that the computation complexity has been significantly reduced by adopting a shared-memory parallel programming paradigm for all the algorithms we have implemented.

Improving hardware transactional memory parallelization of computational geometry algorithms using privatizing transactions

Hardware transactional memory is a new parallel programming paradigm supported by current commercial multiprocessors. This paradigm provides optimistic concurrency and overcomes some of the problems associated with classical lock-based synchronization, such as deadlock and serialization. Certain algorithms of computational geometry are found to be good candidates for parallelization with this paradigm. However, hardware transactional approaches to these algorithms lead to poor performance as the resulting transactions are too large for the underlying hardware to deal with. Large transactions overflow hardware resources serializing the execution.In this paper, we propose using privatizing transactions to parallelize two computational geometry algorithms: Lee’s algorithm, which solves the shortest-route problem, and Ruppert’s algorithm for Delaunay/Voronoi mesh refinement. Privatizing transactions are based on commercial hardware transactional memory extensions, and their goal is to reduce transaction footprint by means of a non-transactional private execution section. This results in effective smaller transactions. Our implementation is able to further reduce the transaction size as we propose a reduced validation set for privatizing transactions. Programming complexity of these implementations is discussed.Results show that our privatizing transaction implementations indeed enhance performance comparing with existing hardware transactional memory versions. Experiments with Intel’s transactional memory extensions yield speedups ranging from 2× to 3.5× with four threads.

Recognition and Pose Estimation of Auto Parts for an Autonomous Spray Painting Robot

The autonomous operation of industrial robots with minimal human supervision has always been in high demand. To prepare the autonomous operation of a car part spray painting robot, novel object detection, and pose estimation algorithms have been developed in this paper. The object detection part used principal components analysis (PCA) to reduce the dimension of three-dimensional (3-D) point cloud to 2-D binary image. Distance measure between the auto and cross correlation of the binary features was established to find out the similarity between them. Resultantly, the type of auto part was successfully obtained. Furthermore, iterative closest point (ICP) algorithm was used to estimate the pose difference of the auto part with respect to the camera reference frame, which was mounted on the robot. An issue with ICP's lack of robustness to local minimum was solved by the combination of ICP and genetic algorithm (GA). This allowed the optimization of pose error and addressed the problem of local minimum entrapment in ICP. For experimental validation: the proposed object recognition pipeline was implemented in both serial and parallel programming paradigms. The results were obtained for the acquired point clouds of side body car parts and compared with the major 3-D object detection systems in terms of computational cost. Pose estimation error was calculated with both ICP and the modified point set registration schemes, and it was shown to be decreasing in the case of later. All shown results supported the research claims.

Detección de anomalías en grandes volúmenes de datos

El desarrollo de la era digital ha traído como consecuencia un incremento considerable de los volúmenes de datos. A estos grandes volúmenes de datos se les ha denominado big data ya que exceden la capacidad de procesamiento de sistemas de bases de datos convencionales. Diversos sectores consideran varias oportunidades y aplicaciones en la detección de anomalías en problemas de big data. Para realizar este tipo de análisis puede resultar muy útil el empleo de técnicas de minería de datos porque permiten extraer patrones y relaciones desde grandes cantidades de datos. El procesamiento y análisis de estos volúmenes de datos, necesitan de herramientas capaces de procesarlos como Apache Spark y Hadoop. Estas herramientas no cuentan con algoritmos específicos para la detección de anomalías. El objetivo del trabajo es presentar un nuevo algoritmo para la detección de anomalías basado en vecindad para de problemas big data. A partir de un estudio comparativo se seleccionó el algoritmo KNNW por sus resultados, con el fin de diseñar una variante big data. La implementación del algoritmo big data se realizó en la herramienta Apache Spark, utilizando el paradigma de programación paralela MapReduce. Posteriormente se realizaron diferentes experimentos para analizar el comportamiento del algoritmo con distintas configuraciones. Dentro de los experimentos se compararon los tiempos de ejecución y calidad de los resultados entre la variante secuencial y la variante big data. La variante big data obtuvo mejores resultados con diferencia significativa. Logrando que la variante big data, KNNW-BigData, pueda procesar grandes volúmenes de datos.

Accelerated 2D FWI using the symmetry on inner product spaces

Full Waveform Inversion (FWI) is a common technique used in the oil and gas industry due to its capabilities to estimate subsurface characteristics such as material’s density and sound velocity with high resolution. The 2D time domain FWI method involves the modeling of the forward wavefield of the source and the backpropagated field of the difference between the modeled and observed data. Therefore, due to its high computational cost in terms of RAM consumption and execution time, the High Performance Computing (HPC) field is very useful to deal with these problems. There are computational state-of-the-art solutions that allow to increase the execution time such as the parallel programming paradigm that involves the use of multicore processor systems. Furthermore, there are mathematical solutions leveraging on the properties of the algorithm used that make it possible to enhance performance of the method. We propose in this paper a new way to compute the FWI gradient, by taking advantage of an inner product property. Additionally, a computational strategy is combined with this proposal in the inversion scheme, thus improving FWI performance.

The benefits of concurrent computing in tribology system design

Abstrect Many optimum designs of tribological components are highly time-constrained before final productions. It is well-known that the process of a complex simulated design can be considerably accelerated by using some form of parallel computing. Also, for many tribological models additional assumptions can be relaxed with stricter design constrains, if the execution can be speeded up. In this study, the concurrent computing for tribological design is proposed, which is to perform parallel computing using the multitasking capability of today's operating system. In the concurrent computing a master program, which manages the process of the optimization, is used to launch a number of standalone slave programs (air bearing models) in a quick succession. And the operating system (MS-Windows) of the computer manages the parallel execution of the slave programs. Other than the standard programming language (Fortran 95) this approach uses none of the general parallel programming paradigms or directives, such as message passing interface, OpenMP, or coding using graphics processing units. In this study, the algorithm for the multiobjective optimization is group inching fortification method and the concurrent computing is executed in the algorithm-level. High parallel computing speedups are obtained in the simulated bearing designs. The approach can also be applied in using commercial general-purpose packages for modelling and self-coded methods for optimum design of tribological components or systems.